Intra prediction method and apparatus based on interpolation filtering, video encoding method and apparatus, video decoding method and apparatus, and system

MYPI2025005152A0Undetermined Publication Date: 2024-08-24GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
MYPI2025005152
Authority / Receiving Office
MY · MY
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-02-24
Publication Date
2024-08-24

AI Technical Summary

Technical Problem

Existing digital video compression technology still has room for improvement in reducing video transmission bandwidth and traffic pressure. Especially in the context of pursuing higher video definition, the coding performance of intra-frame prediction mode needs to be improved.

Method used

The intra prediction method based on interpolation filtering is used to determine the filter coefficient according to the filter window and the pixel reconstruction value of the adjacent reconstruction area, and perform point-by-point interpolation filtering on the pixels in the current block to obtain the prediction block, which is introduced in video coding. Interpolation filtering mode to optimize intra prediction.

Benefits of technology

It improves the performance of video encoding and reduces the cost of rate distortion through more accurate prediction values, thus improving the efficiency and quality of video transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure PI2025005152_DRAWING_001
    Figure PI2025005152_DRAWING_001
  • Figure PI2025005152_DRAWING_002
    Figure PI2025005152_DRAWING_002
  • Figure PI2025005152_DRAWING_003
    Figure PI2025005152_DRAWING_003
Patent Text Reader

Abstract

An intra frame prediction method and apparatus based on interpolation filtering, a video encoding method and apparatus, a video decoding method and apparatus, and a system. The intra prediction method based on interpolation filtering comprises: during the intra prediction process of video encoding and decoding, determining, according to a filtering window of the current block and reconstruction values of pixel points in a reconstruction region adjacent to the current block, a filtering coefficient which is used when interpolation filtering is performed on the current block; and performing point-by-point prediction on the pixel points in the current block according to the filtering window and the filtering coefficient, so as to obtain a predicted block of the current block. In the embodiments of the present disclosure, a predicted block of the current block is obtained using the current block and the spatial redundancy of a reconstruction region adjacent thereto, such that the encoding performance can be improved. (Fig. 3)
Need to check novelty before this filing date? Find Prior Art

Description

Intra-frame prediction, video encoding and decoding method, device and system based on interpolation filtering Technical Field

[0001] The embodiments of the present disclosure relate to, but are not limited to, video technology, and more specifically, to an intra-frame prediction and video encoding and decoding method, device, and system based on interpolation filtering. Background Art

[0002] Digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. Currently popular video codec standards, such as H.266 / Versatile Video Coding (VVC), employ a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of equal size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) based on a specific rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction, which are used to reduce or remove inherent redundancy in the video. Intra-frame blocks are predicted using neighboring pixels as reference, while inter-frame blocks use information from spatially adjacent blocks and reference information from other frames. In contrast to the prediction signal, the residual information is transformed, quantized, and entropy coded on a block-by-block basis to generate a bitstream. These techniques are described in standards and implemented in various fields related to video compression.

[0003] With the surge in Internet videos and people's increasing demand for video clarity, although existing digital video compression standards can save a lot of bandwidth, there is still a need to pursue better digital video compression technologies to reduce the bandwidth and traffic pressure of digital video transmission.

[0004] SUMMARY OF THE INVENTION

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] An embodiment of the present disclosure provides an intra-frame prediction method based on interpolation filtering, including:

[0007] Determine, based on the filter window of the current block and the reconstructed values ​​of the pixels in the reconstructed area adjacent to the current block, the filter coefficients used when performing interpolation filtering on the current block;

[0008] Performing point-by-point prediction on pixels in the current block according to the filter window and the filter coefficient to obtain a prediction block of the current block;

[0009] When predicting the current pixel, reference pixels adjacent to the current pixel are determined according to the filtering window, and interpolation filtering is performed based on the pixel values ​​of the reference pixels and the filtering coefficients to obtain a predicted value of the current pixel.

[0010] The present disclosure also provides a video decoding method, including:

[0011] Decode the interpolation filter mode flag of the current block;

[0012] When determining, based on the decoded interpolation filter mode flag, that the current block uses the interpolation filter mode for intra-frame prediction, determining the filter window and reconstruction area selected for the current block;

[0013] Based on the determined filtering window and reconstruction area, intra-frame prediction is performed on the current block according to the intra-frame prediction method based on interpolation filtering described in any embodiment of the present disclosure.

[0014] The present disclosure also provides a video encoding method, including:

[0015] Performing intra-frame prediction on the current block based on multiple intra-frame prediction modes, wherein, if it is determined that the current block allows the use of the interpolation filtering mode, performing intra-frame prediction on the current block in the interpolation filtering mode according to the intra-frame prediction method described in any embodiment of the present disclosure;

[0016] When it is determined according to the rate-distortion costs of the multiple intra prediction modes that the current block uses the interpolation filter mode for intra prediction, the interpolation filter mode flag of the current block is encoded to indicate that the current block uses the interpolation filter mode for intra prediction.

[0017] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated according to the video encoding method described in any embodiment of the present disclosure.

[0018] An embodiment of the present disclosure further provides an intra-frame prediction device based on interpolation filtering, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, it can implement the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure.

[0019] An embodiment of the present disclosure further provides a video decoding device, comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0020] An embodiment of the present disclosure further provides a video encoding device, including a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0021] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0022] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure, or implements the video decoding method as described in any embodiment of the present disclosure, or implements the video encoding method as described in any embodiment of the present disclosure.

[0023] An embodiment of the present disclosure further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it is capable of implementing the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure, or implementing the video decoding method as described in any embodiment of the present disclosure, or implementing the video encoding method as described in any embodiment of the present disclosure.

[0024] Summary of the Figures

[0025] The accompanying drawings are used to provide an understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure and do not constitute a limitation to the technical solutions of the present disclosure.

[0026] FIG1A is a schematic diagram of a coding and decoding system according to an embodiment of the present disclosure;

[0027] FIG1B is a framework diagram of an encoding end according to an embodiment of the present disclosure;

[0028] FIG1C is a framework diagram of a decoding end according to an embodiment of the present disclosure;

[0029] FIG2 is a schematic diagram of an intra-frame prediction mode according to an embodiment of the present disclosure;

[0030] FIG3 is a flowchart of an intra-frame prediction method based on interpolation filtering according to an embodiment of the present disclosure;

[0031] FIG4A is a schematic diagram of a filtering window used in an embodiment of the present disclosure;

[0032] FIG4B is a schematic diagram of an interpolation filter used in an embodiment of the present disclosure;

[0033] 5A, 5B, and 5C are schematic diagrams of the positional relationships between the filter window, the current block, and the reconstruction area when the current pixel is located at three different positions of the current block according to an embodiment of the present disclosure;

[0034] FIG6A , FIG6B , and FIG6C are schematic diagrams of three different spatial position relationships between a reference pixel and a current pixel according to an embodiment of the present disclosure;

[0035] 7A , 7B , and 7C are schematic diagrams of three different shapes of filter windows corresponding to FIG. 6A , FIG. 6B , and FIG. 6C , respectively, according to an embodiment of the present disclosure;

[0036] 8A , 8B , and 8C are schematic diagrams of an embodiment of the present disclosure in which the reconstruction areas adjacent to the current block are an upper reconstruction area, a left reconstruction area, and an L-shaped reconstruction area;

[0037] 9A and 9B are schematic diagrams of sample positions selected according to different step lengths when constructing the Wienerhof equation according to an embodiment of the present disclosure;

[0038] FIG10 is a schematic diagram of a partial reconstruction area used for averaging when an L-shaped reconstruction area is used in an embodiment of the present disclosure;

[0039] FIG11 is a flow chart of a video encoding method according to an embodiment of the present disclosure;

[0040] FIG12 is a flowchart of a video decoding method according to an embodiment of the present disclosure;

[0041] 13A to 13F are schematic diagrams of six different reconstruction areas used in embodiments of the present disclosure;

[0042] 14A and 14B are schematic diagrams of two different ways of performing ternary tree partitioning on a current block according to an embodiment of the present disclosure;

[0043] FIG15 is a schematic structural diagram of an intra-frame prediction apparatus based on interpolation filtering according to an embodiment of the present disclosure;

[0044] 16A to 16F are schematic diagrams of six different scanning modes for point-by-point prediction according to an embodiment of the present disclosure.

[0045] Details

[0046] The present disclosure describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure.

[0047] In the description of this disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment described in this disclosure as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments. "And / or" in this disclosure describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, or B exists alone. "Multiple" refers to two or more than two. In addition, to facilitate the clear description of the technical solutions of the embodiments of this disclosure, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or order of execution, and words such as "first" and "second" do not necessarily define differences. "Including any one or more of: option one, option two, ..." or "Including any one or more of option one, option two, ..." in this disclosure means including any one of the listed options, or including any combination of multiple items of the listed options. For example, "including any one or more of the following: A, B" or "including any one or more of A and B" means including only A, or only B, or including A and B; another example: "including any one or more of the following: A, B, C" or "including any one or more of A, B, C" means including only A, or only B, or only C, or including A and B, or including A and C, or including B and C, or including A, B and C. The same applies to options with more options.

[0048] When describing representative exemplary embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art can readily understand that these sequences can vary and still remain within the spirit and scope of the disclosed embodiments.

[0049] The intra-frame prediction method and video coding and decoding method based on interpolation filtering in the embodiments of the present disclosure can be applied to various video coding and decoding standards, such as: H.264 / Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC), H.266 / Versatile Video Coding (VVC), AVS (Audio Video Coding Standard), and other standards developed by MPEG (Moving Picture Experts Group), AOM (Alliance for Open Media), JVET (Joint Video Experts Team) and extensions of these standards, or any other customized standards.

[0050] Figure 1A is a block diagram of a video encoding and decoding system that can be used in embodiments of the present disclosure. As shown in the figure, the system is divided into an encoding end 1 and a decoding end 2. The encoding end 1 generates a bitstream. The decoding end 2 decodes the bitstream. The decoding end 2 can receive the bitstream from the encoding end 1 via a link 3. The link 3 includes one or more media or devices capable of transmitting the bitstream from the encoding end 1 to the decoding end 2. In one example, the link 3 includes one or more communication media that enable the encoding end 1 to send the bitstream directly to the decoding end 2. The encoding end 1 modulates the bitstream according to a communication standard and sends the modulated bitstream to the decoding end 2. The one or more communication media may include wireless and / or wired communication media and may form part of a packet network. In another example, the bitstream can also be output from the output interface 15 to a storage device, and the decoding end 2 can read the stored data from the storage device via streaming or downloading.

[0051] As shown in the figure, encoding end 1 includes a data source 11, a video encoding device 13, and an output interface 15. Data source 11 may include a video capture device (e.g., a camera), an archive containing previously captured data, a feed interface for receiving data from a content provider, a computer graphics system for generating data, or a combination of these sources. Video encoding device 13, also known as a video encoder, encodes data from data source 11 and outputs it to output interface 15. Output interface 15 may include at least one of a modulator, a modem, and a transmitter. Decoding end 2 includes an input interface 21, a video decoding device 23, and a display device 25. Input interface 21 includes at least one of a receiver and a modem. Input interface 21 can receive a bitstream via link 3 or from a storage device. Video decoding device 23, also known as a video decoder, decodes the received bitstream. Display device 25 displays the decoded data. Display device 25 may be integrated with other devices in decoding end 2 or provided separately; display device 25 is optional for the decoding end. In other examples, the decoding end may include other devices or equipment for applying the decoded data.

[0052] FIG1B is a block diagram of an exemplary video encoding device that can be used in an embodiment of the present disclosure. As shown in the figure, the video encoding device 10 includes:

[0053] The division unit 101 is configured to cooperate with the prediction unit 100 to divide the received video data into slices, coding tree units (CTUs) or other larger units. The received video data may be a video sequence including video frames such as I frames, P frames or B frames.

[0054] The prediction unit 100 is configured to divide a CTU into coding units (CUs) and perform intra-frame prediction coding or inter-frame prediction coding on the CU. When performing intra-frame prediction and inter-frame prediction on the CU, the CU can be divided into one or more prediction units (PUs).

[0055] The prediction unit 100 includes an inter-frame prediction unit 121 and an intra-frame prediction unit 126 .

[0056] The inter-frame prediction unit 121 is configured to perform inter-frame prediction on the PU and generate prediction data for the PU, wherein the prediction data includes the prediction block of the PU, the motion information of the PU, and various syntax elements. The inter-frame prediction unit 121 may include a motion estimation (ME) unit and a motion compensation (MC) unit. The motion estimation unit can be used to perform motion estimation to generate a motion vector, and the motion compensation unit can be used to obtain or generate a prediction block based on the motion vector.

[0057] The intra prediction unit 126 is configured to perform intra prediction on a PU and generate prediction data for the PU. The prediction data for the PU may include a prediction block and various syntax elements of the PU.

[0058] The residual generating unit 102 (indicated by the circle with a plus sign after the dividing unit 101 in the figure) is configured to subtract the prediction block of the PU into which the CU is divided from the original block of the CU to generate a residual block of the CU.

[0059] The transform processing unit 104 is configured to partition a CU into one or more transform units (TUs). The partitioning of prediction units and transform units may be different. A TU-associated residual block is a sub-block obtained by partitioning the residual block of the CU. A TU-associated coefficient block is generated by applying one or more transforms to the TU-associated residual block.

[0060] The quantization unit 106 is configured to quantize the coefficients in the coefficient block based on a quantization parameter. The quantization degree of the coefficient block can be changed by adjusting the quantization parameter (QP: Quantizer Parameter).

[0061] The inverse quantization unit 108 and the inverse transform unit 110 are configured to apply inverse quantization and inverse transform to the coefficient block, respectively, to obtain a reconstructed residual block associated with the TU.

[0062] The reconstruction unit 112 (represented by the circle with a plus sign after the inverse transform processing unit 110 in the figure) is configured to add the reconstructed residual block and the prediction block generated by the prediction unit 100 to generate a reconstructed image.

[0063] The filter unit 113 is configured to perform loop filtering on the reconstructed image.

[0064] The decoded image buffer 114 is configured to store the reconstructed image after loop filtering. The intra prediction unit 126 can extract reference images of blocks adjacent to the current block from the decoded image buffer 114 to perform intra prediction. The inter prediction unit 121 can use the reference image of the previous frame cached in the decoded image buffer 114 to perform inter prediction on the PU of the current frame image.

[0065] The entropy coding unit 115 is configured to perform entropy coding operations on received data (such as syntax elements, quantized coefficient blocks, motion information, etc.) to generate a video bitstream.

[0066] In other examples, the video encoding apparatus 10 may include more, fewer, or different functional components than those in this example, for example, the transform processing unit 104 and the inverse transform processing unit 110 may be eliminated.

[0067] FIG1C is a block diagram of an exemplary video decoding device that can be used in an embodiment of the present disclosure. As shown in the figure, the video decoding device 15 includes:

[0068] Entropy decoding unit 150 is configured to perform entropy decoding on the received encoded video stream, extracting syntax elements, quantized coefficient blocks, and motion information for PUs. Prediction unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 159 may each perform corresponding operations based on the syntax elements extracted from the stream.

[0069] The inverse quantization unit 154 is configured to perform inverse quantization on the coefficient block associated with the quantized TU.

[0070] The inverse transform processing unit 156 is configured to apply one or more inverse transforms to the inverse quantized coefficient block to generate a reconstructed residual block for the TU.

[0071] Prediction unit 152 includes an inter-prediction unit 162 and an intra-prediction unit 164. If the current block is encoded using intra prediction, intra prediction unit 164 determines an intra prediction mode for the PU based on syntax elements decoded from the codestream, and performs intra prediction in conjunction with reconstructed reference information adjacent to the current block obtained from decoded image buffer 160. If the current block is encoded using inter prediction, inter prediction unit 162 determines a reference block for the current block based on motion information of the current block and corresponding syntax elements, and performs inter prediction on the reference block obtained from decoded image buffer 160.

[0072] The reconstruction unit 158 ​​(represented by a circle with a plus sign after the inverse transform processing unit 155 in the figure) is set to perform intra-frame prediction or inter-frame prediction on the current block based on the reconstructed residual block associated with the TU and the prediction unit 152 to obtain a reconstructed image.

[0073] The filter unit 159 is configured to perform loop filtering on the reconstructed image.

[0074] The decoded image buffer 160 is configured to store the reconstructed image after loop filtering as a reference image for subsequent motion compensation, intra-frame prediction, inter-frame prediction, etc. The filtered reconstructed image can also be output as decoded video data for presentation on a display device.

[0075] In other embodiments, the video decoding device 15 may include more, fewer, or different functional components. For example, the inverse transform processing unit 155 may be eliminated in some cases.

[0076] Based on the above-described video encoding and decoding devices, the following basic encoding and decoding process can be performed. On the encoding side, a frame of an image is divided into blocks. Intra-frame prediction, inter-frame prediction, or other algorithms are performed on the current block to generate a predicted block for the current block. The predicted block is subtracted from the original block of the current block to obtain a residual block. The residual block is transformed and quantized to obtain quantized coefficients, and the quantized coefficients are entropy encoded to generate a bitstream. On the decoding side, intra-frame prediction or inter-frame prediction is performed on the current block to generate a predicted block for the current block. The quantized coefficients obtained from the decoded bitstream are then inversely quantized and inversely transformed to obtain a residual block. The predicted block and residual block are added together to obtain a reconstructed block. The reconstructed block is then loop-filtered on the reconstructed image based on an image or block basis to obtain a decoded image. The encoding side also performs similar operations as the decoding side to obtain a decoded image, also known as a reconstructed image after loop filtering. The reconstructed image after loop filtering can be used as a reference frame for inter-frame prediction of subsequent frames. Block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode and parameter information determined by the encoding side can be written into the bitstream. The decoding end determines the block division information, prediction, transformation, quantization, entropy coding, loop filtering and other mode information and parameter information used by the encoding end by decoding the code stream or analyzing according to the setting information, so as to ensure that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end.

[0077] Although the above example uses a block-based hybrid coding framework, the embodiments of the present disclosure are not limited thereto. As technology advances, one or more modules in the framework and one or more steps in the process may be replaced or optimized. The embodiments of the present disclosure relate to, but are not limited to, the intra-frame prediction units and corresponding intra-frame prediction methods in the encoding and decoding ends described above.

[0078] In this article, the current block can be the current coding unit (CU) in the current image, a coding tree unit (CTU), a current prediction unit (PU), or any other block-level coding unit, such as the sub-blocks of the current CU or PU. In intra-frame prediction, the current block is also referred to as the block to be predicted or the current block to be predicted.

[0079] When performing intra-frame prediction, the encoder typically uses various angle modes and non-angle modes to predict the current block to obtain a prediction block. Based on the rate-distortion information calculated between the prediction block and the original block, the encoder selects the optimal intra-frame prediction mode for the current block, encodes the information about the intra-frame prediction mode (such as the index), and transmits it to the decoder via the bitstream. The decoder determines the currently selected intra-frame prediction mode through decoding and performs intra-frame prediction on the current block according to this intra-frame prediction mode. Through the development of successive digital video coding standards, non-angle modes have remained relatively stable, including the mean mode (i.e., DC mode) and the planar mode (i.e., Planar mode). Angled modes have continued to increase with the evolution of digital video coding standards. Taking the H series of international digital video coding standards as an example, the H.264 / AVC standard only has 8 traditional angle prediction modes and 1 traditional non-angle prediction mode; H.265 / HEVC has expanded to 33 traditional angle prediction modes and 2 traditional non-angle prediction modes. In H.266 / VVC, traditional intra-frame prediction modes include Planar mode, DC mode, and 65 angle modes, as shown in Figure 2. The DC mode is suitable for large flat areas, where the prediction value is calculated by averaging the reference pixels to the left and / or above. The Planar mode is suitable for areas with gradual pixel changes, that is, areas with slowly varying pixel values. The coding performance achieved by current intra-frame prediction modes still has room for improvement.

[0080] An embodiment of the present disclosure provides an intra-frame prediction method based on interpolation filtering, as shown in FIG3 , including:

[0081] Step S110, determining a filter coefficient used for interpolation filtering of the current block according to the filter window of the current block and the reconstructed values ​​of the pixels in the reconstructed area adjacent to the current block;

[0082] Step S120, performing point-by-point prediction on the pixels in the current block according to the filter window and the filter coefficient to obtain a predicted block of the current block;

[0083] When predicting the current pixel, reference pixels adjacent to the current pixel are determined according to the filtering window, and interpolation filtering is performed based on the pixel values ​​of the reference pixels and the filtering coefficients to obtain a predicted value of the current pixel.

[0084] The disclosed embodiment utilizes the spatial redundancy of the current block and its adjacent reconstructed areas, applies the obtained filter coefficients to the pixels of the current block one by one, performs interpolation filtering based on the reconstructed values ​​or predicted values ​​of the adjacent pixels, and obtains the predicted value of the current pixel, until the predicted values ​​of all the pixels of the current block are obtained, that is, the predicted block of the current block is obtained.

[0085] This intra-frame prediction method of the embodiment of the present disclosure is referred to herein as intra-frame prediction in interpolation filtering mode, which is an optional intra-frame prediction mode in video encoding and decoding to improve encoding performance.

[0086] The filter used in the interpolation filtering of the disclosed embodiments is called an interpolation filter (referred to as a filter). The filter window and filter coefficients used in the interpolation filtering are referred to as the filter window and filter coefficients used by the interpolation filter. The shape and number of taps of the filter window used by the interpolation filter are also referred to as the shape and number of taps of the interpolation filter.

[0087] The filter window can be of any shape and any size greater than 3 pixels. When the filter window is small, the filter coefficients (referred to as coefficients) are small and the prediction complexity is low, but the number of reference pixels used in the prediction is small, and the prediction error may be large. When the filter window is large, the filter coefficients are large, the prediction complexity is high, and the prediction error may become smaller, but a filter window that is too large may also cause overfitting. The shape and size of the filter window disclosed in the present invention are not limited.

[0088] In one example, a 4×4 filter window as shown in FIG4A is used. The filter window includes 1 current position and 15 adjacent positions P0 to P14. When the current position is aligned with the position of the current pixel to be predicted, the 15 pixels on P0 to P14 are the 15 reference pixels adjacent to the current pixel. When calculating the predicted value of the current pixel by interpolation filtering, the pixel values ​​of the 15 reference pixels need to be multiplied by their corresponding filter coefficients C0 to C14. FIG4B shows a 15-tap interpolation filter. The filter uses a set of filter coefficients including 15 filter coefficients corresponding one-to-one to the 15 adjacent positions in the filter window of FIG4A. This set of filter coefficients is used when performing interpolation filtering on each pixel in the current block.

[0089] In addition to rectangular filter windows, filter windows of other shapes and numbers of neighboring positions may also be used. In an exemplary embodiment of the present disclosure, the filter window of the current block is determined based on block parameters of the current block, wherein the block parameters include at least one of shape, size, and position, such as size, such as width and height. For example, a 4×4 filter window is used for an 8×8 current block, and a 5×5 filter window is used for a 16×16 current block.

[0090] In an exemplary embodiment of the present disclosure, filters with multiple tap numbers are allowed to be used to predict the current block, obtaining different prediction values, and then an optimal filter is selected from them to participate in rate-distortion optimization to obtain better coding performance.

[0091] When performing point-by-point prediction on the current block, if the current pixel to be predicted is at a different position in the current block, the reference pixels adjacent to the current pixel may be pixels in the reconstructed area or pixels in the current block. As shown in FIG5A , when the current pixel to be predicted is at the upper left corner of the current block, after aligning the current pixel with the current position of the filter window, the reference pixels adjacent to the current pixel are all reconstructed pixels in the reconstructed area. As shown in FIG5B , when the current pixel to be predicted is at a position other than the upper left corner in the first row of the current block (shown in the first row and second column), after aligning the current pixel with the current position of the filter window, the reference pixels adjacent to the current pixel include both reconstructed pixels in the reconstructed area and predicted pixels in the current block (such as the pixel in the upper left corner of the current block). The order of point-by-point prediction ensures that the reference pixels at this point have already been predicted. As shown in the example of FIG5C , when the current pixel to be predicted is located in the 4th row and the 4th column within the current block, after aligning the current pixel with the current position of the 4×4 filtering window, the reference pixels adjacent to the current pixel are all predicted pixels in the current block.

[0092] In an exemplary embodiment of the present disclosure, the reference pixel point includes at least one of the following: a reconstructed pixel point in a reconstructed area adjacent to the current block, a predicted pixel point in the current block;

[0093] When the reference pixels are all reconstructed pixels, the pixel values ​​of the reference pixels refer to the reconstructed values ​​of the reference pixels;

[0094] When the reference pixels are all predicted pixels, the pixel values ​​of the reference pixels refer to the predicted values ​​of the reference pixels;

[0095] When the reference pixel points include reconstructed pixel points and predicted pixel points, the pixel values ​​of the reference pixel points include the reconstructed values ​​of the reconstructed pixel points in the reference pixel points and the predicted values ​​of the predicted pixel points in the reference pixel points.

[0096] This embodiment uses the reconstructed values ​​of reconstructed pixels and / or the predicted values ​​of predicted pixels as input to the interpolation filter to calculate the predicted value of the current pixel. To ensure that the reference pixels in the current block have predicted values, the interpolation filter prediction needs to be performed point by point in a certain order.

[0097] Different filter windows (at least one of shape and size) can be set for different current blocks. Multiple filter windows can also be set for the same current block. A larger filter window contains more neighboring locations and more corresponding filter coefficients, allowing for more reference to neighboring spatial information for interpolation. Filters of different shapes can interpolate neighboring spatial information from different directions.

[0098] The filter window can be divided into three types, as shown in Figures 6A to 6C. The crossed rectangle in the figure represents the current pixel to be predicted, and the fan-shaped area represents the fan-shaped area where the adjacent position in the filter window is located when the current position of the filter window is aligned with the position of the current pixel, that is, the fan-shaped area where the reference pixel is located. The input of the interpolation filter is the reconstructed value or predicted value of the pixel in the direction restricted by the fan-shaped area. In the fan-shaped area represented by Figure 6A, the reference pixel is spatially located in the area directly above, to the upper left, and to the left of the current pixel, and there is no reference pixel in the lower left or upper right direction of the current pixel; in the fan-shaped area represented by Figure 6B, the reference pixel is spatially located in the area directly above, to the upper left, to the left, and to the upper right direction of the current pixel, and there is no reference pixel in the lower left direction of the current pixel; in the fan-shaped area represented by Figure 6C, the reference pixel is spatially located in the area directly above, to the upper left, to the left, and to the lower left direction of the current pixel, and there is no reference pixel in the upper right direction of the current pixel.

[0099] In an exemplary embodiment of the present disclosure, corresponding to the three situations shown in FIG. 6A to FIG. 6C , the filter windows have the following three types:

[0100] Multiple adjacent positions in the filter window are located directly above, above and to the left of the current position; an example of such a filter window is shown in FIG7A , corresponding to FIG6A ;

[0101] Multiple adjacent positions in the filter window are located directly above, above and to the left, to the left, and above and to the right of the current position; an example of such a filter window is shown in FIG7B , corresponding to FIG6B ;

[0102] Multiple neighboring positions in the filter window are located directly above, above and to the left, to the left, and below and to the left of the current position; an example of such a filter window is shown in FIG. 7C , corresponding to FIG. 6C .

[0103] In Figures 7A to 7C , the diagonally-lined squares represent the current position, and the other squares represent adjacent positions. In this embodiment, the filter window of the current block is rectangular or L-shaped, including one current position corresponding to the current pixel and at least two adjacent positions corresponding to reference pixels.

[0104] When the types of filter windows are different, in order to ensure that the reference pixels in the current block are all predicted pixels during point-by-point prediction, the prediction orders that can be selected are different.

[0105] In an example of this embodiment, the prediction order of different filtering windows is as follows:

[0106] When the plurality of adjacent positions in the filtering window are located directly above, above and to the left of the current position, the pixel-by-pixel prediction of the current block is performed in the current block in a left-to-right and then top-to-bottom order, or a top-to-bottom and then left-to-right order, or a diagonal order, or a Zig-Zag order;

[0107] When the plurality of adjacent positions in the filtering window are located directly above, above and to the left, to the left, and below and to the left of the current position, the pixel-by-pixel prediction of the current block is performed in the current block in a top-to-bottom and then left-to-right order;

[0108] When multiple adjacent positions in the filtering window are located directly above, above the left, to the left, and above the right of the current position, the point-by-point prediction of the pixels in the current block is performed in the current block in an order from left to right and then from top to bottom.

[0109] Taking the filter window shown in Figure 4A as an example, the order from left to right and then from top to bottom is P0, P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, and the order from top to bottom and then from left to right is P0, P4, P8, P12, P1, P5, P9, P13, P2, P6, P10, P14, P3, P7 The order of the diagonal direction is P0, P1, P4, P2, P5, P8, P3, P6, P9, P12, P7, P10, P13, P11, P14 or P0, P4, P1, P8, P5, P2, P12, P9, P6, P3, P13, P10, P7, P14, P11. The Zig-Zag order is also called a zigzag order. Among them, from left to right is also called horizontal scanning, as shown in Figure 16A; from top to bottom is also called vertical scanning, as shown in Figure 16B. Two diagonal scanning methods are shown in Figures 16C and 16D. Two zigzag scanning methods are shown in Figures 16E and 16F.

[0110] Using the above filtering window, the filter input can come from the reconstructed values ​​of reconstructed pixels and / or predicted values ​​of predicted pixels in the upper left, upper right, upper left, or lower left portion of the current pixel to be predicted (i.e., the current interpolation position). When predicting a current block point by point, it is allowed to try out multiple filtering windows of different shapes and / or sizes to improve prediction performance.

[0111] In addition to the filter window, the shape and size of the reconstruction area will also affect the prediction results.

[0112] In an exemplary embodiment of the present disclosure, the reconstructed area adjacent to the current block includes any one or more of the following areas adjacent to the current block:

[0113] An upper reconstruction region located above the current block, each upper reconstruction region including one region, two consecutive regions, or three consecutive regions among the upper left region, the upper region, and the upper right region of the current block;

[0114] A left reconstruction region located on the left side of the current block, each left reconstruction region including one region, two consecutive regions, or three consecutive regions among the upper left region, the left region, and the lower left region of the current block;

[0115] L-shaped reconstruction areas located above and to the left of the current block, each L-shaped reconstruction area includes the area directly above, the area directly above and the area directly below the current block, or includes one or two of the area directly above, the area directly above and the area directly below the current block, as well as the area directly above and the area directly below the current block.

[0116] Herein, the above-mentioned “upper reconstruction region”, “left reconstruction region” and “L-shaped reconstruction region” are referred to as types of reconstruction regions.

[0117] As shown in FIG8A , a reconstruction area adjacent to the current block is an upper reconstruction area, which includes the area immediately above, the upper left area, and the upper right area of ​​the current block. The upper reconstruction area may also include only the area immediately above the current block, or only the upper left area of ​​the current block, or only the upper right area of ​​the current block, or only the area immediately above and the upper right area of ​​the current block, or only the area immediately above and the upper left area of ​​the current block. In this embodiment, the upper reconstruction area is a continuous area.

[0118] As shown in FIG8B , a reconstructed area adjacent to the current block is the left reconstructed area, which includes the upper left area, the left area, and the lower left area of ​​the current block. The left reconstructed area may also include only the upper left area of ​​the current block, or only the left area of ​​the current block, or only the lower left area of ​​the current block, or only the left area and the upper left area of ​​the current block, or only the left area and the lower left area of ​​the current block. In this embodiment, the left reconstructed area is a continuous area.

[0119] As shown in FIG8C , the reconstruction region adjacent to the current block is an L-shaped reconstruction region, including the region immediately above, the upper left region, the upper left region, the upper right region, and the lower left region of the current block. The position of each region relative to the current block is defined as shown in the figure. The L-shaped reconstruction region may also include only the region immediately above, the upper left region, and the upper left region of the current block, or only the region immediately above, the upper left region, the upper left region, and the upper right region of the current block, or only the region immediately above, the upper left region, the upper left region, and the lower left region of the current block. In this embodiment, the L-shaped reconstruction region is a continuous region.

[0120] In addition to having multiple types, the size of the reconstructed area adjacent to the current block can also be determined based on the size of the current block (i.e., it can be a dependent variable with the size of the current block as the independent variable). As shown in Figures 8A to 8C, the current block in the figure is a rectangular current block with a width of M and a height of N. M and N can be represented by the number of pixels, such as 8×8, 8×16, 4×8, etc. Taking Figure 8C as an example, the L-shaped reconstructed area adjacent to the current block located in the upper left, directly above, and upper right of the current block has an overall width of 2M+L and a height of K; the L-shaped reconstructed area located in the directly left and lower left of the current block has an overall width of L and a height of 2N. After the width and height of the current block are determined, combined with the set K and L, the reconstructed area point can be uniquely determined, where K represents the number of rows of the reconstructed area above the current block, and L represents the number of columns of the reconstructed area to the left of the current block. Similar to FIG8C , the sizes of the upper reconstruction area in FIG8A and the left reconstruction area in FIG8B also change with the size of the current block. The corresponding reconstruction area can be uniquely determined by combining the set K and L.

[0121] In an exemplary embodiment of the present disclosure, there is one reconstruction area adjacent to the current block; or, there are multiple reconstruction areas adjacent to the current block, and at least one of type, width, height and position of different reconstruction areas is different.

[0122] Herein, the types of reconstruction regions include an upper reconstruction region, a left reconstruction region, and an L-shaped reconstruction region. A reconstruction region may be an upper reconstruction region, a left reconstruction region, or an L-shaped reconstruction region.

[0123] In the embodiments of the present disclosure, for a current block of the same size, not only can multiple types of reconstruction regions be set, but multiple reconstruction regions of the same type but different sizes can also be set. Taking the L-shaped reconstruction region shown in FIG8A as an example, setting K=L=8 results in a reconstruction region adjacent to the current block, and setting K=L=13 results in another reconstruction region adjacent to the current block. If the three reconstruction regions shown in FIG8A to FIG8C are set for the current block, and each reconstruction region further obtains two reconstruction regions by setting K=L=8 and K=L=13, a total of six reconstruction regions adjacent to the current block can be obtained. In other examples, the values ​​of K and L can be other values, such as being associated with the size of the current block, with different values ​​of K and L for current blocks of different sizes, with K and L being integers greater than zero. In other examples, more than three values ​​can be set for the combination of K and L to obtain more different reconstruction regions.

[0124] The number of rows above the current block (corresponding to K in FIG. 10 ) and the number of columns to the left of the current block (corresponding to L in FIG. 10 ) in the reconstructed area adjacent to the current block may also be determined based on the width and / or height of the current block.

[0125] For a current block, assume that two combinations of K and L are set: K1 and L1, and K2 and L2, then:

[0126] In one example, K1 is equal to L1, K2 is equal to L2, and the values ​​of K1 and K2 correspond to the smaller value of the width and height of the current block minWh=min(width, height), for example, K1 is equal to 2minWh, and K2 is equal to minWh.

[0127] In one example, K1 and K2 are related to the width of the current block, and L1 and L2 are related to the height of the current block. For example, K1 is equal to 2width, K2 is equal to width, L1 is equal to 2height, and L2 is equal to height.

[0128] In one example, K1 and K2 are related to the height of the current block, and L1 and L2 are related to the width of the current block. For example, K1 is equal to 2height, K2 is equal to height, L1 is equal to 2width, and L2 is equal to width.

[0129] In other examples, K1, K2, L1, and L2 are related to the length and height of the current block but are not multiples of each other.

[0130] In an exemplary embodiment of the present disclosure, there are one or more filter windows for the current block, and there are one or more reconstruction areas adjacent to the current block;

[0131] The determining, based on the filter window of the current block and the reconstructed values ​​of the pixels in the reconstructed area adjacent to the current block, filter coefficients used when performing interpolation filtering on the current block includes: for each combination of the filter window of the current block and the reconstructed area adjacent to the current block, determining a corresponding set of filter coefficients based on the filter window in the combination and the reconstructed values ​​of the pixels in the reconstructed area in the combination;

[0132] The method includes performing point-by-point prediction on pixels in the current block according to the filter window and the filter coefficient to obtain a prediction block of the current block, comprising: for each combination of the filter window of the current block and a reconstructed area adjacent to the current block, performing point-by-point prediction on pixels in the current block according to the filter window in the combination and a set of filter coefficients corresponding to the combination to obtain a prediction block of the current block.

[0133] It's easy to understand that when the filter window is the same but the reconstruction regions are different, the reconstruction values ​​used to calculate the filter coefficients are different, resulting in different filter coefficients. Conversely, when the reconstruction regions are the same but the filter windows are different, the reconstruction values ​​used to calculate the filter coefficients are also different, resulting in different filter coefficients. For a combination of the filter window of the current block and the reconstruction region adjacent to the current block (each combination includes a filter window and a reconstruction region), a set of filter coefficients can be calculated. If there are multiple combinations of the filter window of the current block and the reconstruction region adjacent to the current block, multiple sets of filter coefficients can be calculated. Each combination can serve as a submode of the interpolation filter mode. For example, using the filter window shown in FIG4A and six reconstruction regions (including the three types shown in FIG8A through FIG8C , each type including two reconstruction regions of different sizes), six sets of filter coefficients can be calculated. When using the interpolation filter mode for intra-frame prediction, there are six submodes, and predictions for the current block obtained using these six submodes are different.

[0134] In one example of this embodiment, each filter window of the current block includes a current position and multiple adjacent positions, and a set of filter coefficients corresponding to the combination including the filter window includes multiple filter coefficients corresponding one-to-one to the multiple adjacent positions;

[0135] The method performs point-by-point prediction on the pixels in the current block based on the filter window in the combination and a set of filter coefficients corresponding to the combination, including: aligning the current pixel with the current position of the filter window in the combination, determining the pixels at all adjacent positions of the filter window in the combination as reference pixels adjacent to the current pixel, and performing interpolation filtering based on the pixel values ​​of the reference pixels and a set of filter coefficients corresponding to the combination to obtain a predicted value of the current pixel.

[0136] Taking FIG. 4A as an example, when the position of the current pixel is aligned with the current position of the filtering window, the pixels at the 15 adjacent positions P0 to P14 are the reference pixels adjacent to the current pixel.

[0137] In an exemplary embodiment of the present disclosure, determining a corresponding set of filter coefficients according to the filter window in the combination and the reconstructed values ​​of the pixels in the reconstructed area in the combination includes:

[0138] The filter window in the combination is placed at M different positions with reconstructed pixels in the reconstruction area of ​​the combination, where M≥1;

[0139] Taking the M current positions in the M different positions of the filter window as sample positions and the reconstructed values ​​of the pixels at all adjacent positions in the M different positions of the filter window as input, constructing the Wienerhof equation;

[0140] With the goal of minimizing the error between the predicted values ​​and the reconstructed values ​​of the M current positions, the Wienerhof equation is solved to obtain a set of filter coefficients corresponding to the combination.

[0141] In this embodiment, when M is greater than 1, the M filter windows at different positions may be determined by moving the filter window at the initial position according to a set step size and direction.

[0142] In this embodiment, for different current blocks, the reconstruction areas are different, so the calculated filter coefficients are usually different, that is, different current blocks will adaptively use their own filter coefficients.

[0143] The following describes how to construct the Wienerhof equations. When constructing the Wienerhof equations, multiple sample positions must be determined. These sample positions constitute portions of the reconstruction area for the current block. As shown in Figure 9A, assuming a 4×4 filter window is used and the current position is located at the lower right corner of the filter window, when the filter window is positioned at M locations within the reconstruction area, each current position of the filter window is a sample position. The small squares marked with crosses in the figure represent sample positions, and the range within which these sample positions lie constitutes a portion of the reconstruction area. The reconstruction area in the figure includes the six rows above and six columns to the left of the current block. The 4×4 filter window must be completely positioned within the reconstruction area, so the sample positions fall within a continuous range of two rows above and two columns to the left of the current block. Figure 9A shows the filter window positioned at every possible location in the reconstruction area, but it is also possible to select a subset of locations to reduce computational complexity. As shown in Figure 9B, a subset of the sample positions in Figure 9A is selected to construct the Wienerhof equations. The sample position step size is 2 in the width direction and 1 in the height direction. The sample position range is a set of discrete locations. Furthermore, if the current block position is different, portions of the reconstruction area may not contain reconstructed pixels. In this example, when the filter window in the combination is placed in the reconstruction area in the combination, the selected position makes the pixel points covered by the filter windows at M different positions all reconstructed pixel points; but in other examples, the pixel values ​​of some of the pixel points covered by the filter windows at the M different positions can also be pixel values ​​obtained by filling.

[0144] The sample position is recorded as r, and the range of the sample position is recorded as When the filter window in the combination is placed at M different positions with reconstructed pixels in the reconstruction area of ​​the combination, the range There are M sample locations in the . Let the coordinate of r be (x , y), then the range Each sample position in has a set of (x, y) values. For each sample position r, the neighboring positions in the filter window where the sample position is located can be expressed as r+p0, r+p1,…, r+p N-1 ,{p0,p1,…,p N-1} is the offset relative to the sample position, which can also be regarded as the index of multiple adjacent positions in the filter window with the current position being the sample position r, and N is the number of adjacent positions in the filter window. In the examples of Figures 4A and 4B, N = 15. N-1 ] represent the reconstructed values ​​of the pixels at N adjacent positions, which are the inputs for solving the Wienerhof equation. t[r] represents the reconstructed value of the pixel at the sample position r. The set of filter coefficients to be solved is denoted as c=[c0,c1,…,c N-1 ] T, where c0 is the filter coefficient corresponding to the p0 position, c1 is the filter coefficient corresponding to the p1 position, and so on.

[0145] The constructed Wienerhof equation is as follows:

[0146] The N×N matrix on the left side of the above formula is the autocorrelation coefficient matrix of the Wienerhof equation, where Indicates the range The t[r+p0]t[r+p0] corresponding to all sample positions r in the t[r+p0] are accumulated, and so on.

[0147] After constructing the above Wienerhof equation, the goal is to minimize the error between the predicted value and the reconstructed value of the M current positions. The Wienerhof equation is solved to obtain a set of filter coefficients corresponding to the combination. Assume that the predicted value of the sample position r after interpolation filtering is t[r]′, and minimize the error between the predicted value and the reconstructed value of the M current positions, that is, minimize the error between t[r]′ and t[r], where

[0148] Since the filter coefficients need to be derived from the encoder and decoder through the reconstruction area to consistent values, taking into account platform issues and computational complexity, this embodiment uses integer precision to calculate the filter coefficients (i.e., full integer calculations), and the solution method is not limited, including but not limited to decomposing the autocorrelation coefficient matrix of the Wienerhof equation through Kresky decomposition or lower unit triangular matrix (LDL) decomposition.

[0149] In an example of an embodiment of the present disclosure, performing interpolation filtering based on the pixel value of the reference pixel and a set of filter coefficients corresponding to the combination to obtain a predicted value of the current pixel includes:

[0150] The predicted value pred of the current pixel is calculated according to the following formula:

[0151] Alternatively, the predicted value pred of the current pixel is calculated according to the following formula:

[0152] in, is the pixel value of the nth reference pixel of the current pixel, n=0,1,…,N-1, c n is the filter coefficient corresponding to the nth reference pixel in the set of filter coefficients, N is the number of reference pixels, that is, the number of filter coefficients included in the set of filter coefficients, offset and shift are predefined values, ">>" represents a right shift operation of a binary number, for example, ">>3" represents a right shift of 3 bits, and formula (2) is a right shift of shift bits.

[0153] The above formula (2) is a specific algorithm proposed to avoid decimal operations, where c n Relative to c in formula (1) n After amplification, the accumulated result needs to be corrected by the coefficients offset and shift, where offset is used for offset and shift is used for scaling to obtain accurate pred.

[0154] In another exemplary embodiment of the present disclosure, when obtaining filter coefficients using the Wienerhof equation, a de-averaging operation is performed on the input reconstruction value. This embodiment determines a corresponding set of filter coefficients based on the filter window in the combination and the reconstruction values ​​of the pixels in the reconstruction area in the combination, including:

[0155] The filter window in the combination is placed at M different positions with reconstructed pixels in the reconstruction area of ​​the combination, where M≥1;

[0156] Taking the M current positions in the M different positions of the filter window as sample positions and the reconstructed values ​​of all adjacent positions of the pixel points in the M different positions of the filter window as input, constructing the Wienerhof equation;

[0157] With the goal of minimizing the error between the predicted values ​​and the reconstructed values ​​of the M current positions, the Wienerhof equation is solved to obtain a set of filter coefficients corresponding to the combination.

[0158] The difference between the method of constructing the Wienerhof equation in this embodiment and the above-mentioned embodiment is that the previous embodiment uses the reconstructed value as input to construct the Wienerhof equation, and the present embodiment uses the reconstructed value after de-averaging as input to construct the Wienerhof equation.

[0159] Assuming that the mean is m, the Wienerhof equation constructed in this embodiment is as follows:

[0160] In an example of this embodiment, the reconstructed value after removing the mean is obtained by subtracting the mean from the reconstructed value, and the mean is obtained in the following manner:

[0161] The reconstruction values ​​of all pixels in the reconstruction area of ​​the combination are averaged to obtain the average value: or

[0162] averaging the reconstructed values ​​of some pixels in the reconstructed area of ​​the combination to obtain the mean value; or

[0163] The reconstructed value of a designated pixel point in the reconstructed area of ​​the combination is used as the mean value.

[0164] Referring to the example shown in FIG10 , the reconstructed region of the current block includes K rows above and L columns to the left of the current block. When averaging the reconstructed values ​​of all pixels in the reconstructed region of the combination, the reconstructed values ​​of all pixels in the entire reconstructed region are averaged to obtain the average. When averaging the reconstructed values ​​of a portion of the pixels in the reconstructed region of the combination, the portion of pixels may include: pixels in one or more rows above the current block and / or pixels in one or more columns to the left of the current block. The exemplary portion of pixels in FIG10 is located in the hatched region one row above and one row to the left of the current block, including pixels in the reconstructed region one row above and directly above and to the upper right of the current block, and pixels in the reconstructed region one column to the left of the current block and directly to the left and to the lower left of the current block. The reconstructed value of a specified pixel in the reconstructed region of the combination used as the average may be the reconstructed value of a pixel in the reconstructed region adjacent to the upper left corner of the current block, indicated by a crossed-out square in the figure. Using the reconstructed values ​​of some pixels to calculate the mean can simplify the calculation. Using the reconstructed value of a single pixel as the mean not only simplifies the calculation but also avoids the division operation when calculating the mean.

[0165] When the filter coefficient is solved based on the reconstructed value after de-averaging, when predicting the current pixel point, the pixel value of the reference pixel point needs to be de-averaged and then multiplied by the corresponding filter coefficient and accumulated, and the accumulated result needs to be added with the mean value.

[0166] In an example of this embodiment, when the set of filter coefficients is obtained by solving the Wienerhof equation with the reconstructed value after de-averaging as input, performing interpolation filtering based on the pixel value of the reference pixel and the set of filter coefficients corresponding to the combination to obtain the predicted value of the current pixel includes: the predicted value pred of the current pixel is calculated according to the following formula:

[0167] in, is the pixel value of the nth reference pixel of the current pixel, n=0,1,…,N-1, c n is the filter coefficient corresponding to the nth reference pixel in the set of filter coefficients, N is the number of reference pixels, that is, the number of filter coefficients included in the set of filter coefficients, and m is the mean used when calculating the reconstructed value after de-averaging.

[0168] This embodiment uses interpolation filtering with de-averaging, and the corresponding filter can be called a de-averaging filter. Using the reconstructed value after de-averaging as the input to construct the filter coefficients obtained by the Wienerhof equation can achieve better prediction results during prediction. The mean value used when de-averaging can be the mean of the reconstructed values ​​of the pixels on the entire reconstructed area, or it can be the mean value derived from the reconstructed values ​​of some pixels (which can be one or more pixels) on the reconstructed area.

[0169] Because the intra-frame prediction method based on interpolation filters proposed in the embodiments of the present disclosure requires previously calculated predicted values ​​as input when predicting certain positions, the error of the predicted values ​​relative to the original values ​​is often greater than the error of the reconstructed values ​​relative to the original values. This recursive prediction may lead to an increase in error accumulation. The impact of error accumulation on prediction performance can be mitigated by limiting the range of the interpolation filter output.

[0170] In an exemplary embodiment of the present disclosure, performing interpolation filtering based on the pixel value of the reference pixel and the filter coefficient to obtain a predicted value of the current pixel includes:

[0171] Performing interpolation filtering based on the pixel value of the reference pixel and the filter coefficient to obtain an initial prediction value;

[0172] When the initial prediction value exceeds the prediction value range of the current block, modifying the initial prediction value so that the modified prediction value is within the prediction value range of the current block; and

[0173] The corrected prediction value is used as the prediction value of the current pixel.

[0174] In an example of an embodiment of the present disclosure, the correcting the initial prediction value includes:

[0175] When the initial predicted value is greater than the maximum value of the predicted value range, the maximum value is used as the revised predicted value;

[0176] When the initial prediction value is smaller than the minimum value of the prediction value range, the minimum value is used as the corrected prediction value.

[0177] In this example, the predicted value range can be determined by one of the following methods:

[0178] Make each output value within the prediction value range allowed by the data. The prediction value range can be determined according to the data depth of the prediction value. For example, for a prediction value with an 8-bit depth, the prediction result should be limited to 0 to 2 8 -1 range;

[0179] This range can also be narrowed, for example, based on the maximum and minimum reconstruction values ​​of the pixels in the reconstruction area or its local area used when determining the filter coefficient. In one example, after finding the minimum and maximum reconstruction values ​​of the pixel values ​​in the corresponding reference area or its local area, the minimum reconstruction value is used as the minimum value of the prediction value range, and the maximum reconstruction value is used as the maximum value of the prediction value range. In another example, the range of the minimum and maximum values ​​in the previous item can also be appropriately relaxed or narrowed, and the minimum reconstruction value found minus a set deviation is used as the minimum value of the prediction value range, and the minimum reconstruction value found plus another set deviation is used as the maximum value of the prediction value range, and the set deviation is positive or negative.

[0180] The embodiment of the present disclosure can avoid excessive error accumulation caused by point-by-point prediction by limiting the output range of the interpolation filter.

[0181] An embodiment of the present disclosure further provides a video encoding method, as shown in FIG11 , including:

[0182] Step S210: performing intra-frame prediction on the current block based on multiple intra-frame prediction modes. If it is determined that the current block allows the use of the interpolation filtering mode, performing intra-frame prediction in the interpolation filtering mode on the current block according to the intra-frame prediction method based on interpolation filtering according to any embodiment of the present disclosure;

[0183] Step S220 , when determining that the current block uses an interpolation filter mode for intra-frame prediction according to the rate-distortion costs of the multiple intra-frame prediction modes, encodes an interpolation filter mode flag of the current block to indicate that the current block uses the interpolation filter mode for intra-frame prediction.

[0184] The embodiment of the present disclosure introduces an interpolation filter mode as an intra-frame prediction mode. When determining that the current block uses the interpolation filter mode for intra-frame prediction, the interpolation filter mode flag of the current block is encoded. The use of the newly added intra-frame prediction mode can improve compression efficiency.

[0185] In an example of this embodiment, the rate-distortion costs of the multiple intra-frame prediction modes may be compared, and when the rate-distortion cost of the interpolation filtering mode is the smallest, it is determined that the current block uses the interpolation filtering mode for intra-frame prediction.

[0186] Some restrictions can be added to the use of the interpolation filter mode. For example, when the current block is a chroma block, the interpolation filter mode is not used for intra-frame prediction during encoding, but the DM (direct mode) mode can be selected for the chroma block so that the chroma block can directly use the intra-frame prediction mode of the luminance block. For another example, flags at the sequence level, image level, and slice level can be used to indicate whether the interpolation filter mode is allowed when encoding the current block. The relevant general constraints information (GCI) can also be set to indicate whether the interpolation filter mode is prohibited. For another example, the size of the current block using the interpolation filter mode can be restricted. In addition, when the current block is located at the image boundary, the interpolation filter mode can be disabled because there is not enough reconstruction area to place the filter window.

[0187] In an exemplary embodiment of the present disclosure, determining that the current block is allowed to use the interpolation filtering mode includes: when none of the conditions for prohibiting the use of the interpolation filtering mode are met, determining that the current block is allowed to use the interpolation filtering mode; wherein the conditions for prohibiting the use of the interpolation filtering mode include any one or more of the following conditions:

[0188] The current block is a chroma block;

[0189] There is a sequence-level flag indicating that interpolation filter mode is not allowed;

[0190] The presence of an image-level flag indicating that interpolation filter mode is not allowed;

[0191] There is a slice-level flag indicating that interpolation filter mode is not allowed;

[0192] The presence of a GCI flag indicating that the interpolation filter mode is prohibited;

[0193] The size of the current block does not meet the maximum block size and / or minimum block size for enabling the interpolation filtering mode;

[0194] The difference between the horizontal coordinate of the upper left corner of the current block and the horizontal coordinate of the upper left corner of the current image is less than a set first threshold;

[0195] The difference between the vertical coordinate of the upper left corner of the current block and the vertical coordinate of the upper left corner of the current image is less than a set second threshold.

[0196] In an exemplary embodiment of the present disclosure, at the encoding end, as described in the aforementioned embodiment, a current block can use a combination of multiple filter windows and reconstruction areas to perform intra-frame prediction based on interpolation filtering, and each combination can be regarded as a sub-mode of the interpolation filtering mode.

[0197] This embodiment performs intra-frame prediction in an interpolation filtering mode on the current block, including:

[0198] Determine a combination of a filter window of the current block and a reconstructed area adjacent to the current block to be used when performing intra-frame prediction of the current block in an interpolation filtering mode, where K combinations correspond to K sub-modes of the interpolation filtering mode, and perform intra-frame prediction based on the interpolation filtering on the current block based on the filter window of the current block and the reconstructed area adjacent to the current block in the combination corresponding to each sub-mode, where K is the number of combinations, and K ≥ 2;

[0199] When determining, based on the rate-distortion costs of the multiple intra-frame prediction modes, to use the interpolation filtering mode for intra-frame prediction of the current block, a minimum value of the rate-distortion costs of the K sub-modes is used as the rate-distortion cost of the interpolation filtering mode, and compared with the rate-distortion costs of other intra-frame prediction modes among the multiple intra-frame prediction modes;

[0200] When the rate-distortion cost of the interpolation filtering mode is the smallest, the method further includes: encoding index information for indicating the sub-mode with the smallest rate-distortion cost among the K sub-modes.

[0201] When the encoder determines that the current block uses the interpolation filter mode for intra-frame prediction and there are multiple sub-modes, in order to avoid the decoder from reselecting the sub-mode, the index of the sub-mode with the lowest rate-distortion cost (i.e., the selected sub-mode) can be encoded into the bitstream.

[0202] In an exemplary embodiment of the present disclosure, the index representing the sub-mode with the smallest rate-distortion cost among the K sub-modes includes any one or more of the following indexes:

[0203] Sub-mode index, each sub-mode index corresponds to a combination of the filter window of the current block and the reconstructed area adjacent to the current block;

[0204] The reconstruction area index of the current block, each reconstruction area index corresponds to a reconstruction area;

[0205] a parameter index of a reconstructed area of ​​the current block, wherein the parameters of the reconstructed area include at least one of a size and a type, and the size includes at least one of the number of rows of the reconstructed area above the current block and the number of columns of the reconstructed area to the left of the current block; the reconstructed area may have multiple parameter indexes, such as a size index and a type index, or may have only one parameter index, for example, a value of the parameter index represents a combination of the size and type of the current block.

[0206] The filter window index of the current block, each filter window index corresponds to a filter window;

[0207] A parameter index of a filter window of a current block, where the parameters of the filter window include at least one of a shape, a size, and a number of neighboring positions.

[0208] The submode index information in this embodiment can be represented in many ways. In the example above, the current block uses a single filter window (4×4) and six reconstruction regions (two each for the upper reconstruction region, the left reconstruction region, and the L-shaped reconstruction region) for intra-frame prediction based on interpolation filtering. There are a total of six filter window and reconstruction region combinations, resulting in six submodes. The submode index is the index obtained by uniformly numbering all submodes. For example, if the first submode has the lowest rate-distortion cost, the submode index is 0; if the fifth submode has the lowest rate-distortion cost, the submode index is 4. Because there is only one filter window, the submode index in this example can also be considered a reconstruction region index. In addition to uniformly numbering submodes, in other examples, the filter windows and reconstruction regions involved in the combination can also be numbered separately to obtain their own indices. For example, if the current block uses three filter windows (see Figures 7A to 7C) and six reconstruction regions, if the second filter window and the fourth reconstruction region have the lowest rate-distortion cost, a filter window index of 1 and a reconstruction region index of 3 can be encoded. In addition, more detailed parameter indexes can be used to represent the index information of the sub-mode. For example, when the current block uses a filter window and 6 reconstruction areas, and the 6 reconstruction areas have 3 types, namely the upper reconstruction area, the left reconstruction area, and the L-shaped reconstruction area, and each type has 2 reconstruction areas of different sizes (such as K=L=8 and K=L=13), a 2-bit reconstruction area type index can be used to represent the type of the selected sub-mode, and a 1-bit size index can be used to represent the K and L values ​​of the selected sub-mode, so as to uniquely determine the reconstruction area in the combination corresponding to the selected sub-mode. When the current block uses multiple filter windows and the multiple filter windows are divided based on multiple parameters, the parameter index of the filter window can also be used to represent the filter window in the combination corresponding to the selected sub-mode, which will not be repeated here.

[0209] Before deciding the prediction mode and transform mode for each intra-frame coding block, the disclosed embodiment determines whether the current block can use the intra-frame prediction technology of the interpolation filter mode. If the current sequence level flag and other related flags allow the interpolation filter prediction technology, and the size, position and other conditions of the current block meet the requirements, the filter coefficients of the corresponding sub-mode are obtained based on each combination of the filter window of the current block and the reconstructed area adjacent to the current block. The current block is then predicted point by point based on the filter coefficients of the sub-mode to obtain the predicted block of the current block.

[0210] The cost of the various intra prediction modes for the current block can be compared in two stages to select a mode for the current block. In the coarse selection stage of the prediction mode for the current block, the cost of each potential intra prediction mode (including each sub-mode of the interpolation filter mode) is calculated. The cost calculation formula is as follows: cost = D + λR

[0211] Where R represents the bit overhead expected for the intra-frame prediction mode, λ is the Lagrange multiplier, which is related to the quantization parameter used in the current encoding, and D represents the distortion value between the predicted block and the original block in the current prediction mode. The distortion value can be calculated as follows: D = min(SAD × 2, SATD)

[0212] Among them, SAD represents the sum of absolute difference between the predicted block and the original block, and SATD represents the sum of transformed difference between the predicted block and the original block.

[0213] The prediction modes that have passed the coarse screening are then fully transformed, quantized, inversely quantized, inversely transformed, and reconstructed. The rate-distortion cost of each mode combination (i.e., prediction mode + transform mode + quantization mode) is compared to determine the final prediction mode, transform mode, and quantized residual value. At this stage, the rate-distortion cost calculation is still cost = D + λR, but here D represents the sum of squared error (SSE) between the reconstructed block and the original block, and R represents the total bit overhead of encoding the current block's mode identifier, coefficients, and so on.

[0214] The rate-distortion cost algorithm used in the present disclosure is not limited thereto. For example, the error in the rate-distortion cost formula may also be represented by mean absolute difference (MAD), mean squared error (MSE), and the like.

[0215] An embodiment of the present disclosure further provides a video decoding method, as shown in FIG12 , including:

[0216] Step S310, decoding the interpolation filter mode flag of the current block;

[0217] Step S320 , determining, based on the decoded interpolation filter mode flag, that the current block uses the interpolation filter mode for intra-frame prediction, and determining the filter window and reconstruction area selected for the current block;

[0218] Step S330 : Based on the determined filtering window and reconstruction area, perform intra-frame prediction on the current block according to the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure.

[0219] The above-mentioned filter window and reconstruction region are selected for the current block, i.e., a filter window and a reconstruction region from a combination (corresponding to the selected sub-mode) selected by the encoder for the current block. If the encoder's interpolation filtering mode offers multiple combinations of filter windows and reconstruction regions, the encoder can encode the index information of the selected combination into the bitstream. The decoder only needs to perform interpolation filtering based on one filter window and reconstruction region combination, eliminating the need to select multiple combinations.

[0220] The video decoding method of the disclosed embodiment can determine whether to use the interpolation filter mode when performing intra-frame prediction on the current block by parsing the syntax elements related to the interpolation filter mode of the current block. By introducing the interpolation filter mode, the efficiency of the entire video encoding and decoding can be improved.

[0221] In an exemplary embodiment of the present disclosure, before decoding the interpolation filter mode flag of the current block, the method further includes any one or more of the following processes:

[0222] Decoding a sequence-level flag indicating whether an interpolation filtering mode is allowed, and if the value of the sequence-level flag indicates that the interpolation filtering mode is allowed, continuing to decode the interpolation filtering mode flag of the current block; and if the value of the sequence-level flag indicates that the interpolation filtering mode is not allowed, skipping decoding the interpolation filtering mode flag of the current block;

[0223] decoding a picture-level flag indicating whether an interpolation filtering mode is allowed to be used, and if the value of the picture-level flag indicates that the interpolation filtering mode is allowed to be used, continuing to decode the interpolation filtering mode flag of the current block; and if the value of the picture-level flag indicates that the interpolation filtering mode is not allowed to be used, skipping decoding the interpolation filtering mode flag of the current block;

[0224] decoding a slice-level flag indicating whether an interpolation filtering mode is allowed, and continuing to decode the interpolation filtering mode flag of the current block only if the value of the slice-level flag indicates that the interpolation filtering mode is allowed, and skipping decoding the interpolation filtering mode flag of the current block if the value of the slice-level flag indicates that the interpolation filtering mode is not allowed;

[0225] Decode the general restriction information GCI identifier indicating whether the use of the interpolation filtering mode is prohibited. When the value of the GCI identifier indicates that the use of the interpolation filtering mode is not prohibited, continue decoding the sequence-level flag indicating whether the use of the interpolation filtering mode is allowed. When the value of the GCI identifier indicates that the use of the interpolation filtering mode is prohibited, skip decoding the sequence-level flag indicating whether the use of the interpolation filtering mode is allowed. By default, the sequence-level flag indicates that the use of the interpolation filtering mode is not allowed.

[0226] In an example of this embodiment, a sequence-level flag is used to indicate whether the interpolation filtering mode is allowed. The relevant syntax elements are shown in the following table:

[0227] Sequence parameter set (SPS) syntax element identifier (sequence parameter set syntax)

[0228] In the above table, sps_eip_enabled_flag equal to 1 indicates that the interpolation filtering mode is allowed, that is, it is defined that the interpolation filtering intra-frame prediction technology is allowed to be turned on in the coding layer video sequence (CLVS: coded layer video sequence); sps_eip_enabled_flag equal to 0 indicates that the interpolation filtering mode is not allowed, that is, it is defined that the interpolation filtering intra-frame prediction technology is turned off in the coding layer video sequence.

[0229] In an example of this embodiment, a GCI flag is used to indicate whether to prohibit the use of the interpolation filtering mode. The relevant syntax elements are shown in the following table:

[0230] General constraints information (GCI) syntax element identifier

[0231] In the above table, gci_no_eip_constraint_flag equal to 1 indicates that the interpolation filter mode is prohibited, that is, it defines that the sequence-level interpolation filter intra-frame prediction technology must be 0 in all images and is not allowed to be used; gci_no_eip_constraint_flag equal to 0 indicates that the interpolation filter mode is not prohibited, and the restriction of prohibiting the use of the interpolation filter mode will not be introduced.

[0232] In an example of this embodiment, a picture-level flag is used to indicate whether the interpolation filter mode is allowed. The relevant syntax elements are shown in the following table:

[0233] Picture header structure syntax element identification (picture header structure syntax)

[0234] In the above table, ph_eip_enabled_flag equal to 1 indicates that the interpolation filtering mode is allowed, that is, it defines that the interpolation filtering intra-frame prediction technology is allowed to be turned on in the current image; ph_eip_enabled_flag equal to 0 indicates that the interpolation filtering mode is not allowed, that is, it defines that the interpolation filtering intra-frame prediction technology is turned off in the current image. If this syntax element does not exist in the current sequence, the default value is 0.

[0235] The picture-level flag indicating whether the interpolation filter mode is allowed to be used may be a syntax identifier in the picture header structure or a picture parameter set syntax original identifier (PPS: picture parameter set syntax).

[0236] In an example of this embodiment, a slice-level flag is used to indicate whether the interpolation filtering mode is allowed. The relevant syntax elements are shown in the following table:

[0237] Slice header syntax element identifier (slice header syntax)

[0238] In the above table, sh_eip_enabled_flag equal to 1 indicates that the interpolation filtering mode is allowed, that is, it is defined that the interpolation filtering intra-frame prediction technology can be used for the current slice; sh_eip_enabled_flag equal to 0 indicates that the interpolation filtering mode is not allowed, that is, it is defined that the interpolation filtering intra-frame prediction technology cannot be used for the current slice.

[0239] In an exemplary embodiment of the present disclosure, determining a filter window selected for the current block includes:

[0240] Determine a set filter window as the filter window for the current block; this is for the case where both the encoder and decoder use the same filter window when performing intra-frame prediction in interpolation filter mode on different current blocks, for example, a 4x4 filter window is used. In this case, the encoder does not need the index information of the encoding filter window, and the decoder can directly determine the set filter window as the filter window selected for the current block; or

[0241] Determining a filter window corresponding to block parameters of the current block as the filter window selected for the current block, wherein the block parameters include at least one of a position, a shape, and a size; this is a filter window that can be used when the encoder and decoder can determine the intra-frame prediction of the current block in an interpolation filter mode based on the block parameters of the current block; in this case, index information of the encoding filter window is not required, and the decoder determines the filter window selected for the current block based on the block parameters of the current block; or

[0242] Continue decoding the submode index of the current block, and determine the filter window selected for the current block according to the submode index, where each submode index corresponds to a combination of the filter window of the current block and a reconstructed area adjacent to the current block;

[0243] Continue decoding the filter window index of the current block, and determine the filter window selected for the current block according to the filter window index, where each filter window index corresponds to a filter window; or

[0244] Continue decoding a parameter index of a filter window of the current block, determine parameters of the filter window according to the parameter index of the filter window, and determine the filter window selected for the current block according to the parameters, wherein the parameters of the filter window include at least one of a shape, a size, and a number of neighboring positions.

[0245] In an exemplary embodiment of the present disclosure, determining a neighboring reconstruction area selected for the current block includes:

[0246] Determine the reconstructed region selected for the current block based on the parameters of the reconstructed region set, which is for the case where both the encoder and decoder use a reconstructed region with pre-set parameters when performing intra-frame prediction of interpolation filter modes on different current blocks, and does not require decoding index information; or

[0247] Determine a reconstruction area corresponding to the block parameters of the current block as the reconstruction area adjacent to the current block, wherein the block parameters include at least one of position, shape, and size; this is a case where the encoder and decoder can determine a reconstruction area that can be used when the current block is intra-frame predicted in the interpolation filter mode based on the block parameters of the current block; for example, when the L-shaped reconstruction area shown in FIG8C is used and K=L=13, index information of the reconstruction area is not required in this case, and the decoder can determine the filter window selected for the current block based on the block parameters of the current block and the set type, K, and L values; or

[0248] Continue decoding the submode index of the current block, and determine the reconstructed region selected for the current block according to the submode index, where each submode index corresponds to a combination of the filter window of the current block and a reconstructed region adjacent to the current block; or

[0249] Continue decoding the reconstruction region index of the current block, and determine the reconstruction region selected for the current block according to the reconstruction region index, where each reconstruction region index corresponds to a reconstruction region; or

[0250] Continuing to decode a parameter index of a reconstructed region of the current block, determining parameters of the reconstructed region based on the parameter index of the reconstructed region, and determining the reconstructed region selected for the current block based on the parameters. In one example, the parameter index of the reconstructed region includes at least one of a size index of the reconstructed region and a type index of the reconstructed region, and the size of the reconstructed region includes at least one of the number of rows of the reconstructed region above the current block and the number of columns of the reconstructed region to the left of the current block.

[0251] The above sub-mode index, reconstruction area index, filter window index, reconstruction area parameter index and filter window parameter index can be found in the corresponding description of the encoding method. The above size index can also include multiple indexes such as row number index and column number index.

[0252] It should be noted that when determining the filter window and reconstruction area selected for the current block, the filter window and the adjacent reconstruction area can be determined simultaneously according to the sub-mode index, or they can be determined separately, that is, the filter window is determined according to the setting information or index information of the filter window, and the adjacent reconstruction area is determined according to the setting information or index information of the reconstruction area.

[0253] For a current block predicted intra-frame, the surrounding reconstructed parts (i.e., the adjacent reconstructed areas) may include the areas directly above, to the upper left, to the upper right, to the left, and to the lower left of the current block. Due to the different sizes, positions, and divisions of the blocks, not all of these five areas will have reconstructed pixels. For example, when the current block is at the top edge of the image, there will be no reconstructed pixels above (including directly above, to the upper left, and to the upper right) the current block. When the current block is at the left edge of the image, there will be no reconstructed pixels to the left of the current block (including the upper left, to the left, and to the lower left). Therefore, this situation can be avoided by limiting the difference between the coordinates of the upper left corner of the current block and the coordinates of the upper left corner of the image.

[0254] As shown in Figures 14A and 14B, Figures 14A and 14B are examples of two types of ternary tree partitioning. In these two partitioning cases, due to the different positions, when the lower left area and the upper right area of ​​the current block are in the current CTU, it may cause the farther reference lines in the reconstructed area to have reconstructed values ​​(i.e., reconstructed pixels), while the closer reference lines do not have reconstructed pixels; when the lower left area and the upper right area are both in another CTU, the reference lines in the reconstructed area have reconstructed values.

[0255] In an exemplary embodiment of the present disclosure, before decoding the interpolation filter mode flag of the current block, the method further includes any one or more of the following processes:

[0256] determining whether the size of the current block satisfies a maximum block size and / or a minimum block size for enabling an interpolation filtering mode, and if so, continuing to decode the interpolation filtering mode flag of the current block; if not, skipping decoding the interpolation filtering mode flag of the current block; wherein the size includes at least one of size (i.e., the number of pixels included), width, and height; and setting the maximum block size condition can avoid excessive error accumulation;

[0257] determining whether a difference between the horizontal coordinate of the upper left corner of the current block and the horizontal coordinate of the upper left corner of the current image is greater than or equal to a set first threshold, and if the difference is greater than or equal to the set first threshold, continuing to decode the interpolation filter mode flag of the current block; and if the difference is less than the first threshold, skipping decoding the interpolation filter mode flag of the current block;

[0258] Determine whether the difference between the vertical coordinate of the upper left corner of the current block and the vertical coordinate of the upper left corner of the current image is greater than or equal to a set second threshold. If the difference is greater than or equal to the set second threshold, continue decoding the interpolation filter mode flag of the current block. If the difference is less than the second threshold, skip decoding the interpolation filter mode flag of the current block.

[0259] In addition to using high-level syntax elements to indicate whether the interpolation filtering mode is allowed, this embodiment uses block-level syntax elements to restrict the use of the interpolation filtering mode. These block-level syntax elements can be at the same decoding level as syntax elements related to other block-level intra-frame prediction modes. In VVC, the parsing of the block-level intra-frame prediction mode identifier is at the coding unit (CU) level. In some cases or other standards, the prediction mode identifier can also be at the prediction unit (PU) level. The following takes VVC as an example to illustrate the block-level syntax elements related to the interpolation filtering mode at the decoding end (the ellipsis in the table represents the decoding of some other block-level syntax elements, such as other intra-frame prediction modes, etc.).

[0260] In one example, the encoding code includes three reconstruction areas that can be used to obtain filter coefficients of the current block. The types and sizes of the three reconstruction areas are shown in Figures 13A to 13C, where M and N represent the width and height of the current block respectively.

[0261] The decoding end parses the block-level syntax elements of the current coding unit as shown in the following table:

[0262] In the table above, eip_tpl_type_idx is the type index of the reconstruction region of the current block, defining the type of reconstruction region selected by the encoder for obtaining filter coefficients for the current block. It consists of two bits. x0 and y0 in the table represent the horizontal and vertical coordinate differences between the upper left corner of the current block (in this example, the current coding unit) and the upper left corner of the image. XX and YY represent the coordinates of the upper left corner of the current block in the image that must be met to enable (i.e., allow the use of) the interpolation filtering mode. When the coordinates of the upper left corner of the image are (0,0), XX and YY are the first and second thresholds described above. In the table above, SIZE_A represents the maximum block size requirement required to enable the interpolation filtering mode. In one example, SIZE_A is set to 1024, and XX and YY are set to 13. If there are insufficient reconstructed rows or columns to the left or above the current block, the usage condition is not met.

[0263] According to the above syntax elements, the decoding end will continue to decode the block-level interpolation filter mode flag intra_eip_flag only when the sequence-level flag sps_eip_enabled_flag indicates that the interpolation filter mode is allowed, the size of the current block (width multiplied by height, i.e., cbWidth*cbHeight) is less than or equal to SIZE_A, the horizontal coordinate x0 of the upper left corner of the current block is greater than or equal to XX, and the vertical coordinate of the upper left corner of the current block is greater than or equal to YY. Based on intra_eip_flag, it is determined whether the current block uses the interpolation filter mode for intra-frame prediction. If intra_eip_flag is 1, indicating that the current block uses the interpolation filter mode, the decoding end will continue to decode the reconstruction area type flag eip_tpl_type_idx to determine the type of the reconstruction area selected for the current block. Because the number of rows and columns of the reconstruction area above the current block in this example is both 13, and a filtering window is used, the unique reconstruction area and a combination of the filtering window and the reconstruction area can be directly determined according to eip_tpl_type_idx. The eip_tpl_type_idx can also be regarded as a reconstruction area index and a sub-mode index.

[0264] In this example, the correspondence between eip_tpl_type_idx and the three reconstruction areas shown in FIG. 13A to FIG. 13C is as follows:

[0265] In another example, the encoder uses six reconstruction regions for the current block, as shown in Figures 13A to 13F. This is based on the previous example where K=L=13, with the addition of an upper reconstruction region, a left reconstruction region, and an L-shaped reconstruction region with K=L=8. The decoder then parses the block-level syntax elements of the current coding unit as shown in the following table:

[0266] In the table above, the meanings of sps_eip_enabled_flag, eip_tpl_type_idx, intra_eip_flag, SIZE_A, XX, and YY are the same as in the previous example and are not repeated here. In this example, XX and YY are set to 13. Because other intra-frame prediction techniques in current ECM use a maximum of 13 rows and 13 columns of reference rows, intra-frame interpolation filtering prediction uses 13 rows and 13 columns. This setting does not meet the requirements when there are insufficient reconstructed rows and columns to the left or above the current block.

[0267] Unlike the previous example, this example uses two parameter indexes for the reconstruction area to indicate the selected reconstruction area among the six optional reconstruction areas. In addition to using the reconstruction area type index eip_tpl_type_idx to indicate the type of reconstruction area selected for the current block, the 1-bit reconstruction area size index eip_tpl_size_idx is also used to indicate the number of rows above and columns to the left of the selected reconstruction area above the current block. Because the number of rows and columns is equal, eip_tpl_size_idx can also be regarded as a row index and a column index. If the number of rows and columns varies significantly, separate row and column indices can also be used.

[0268] In this example, if eip_tpl_size_idx is 0, it means that the number of rows above and columns to the left of the current block used for reconstruction is 8. If it is 1, it means that the number of rows above and columns to the left of the current block used for reconstruction is 13. This example uses eip_tpl_type_idx and eip_tpl_size_idx to uniquely identify the reconstruction area selected for the current block. Because there is only one filter window, it can also uniquely identify the sub-mode of the interpolation filter mode selected for the current block.

[0269] The corresponding relationship between the values ​​of eip_tpl_type_idx and eip_tpl_size_idx and the reconstruction area in this example is shown in the following table:

[0270] This embodiment does not limit the decoded binary identifier to be context-based or equiprobable, but in some embodiments, better compression effect can be achieved by changing the binarization method of the sub-mode or changing the probability model of the binary identifier representing the sub-mode.

[0271] This example includes the following steps when determining a reconstruction area selected for the current block:

[0272] Continuing to decode the type index of the reconstruction area of ​​the current block, and determining the type of the reconstruction area selected for the current block according to the decoded type index, the type being an upper reconstruction area, a left reconstruction area, or an L-shaped reconstruction area;

[0273] Continue decoding the size index of the reconstructed region of the current block, and determine the size of the reconstructed region selected for the current block according to the decoded size index, such as the number of rows above the current block and the number of columns to the left of the current block, for example, K=L=8 or K=L=13;

[0274] The reconstruction area selected for the current block is determined according to the type and size of the reconstruction area adjacent to the current block, and the position and size of the current block.

[0275] In another example, the encoder uses one reconstruction region for the current block, for example, an L-shaped reconstruction region as shown in FIG13C and K=L=13, and uses a 4×4 window as the filtering window. In this case, the decoder parses the block-level syntax elements of the current coding unit as shown in the following table:

[0276] In the above table, intra_eip_flag represents the interpolation filter mode flag for the block level (CU). When intra_eip_flag is equal to 1, it indicates that the current block (CU) uses the intra prediction technology of the interpolation filter mode. When it is equal to 0, it indicates that the current block (CU) does not use the intra prediction technology of the interpolation filter mode. When this syntax element is not present in the current bitstream, it defaults to 0, that is, the interpolation filter mode is not used. The above table omits the processing of other syntax elements of the interpolation filter mode when intra_eip_flag is equal to 1, such as the decoding of related indexes. The meaning of sps_eip_enabled_flag is the same as in the above example.

[0277] This example allows the use of interpolation filtering mode for blocks of all sizes. If the use of interpolation filtering intra-frame prediction is based on a block size restriction, it should be determined whether the size of the current block meets the requirements before further decoding the interpolation filtering intra-frame prediction block-level identification bit.

[0278] For example, in another example, if the width of the current block needs to be less than MAX_WIDTH and the height needs to be less than MAX_HEIGHT, the block-level syntax elements are parsed as shown in the following table:

[0279] In the above table, cbWidth < MAX_WIDTH and cbHeight < MAX_HEIGHT are two conditions for the decoding interpolation filter mode flag intra_eip_flag. Another condition is that the sequence-level flag sps_eip_enabled_flag indicating whether to allow the use of the interpolation filter mode is 1. If any of the conditions is not met, the decoding of intra_eip_flag is skipped.

[0280] For another example, in another instance, the width of the current block needs to be greater than MIN_WIDTH and the height needs to be greater than MIN_HEIGHT. Then, when parsing the block-level syntax elements, it is as shown in the following table:

[0281] In the above table, cbWidth > MIN_WIDTH and cbHeight > MIN_HEIGHT are two conditions for the decoding interpolation filter mode flag intra_eip_flag. Another condition is that the sequence-level flag sps_eip_enabled_flag indicating whether to allow the use of the interpolation filter mode is 1. If any of the conditions is not met, the decoding of intra_eip_flag is skipped.

[0282] For another example, in another instance, the width of the current block needs to be less than MAX_WIDTH and greater than MIN_WIDTH, and the height needs to be less than MAX_HEIGHT and greater than MIN_HEIGHT. Then, when parsing the block-level syntax elements, it is as shown in the following table:

[0283] In the table, MIN_WIDTH < cbWidth, cbWidth < MAX_WIDTH, MIN_HEIGHT < cbHeight, and cbHeight < MAX_HEIGHT are four conditions for the decoding interpolation filter mode flag intra_eip_flag. Another condition is that the sequence-level flag sps_eip_enabled_flag indicating whether to allow the use of the interpolation filter mode is 1. If any of the conditions is not met, the decoding of intra_eip_flag is skipped.

[0284] In one example, when the encoder performs interpolation filtering on the current block, multiple filtering windows and multiple reconstruction regions can be used. The parameters of the filtering window include the number of neighboring positions (which can also be referred to as the number of filter coefficients) and the shape (3 types as shown in FIGS. 7A to 7C). In this example, the selected filtering window and reconstruction region are indicated by the parameter index of the filtering window and the parameter index of the reconstruction region. When parsing the block-level syntax elements, it is as shown in the following table:

[0285] In the above table, eip_shape_idx is the filter window shape index, which defines the index of the selected filter shape. eip_num_coeff_idx is the filter window neighbor number index, which defines the number of neighboring positions of the selected filter window. For example, if eip_num_coeff_idx is 0, the number of neighboring positions of the filter window is 15, and if eip_num_coeff_idx is 1, the number of neighboring positions of the filter window is 24. When there is only one filter window shape in the combination corresponding to the sub-mode, the filter window shape index can be omitted in the syntax table, and the same applies to the filter window neighbor number.

[0286] In the above table, eip_tpl_type_idx is the type index of the reconstruction area, and eip_tpl_size_idx is the size index of the reconstruction area, which is the same as the above embodiment and will not be repeated here.

[0287] In another example, instead of using multiple syntax elements to indicate the filter window and reconstruction area selected for the current block, a single syntax element such as a sub-mode index is used to directly indicate the selected filter window and reconstruction area for the current block. The following table shows the syntax elements when parsing the block level:

[0288] The eip_mode_idx in the table above is the sub-mode index of the interpolation filter mode selected for the current block. Each sub-index corresponds to a combination of reconstruction region and filter window. The number of possible combinations is determined by the type, number of rows and columns of the available reconstruction region, as well as the shape and number of neighboring locations of the filter window. Therefore, the sub-mode index can be used to determine the reconstruction region and filter window selected for the current block. Specifically, the combination corresponding to the eip_mode_idx value can be determined by table lookup or calculation.

[0289] In an exemplary embodiment of the present disclosure, the video decoding method further includes: decoding an intra-frame sub-block partitioning ISP mode flag;

[0290] When the ISP mode flag indicates that the current coding unit uses the ISP mode, the current block for which interpolation filtering is performed is a sub-block obtained by dividing the current coding unit;

[0291] When the ISP mode flag indicates that the current coding unit does not use the ISP mode, the current block on which interpolation filtering is performed is the current coding unit.

[0292] In this embodiment, the interpolation filter mode and the intra sub-block partitioning (ISP) technology are used together. When decoding the syntax elements of the intra mode, it is necessary to parse the interpolation filter mode and the intra sub-block partitioning mode at the same time. An example of parsing the block-level intra prediction mode is as follows:

[0293] According to the table above, for a current coding unit where both the ISP mode flag intra_subpartitions_mode_flag and the interpolation filter mode flag intra_eip_flag are enabled (i.e., enabled), and ISP is selected, the unit can be further divided into multiple sub-blocks. In this case, the interpolation filter mode is used for intra-frame prediction for each sub-block. During interpolation filtering, the filter coefficients can be shared by the sub-blocks or obtained separately for each sub-block.

[0294] In an exemplary embodiment of the present disclosure, after performing intra-frame prediction on the current block, the video decoding method further includes:

[0295] Decode the position-dependent prediction combination (PDPC) mode flag;

[0296] When the PDCP mode flag indicates that the current block uses the PDCP mode, the prediction block of the current block obtained by interpolation filtering is weighted using the unsmoothed reconstructed values ​​of the pixels in at least one row and one column adjacent to the current block.

[0297] This embodiment uses the PDPC technique to further optimize the blocks predicted by interpolation filtering. In some embodiments, the PDPC technique may be used by default for all prediction blocks obtained by interpolation filtering. In some embodiments, whether to use the PDPC technique may be determined based on the size and shape of the block. In some embodiments, the PDPC mode flag eip_pdpc_flag may be parsed from the bitstream to determine whether to use the PDPC technique for the current block. An example of the corresponding syntax element parsing is shown in the following table:

[0298] In the table above, eip_pdpc_flag equal to 1 indicates that the PDPC mode (i.e., PCPC technology) is used for the current block, and equal to 0 indicates that the PDPC mode is not used for the current block. When this syntax element does not exist in the bitstream, it defaults to 0, that is, the PDPC mode is not used for the current block. In some embodiments, eip_pdpc_flag can be parsed based on conditions such as block size and shape.

[0299] In an exemplary embodiment of the present disclosure, after performing intra-frame prediction on the current block, the video decoding method further includes: performing intra-frame prediction on the current block according to one or more intra-frame prediction modes different from the interpolation filtering mode, and weighting the prediction block of the current block obtained based on the one or more intra-frame prediction modes and the prediction block of the current block obtained based on the interpolation filtering mode to obtain the final prediction block of the current block. In this embodiment, the interpolation filtering mode can be used together with any one or several intra-frame prediction modes, and the final prediction signal is obtained by weighting the prediction result of the interpolation filter with the prediction results of other prediction modes. The weights used for weighting can be fixed weights or derived weights. When there are multiple combinations of weights, the index of the weights can be parsed from the bitstream.

[0300] In an exemplary embodiment of the present disclosure, after performing intra-frame prediction on the current block, the video decoding method further includes: determining a residual of the current block according to an original value and a predicted value of the current block, and performing transformation and inverse transformation on the residual; wherein:

[0301] When transforming and inversely transforming the residual, the transformation kernel used is a transformation kernel obtained by training based on the residual samples in the interpolation filtering mode; or

[0302] When performing transformation and inverse transformation on the residual, limiting the number of combinations of primary transformation kernels and secondary transformation kernels; or

[0303] When transforming and inversely transforming the residual, limiting the number of combinations of primary transform kernels and secondary transform kernels according to the size and / or shape of the current block; or

[0304] When transforming and inversely transforming the residual, the transform kernel adopted when performing intra-frame prediction based on the planar mode is used.

[0305] In this embodiment, since the interpolation filter mode predicts that the predicted value of the previous pixel in the current block is used as the input of the interpolation filter of the next pixel to be predicted, this will cause the prediction error to accumulate, making the distribution of the residual different from other existing prediction modes. This means that when the residual of the interpolation filter mode is transformed and inversely transformed using an inseparable primary transform or an inseparable secondary transform, the transform kernel used can be a predefined transform kernel trained based on the interpolation filter error. In some embodiments, since the residual distribution of the interpolation filter mode is different from other existing prediction modes, the number of combinations of the primary transform kernel and the secondary transform kernel of the residual can be limited, for example, only DCT transform is allowed to be used for the primary transform. In some embodiments, since the distribution of the residual of the interpolation filter mode is different from other existing prediction modes, the number of combinations of the primary transform kernel and the secondary transform kernel of the residual can be limited according to the block parameters (such as size and shape) of the current block.

[0306] In an exemplary embodiment of the present disclosure, after performing intra-frame prediction on the current block, the video decoding method further includes: determining the residual of the current block based on the original value and the predicted value of the current block, skipping the transformation of the residual using a transform skip mode, and performing block-by-block quantization, dequantization, and reconstruction on the current block; or determining the residual of the current block based on the original value and the predicted value of the current block, skipping the transformation of the residual using a transform skip mode, and performing point-by-point quantization, dequantization, and reconstruction on the pixels of the current block.

[0307] In this embodiment, because the residual distribution of the interpolation filtering mode differs from that of other existing prediction modes, a transform skip mode can be used to skip the transformation of the interpolation filtering prediction mode residual. When skipping the transformation of the interpolation filtering prediction mode residual, the current block can be quantized, dequantized, and reconstructed block by block, or quantized, dequantized, and reconstructed pixel by pixel. This avoids the drawback of using the current position prediction value as the filter input for the next position, which causes excessive error accumulation.

[0308] In an exemplary embodiment of the present disclosure, after performing intra-frame prediction on the current block, the video decoding method further includes: when the current block is used as a neighbor block of other coding blocks to construct a most probable mode (MPM) list, using the planar mode as the intra-frame prediction mode used by the current block.

[0309] In an exemplary embodiment of the present disclosure, decoding of the interpolation filter mode flag of the current block is allowed only when the current block is a luminance block; or, decoding of the interpolation filter mode flag of the current block is allowed only when the current block is a luminance block or a chrominance block.

[0310] In an exemplary embodiment of the present disclosure, the current block is a luminance block; after performing intra-frame prediction on the current block using the interpolation filter mode, the method further includes:

[0311] When the chroma block at the position of the current block adopts the DM mode, determining that the mode for intra-frame prediction of the chroma block is the planar mode (i.e., the Planar mode), in which case the chroma block does not use the interpolation filter mode; or

[0312] When the chroma block at the position of the current block adopts the DM mode, determining that the mode for intra-frame prediction of the chroma block is the interpolation filter mode, in which case the chroma block is allowed to use the interpolation filter mode; or

[0313] If the chroma block at the current block's location uses the DM mode, the interpolation filter mode flag of the chroma block is decoded, and the value of the interpolation filter mode flag is used to determine whether the intra-frame prediction mode for the chroma block is the interpolation filter mode. In this case, the interpolation filter mode is allowed for the chroma block, but the interpolation filter mode flag of the chroma block is used to determine whether to use it.

[0314] The following takes the intra-frame prediction based on interpolation filtering performed by the decoding end as an example to illustrate the specific implementation methods of some processes performed by the above embodiments.

[0315] ■Get the reconstruction area range and maximum and minimum values

[0316] When obtaining the filter coefficients, first decode the size index eip_tpl_size_idx of the reconstruction area of ​​the current block to determine the number of rows and columns of the reconstruction area selected for the current block, assuming it is refSize. And based on the coordinates puX and puY of the pixel point at the upper left corner of the current block, the width puWidth and height puHeight of the current block are obtained as follows:

[0317] The width of the reference area refWidth = puWidth * 2 + refSize

[0318] Reference area height refHeight = puHeight * 2 + refSize

[0319] The horizontal coordinate of the upper left corner of the reference area refPosPicX = puX-refSize

[0320] The vertical coordinate of the upper right corner of the reference area refPosPicY = puY-refSize

[0321] Based on the coordinates and size of the reference region, the reference region's refBuffer is retrieved from the reconstructed image buffer. ref[x][y] represents the reconstructed pixel value in the refBuffer at the top-left corner (x, y) of the reference region. To avoid division when calculating the mean, the reconstructed value of the pixel at the intersection of the first row above and the first side to the left of the current block is used as the mean.

[0322] Mean mean = ref[refSize-1][refSize-1]

[0323] The process of obtaining the minimum and maximum values ​​is as follows:

[0324] ■ Solving the filter coefficients

[0325] The filtering window uses a 4×4 rectangular window to obtain a set of filter coefficients for interpolation filtering based on the reconstruction area selected for the current block, including the following steps:

[0326] first step

[0327] Use arrays A[][] and C[] to cache the reconstructed samples of the corresponding area, where the first index of array A represents the index of the sample position, the second index represents the index of multiple adjacent positions in the filter window after aligning the sample position represented by the first index with the current position of the filter window (also known as the input sample index), and the index of array C represents the index of the filter coefficient output by the interpolation filter (also known as the output sample index). The process of obtaining elements in arrays A and C is as follows:

[0328] In this example, the value of refType is equal to eip_tpl_type_idx, numSamples indicates the number of sample positions in the reconstruction area, and the step size of the filter window movement in this example is 1 in both the width and height directions. refHeight represents the height of the reconstruction area, and refWidth represents the width of the reconstruction area.

[0329] Step 2

[0330] Based on the array A[], construct the autocorrelation coefficient matrix ATA[][] (note that as can be seen above, the coefficients of the autocorrelation coefficient matrix are symmetrical along the diagonal direction, so some calculations can be omitted during construction) and the cross-correlation coefficient matrix ATY[]. The steps are as follows:

[0331] In order to solve the filter coefficients, in addition to constructing the Wiener filter equation to solve it, other new algorithm methods can also be used to solve the filter coefficients.

[0332] Step 3

[0333] Decompose the autocorrelation coefficient matrix and back-substitute to solve for the interpolation filter coefficients. This can be achieved using Kresky decomposition or lower unit triangle (LDL) decomposition.

[0334] ■Prediction process based on interpolation filtering

[0335] The input of this process is: the buffer of the block to be predicted piPred, the width width and height height of the prediction block, the interpolation filter coefficient C={c0,…,c 14}, minimum value min, maximum value max, mean value mean, and reconstructed pixel buffer refBuffer. Cache tempBuff[MAX_CU_SIZE+3][MAX_CU_SIZE+3] used to temporarily store reconstructed pixel values ​​and intermediate values ​​in the prediction process,

[0336] The process of generating predicted values ​​is as follows:

[0337] Step 1. Fill the reconstructed pixels into tempBuff, which can be achieved by the calculation process shown in the following table:

[0338] Step 2. Complete the prediction

[0339] In the table above, shift is the magnification factor when calculating the filter coefficient, and offset is the offset value. (The magnification factor is used to allow the interpolation filter coefficient to have a larger dynamic range, so it is magnified when obtaining the coefficient.)

[0340] ECM (enhanced compression model) is a reference software based on the VVC test software VTM-10.0, integrating various new tools to further explore codec performance. Based on the intra-frame prediction method described in the embodiments of this disclosure, using the six reference regions and a 4×4 filter window as shown in Figures 13A to 13F, the performance on the ECM-6.0 reference software is as follows:

[0341] The meanings of the parameters in the table are as follows:

[0342] EncT: Encoding Time, 10X% means that after the reference row sorting technology is integrated, the encoding time is 10X% compared with before integration, which means that the encoding time increases by X%.

[0343] DecT: Decoding Time, 10X% means that after the reference row sorting technology is integrated, the decoding time is 10X% compared with before integration, which means that the decoding time increases by X%.

[0344] ClassA1 and ClassA2 are test video sequences with a resolution of 3840x2160, ClassB is a test sequence with a resolution of 1920x1080, ClassC is 832x480, ClassD is 416x240, and ClassE is 1280x720; ClassF is a screen content sequence with several different resolutions.

[0345] Y, U, V are the three color components, and the columns where Y, U, V are located represent the BD-rate of the test results on Y, U, V. rate) indicator, the smaller the value, the better the encoding performance.

[0346] All intra indicates the test configuration of all intra-frame configurations.

[0347] Using the interpolation filtering mode of the embodiment of the present disclosure for intra-frame prediction coding and decoding brings an objective compression effect improvement of 0.15% / 0.13% / 0.15% for the Y / U / V components respectively under general test conditions.

[0348] An embodiment of the present disclosure further provides a code stream, wherein the code stream is generated by the video encoding method of any embodiment of the present disclosure.

[0349] An embodiment of the present disclosure also provides an intra-frame prediction device based on interpolation filtering, as shown in Figure 15, including a processor 71 and a memory 73 storing a computer program, wherein when the processor 71 executes the computer program, it can implement the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure.

[0350] An embodiment of the present disclosure further provides a video decoding device, see FIG15 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video decoding method as described in any embodiment of the present disclosure when executing the computer program.

[0351] An embodiment of the present disclosure further provides a video encoding device, see FIG15 , comprising a processor and a memory storing a computer program, wherein the processor can implement the video encoding method as described in any embodiment of the present disclosure when executing the computer program.

[0352] The processors of the above embodiments of the present disclosure may be general-purpose processors, including central processing units (CPUs), network processors (NPs), microprocessors, etc., or other conventional processors; the processors may also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), discrete logic or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or other equivalent integrated or discrete logic circuits, or a combination of the above devices. That is, the processors of the above embodiments may be any processing device or device combination that implements the various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. If the embodiments of the present disclosure are partially implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to implement the methods of the embodiments of the present disclosure. The term "processor" used herein may refer to the above structure or any other structure suitable for implementing the technology described herein.

[0353] An embodiment of the present disclosure further provides a video encoding and decoding system, which includes the video encoding device described in any embodiment of the present disclosure and the video decoding device described in any embodiment of the present disclosure.

[0354] An embodiment of the present disclosure further provides a non-volatile computer-readable storage medium, which stores a computer program, wherein the computer program, when executed by a processor, can implement the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure, or implement the video decoding method as described in any embodiment of the present disclosure, or implement the video encoding method as described in any embodiment of the present disclosure.

[0355] An embodiment of the present disclosure further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it is capable of implementing the intra-frame prediction method based on interpolation filtering as described in any embodiment of the present disclosure, or implementing the video decoding method as described in any embodiment of the present disclosure, or implementing the video encoding method as described in any embodiment of the present disclosure.

[0356] In one or more exemplary embodiments above, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that facilitates the transfer of a computer program from one place to another, such as according to a communication protocol. In this way, a computer-readable medium may generally correspond to a non-transitory tangible computer-readable storage medium or a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this disclosure. A computer program product may include a computer-readable medium.

[0357] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection may also be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient (transient) media, but rather refer to non-transient tangible storage media. As used herein, disk and optical disk include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc, among others, where disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0358] In some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0359] The technical solutions of the embodiments of the present disclosure can be implemented in a wide variety of devices or equipment, including wireless mobile phones, integrated circuits (ICs), or a group of ICs (e.g., chipsets). Various components, modules, or units are described in the embodiments of the present disclosure to emphasize the functional aspects of the devices configured to perform the described techniques, but they do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

Claims

1. An intra-frame prediction method based on interpolation filtering, comprising: Determine, according to the filter window of the current block and the reconstruction values ​​of the pixels in the reconstruction area adjacent to the current block, the filter coefficient used when performing interpolation filtering on the current block; According to the filter window and the filter coefficient, pixel points in the current block are predicted point by point to obtain a predicted block of the current block; When predicting the current pixel, the reference pixel points adjacent to the current pixel are determined according to the filtering window, and interpolation filtering is performed based on the pixel values ​​of the reference pixel points and the filtering coefficients to obtain the predicted value of the current pixel point.

2. The intra-frame prediction method according to claim 1, wherein: The reference pixel point includes at least one of the following: a reconstructed pixel point in a reconstructed area adjacent to the current block, and a predicted pixel point in the current block; When the reference pixels are all reconstructed pixels, the pixel values ​​of the reference pixels refer to the reconstructed values ​​of the reference pixels; When the reference pixels are all predicted pixels, the pixel values ​​of the reference pixels refer to the predicted values ​​of the reference pixels; When the reference pixel includes a reconstructed pixel and a predicted pixel, the pixel value of the reference pixel includes a reconstructed value of the reconstructed pixel in the reference pixel and a predicted value of the predicted pixel in the reference pixel.

3. The intra-frame prediction method according to claim 1, wherein: The filter window of the current block is determined according to a block parameter of the current block, where the block parameter includes at least one of a shape, a size, and a position.

4. The intra-frame prediction method according to claim 1, wherein: There are one or more filter windows for the current block, and there are one or more reconstruction areas adjacent to the current block; The step of determining the filter coefficients used when performing interpolation filtering on the current block according to the filter window of the current block and the reconstruction values ​​of the pixels in the reconstruction area adjacent to the current block comprises: for each combination of the filter window of the current block and the reconstruction area adjacent to the current block, determining a corresponding set of filter coefficients according to the filter window in the combination and the reconstruction values ​​of the pixels in the reconstruction area in the combination; The method comprises: performing point-by-point prediction on the pixel points in the current block according to the filter window and the filter coefficient to obtain a prediction block of the current block, including: for each combination of the filter window of the current block and a reconstruction area adjacent to the current block, performing point-by-point prediction on the pixel points in the current block according to the filter window in the combination and a set of filter coefficients corresponding to the combination to obtain a prediction block of the current block.

5. The intra-frame prediction method according to claim 4, wherein: Each filter window of the current block includes a current position and multiple adjacent positions, and a set of filter coefficients corresponding to the combination including the filter window includes multiple filter coefficients corresponding to the multiple adjacent positions one by one; The method performs point-by-point prediction on the pixels in the current block according to the filter window in the combination and a group of filter coefficients corresponding to the combination, including: aligning the current pixel with the current position of the filter window in the combination, determining the pixel points at all adjacent positions of the filter window in the combination as reference pixel points adjacent to the current pixel, and performing interpolation filtering based on the pixel values ​​of the reference pixel points and a group of filter coefficients corresponding to the combination to obtain a predicted value of the current pixel.

6. The intra-frame prediction method according to claim 5, wherein: Determining a corresponding set of filter coefficients according to the filter window in the combination and the reconstruction values ​​of the pixels in the reconstruction area in the combination includes: The filter window in the combination is placed at M different positions with reconstructed pixels in the reconstruction area of ​​the combination, M≥1; Taking the M current positions in the filtering windows at the M different positions as sample positions, and taking the reconstructed values ​​of the pixels at all adjacent positions in the filtering windows at the M different positions or the reconstructed values ​​after removing the mean as input, constructing the Wienerhof equation; With the goal of minimizing the error between the predicted values ​​and the reconstructed values ​​of the M current positions, the Wienerhof equation is solved to obtain a set of filter coefficients corresponding to the combination.

7. The intra-frame prediction method according to claim 6, wherein: The solution of the Wienerhof equation uses full integer calculation, and the autocorrelation coefficient matrix of the Wienerhof equation is decomposed by Kresky decomposition or lower unit triangular matrix LDL decomposition.

8. The intra-frame prediction method according to claim 6, wherein: The interpolation filtering is performed based on the pixel value of the reference pixel and a set of filter coefficients corresponding to the combination to obtain a predicted value of the current pixel, including: The predicted value pred of the current pixel is calculated according to the following formula: Alternatively, the predicted value pred of the current pixel is calculated according to the following formula: in, is the pixel value of the nth reference pixel of the current pixel, n=0,1,…,N-1, c n is the filter coefficient corresponding to the nth reference pixel in the set of filter coefficients, N is the number of reference pixels, offset and shift are predefined values, and ">>" represents a right shift operation of a binary number.

9. The intra-frame prediction method according to claim 6, wherein: The interpolation filtering is performed based on the pixel value of the reference pixel and a set of filter coefficients corresponding to the combination to obtain a predicted value of the current pixel, including: When the set of filter coefficients is obtained by solving the Wienerhof equation with the reconstructed value after removing the mean as input, the predicted value pred of the current pixel is calculated according to the following formula: Alternatively, the predicted value pred of the current pixel is calculated according to the following formula: in, is the pixel value of the nth reference pixel of the current pixel, n=0,1,…,N-1, c n is the filter coefficient corresponding to the nth reference pixel in the set of filter coefficients, N is the number of reference pixels, m is the mean used in calculating the reconstructed value after de-averaging, offset and shift are predefined values, and ">>" represents a right shift operation of a binary number.

10. The intra-frame prediction method according to claim 6, wherein: The reconstructed value after removing the mean is obtained by subtracting the mean from the reconstructed value, and the mean is obtained in the following manner: The reconstruction values ​​of all pixels in the reconstruction area of ​​the combination are averaged to obtain the average value: or averaging the reconstruction values ​​of some pixels in the reconstruction area of ​​the combination to obtain the mean value; or The reconstructed value of a pixel point specified in the reconstructed area in the combination is used as the mean value.

11. The intra-frame prediction method according to claim 10, wherein: The reconstructed values ​​of some pixels in the reconstructed area of ​​the combination are averaged, wherein the some pixels include: pixels in one or more rows above the current block, and / or pixels in one or more columns on the left side of the current block; The taking the reconstruction value of a pixel point designated in the reconstruction area of ​​the combination as the mean value is to take the reconstruction value of a pixel point adjacent to the upper left corner of the current block in the reconstruction area as the mean value.

12. The intra-frame prediction method according to claim 1, wherein: The reconstruction area adjacent to the current block includes any one or more of the following areas adjacent to the current block: An upper reconstruction region located above the current block, each upper reconstruction region including one region, two consecutive regions, or three consecutive regions among an upper left region, an upper region, and an upper right region of the current block; A left reconstruction area located on the left side of the current block, each left reconstruction area includes one area or two consecutive areas or three consecutive areas among the upper left area, the left area and the lower left area of ​​the current block; An L-shaped reconstruction area located above and on the left side of the current block, each L-shaped reconstruction area includes the area directly above, the area directly above left, and the area directly below left of the current block, or includes one or two of the area directly above, the area directly above left, the area directly below left, the area directly above right, and the area below left.

13. The intra-frame prediction method according to claim 1, wherein: The size of the reconstruction area adjacent to the current block is determined according to the size of the current block; In the reconstruction area adjacent to the current block, the number of rows located above the current block and the number of columns located to the left of the current block are determined according to the width and / or height of the current block; There is one reconstruction region adjacent to the current block; or there are multiple reconstruction regions adjacent to the current block, and at least one of the type, width, height and position of different reconstruction regions is different.

14. The intra-frame prediction method according to claim 5, wherein: The plurality of adjacent positions in the filtering window are located directly above, above and to the left of the current position; or The plurality of adjacent positions in the filtering window are located directly above, above and to the left, to the left, and above and to the right of the current position; or The plurality of adjacent positions in the filtering window are located directly above, above and to the left, to the left, and below and to the left of the current position.

15. The intra-frame prediction method according to claim 14, wherein: When the plurality of adjacent positions in the filtering window are located directly above, above the left, and to the left of the current position, the pixel points in the current block are predicted point by point in the current block in a sequence from left to right and then from top to bottom, or from top to bottom and then from left to right, or in a diagonal direction, or in a Zig-Zag sequence; When the plurality of adjacent positions in the filtering window are located directly above, above left, left and below left of the current position, the pixel points in the current block are predicted point by point, i.e., the pixel points are predicted point by point in the current block in the order from top to bottom and then from left to right; When the multiple neighboring positions in the filtering window are located directly above, above the left, to the left, and above the right of the current position, the point-by-point prediction of the pixels in the current block is performed in the current block in an order from left to right and then from top to bottom.

16. The intra-frame prediction method according to claim 1, wherein: The shape of the filtering window is rectangular or L-shaped, including a current position corresponding to a current pixel point, and at least two adjacent positions corresponding to reference pixel points.

17. The intra-frame prediction method according to claim 1, wherein: The performing interpolation filtering based on the pixel value of the reference pixel and the filter coefficient to obtain a predicted value of the current pixel includes: Performing interpolation filtering based on the pixel value of the reference pixel and the filter coefficient to obtain an initial prediction value; When the initial prediction value exceeds the prediction value range of the current block, the initial prediction value is corrected so that the corrected prediction value is within the prediction value range of the current block; and The corrected prediction value is used as the prediction value of the current pixel.

18. The intra-frame prediction method according to claim 17, wherein: The correction of the initial prediction value includes: when the initial prediction value is greater than the maximum value of the prediction value range, using the maximum value as the corrected prediction value; when the initial prediction value is less than the minimum value of the prediction value range, using the minimum value as the corrected prediction value; wherein the prediction value range is determined according to the data depth of the prediction value, or according to the maximum reconstruction value and the minimum reconstruction value of the pixel points in the reconstruction area or its local area used when determining the filter coefficient.

19. A video decoding method, comprising: Decode the interpolation filter mode flag of the current block; When determining that the current block uses the interpolation filter mode for intra-frame prediction according to the interpolation filter mode flag obtained by decoding, determine the filter window and reconstruction area selected for the current block; Based on the determined filtering window and reconstruction area, intra-frame prediction is performed on the current block according to the intra-frame prediction method based on interpolation filtering as claimed in any one of claims 1 to 18.

20. The video decoding method according to claim 19, wherein: The determining of a filter window selected for the current block includes: Determine a set filter window as the filter window selected by the current block; or Determine a filter window corresponding to a block parameter of the current block as the filter window selected for the current block, wherein the block parameter includes at least one of a position, a shape and a size; or Continue decoding the sub-mode index of the current block, and determine the filter window selected for the current block according to the sub-mode index, wherein each sub-mode index corresponds to a combination of the filter window of the current block and a reconstruction area adjacent to the current block; Continue decoding the filter window index of the current block, and determine the filter window selected for the current block according to the filter window index, where each filter window index corresponds to a filter window; or Continue decoding the parameter index of the filter window of the current block, determine the parameters of the filter window according to the parameter index of the filter window, and determine the filter window selected for the current block according to the parameters, wherein the parameters of the filter window include at least one of shape, size, and number of adjacent positions.

21. The video decoding method according to claim 19, wherein: The determining of a reconstruction area selected as the current block includes: Determine the reconstruction area selected for the current block according to the set parameters of the reconstruction area; or Determine a reconstruction region corresponding to a block parameter of the current block as the reconstruction region selected for the current block, wherein the block parameter includes at least one of a position, a shape, and a size; or Continue decoding the sub-mode index of the current block, and determine the reconstructed area selected for the current block according to the sub-mode index, wherein each sub-mode index corresponds to a combination of the filter window of the current block and the reconstructed area adjacent to the current block; or Continue decoding the reconstruction region index of the current block, and determine the reconstruction region selected for the current block according to the reconstruction region index, where each reconstruction region index corresponds to a reconstruction region; or Continue decoding the parameter index of the reconstruction area of ​​the current block, determine the parameters of the reconstruction area according to the parameter index of the reconstruction area, and determine the reconstruction area selected for the current block according to the parameters.

22. The video decoding method according to claim 21, wherein: The parameter index of the reconstruction area includes at least one of the following: Rebuild the region's type index; A size index of the reconstructed area, where the size of the reconstructed area includes at least one of the number of rows of the reconstructed area above the current block and the number of columns of the reconstructed area on the left side of the current block.

23. The video decoding method according to claim 19, wherein: The step of determining a neighboring reconstruction area selected for the current block includes: Continue decoding the type index of the reconstruction area of ​​the current block, and determine the type of the reconstruction area selected for the current block according to the decoded type index, wherein the type is an upper reconstruction area, a left reconstruction area, or an L-shaped reconstruction area; Continue decoding the size index of the reconstruction area of ​​the current block, and determine the size of the reconstruction area selected for the current block according to the decoded size index; The reconstruction area selected for the current block is determined according to the type, number of rows and number of columns of the reconstruction area adjacent to the current block, and the position and size of the current block.

24. The video decoding method according to claim 21, wherein: When the current block is a luminance block, decoding of the interpolation filter mode flag of the current block is allowed; or When the current block is a luminance block or a chrominance block, the interpolation filter mode flag of the current block is allowed to be decoded.

25. The video decoding method according to claim 21, wherein: Before decoding the interpolation filter mode flag of the current block, the method further includes any one or more of the following processes: Decoding a sequence-level flag indicating whether the interpolation filtering mode is allowed to be used, and if the value of the sequence-level flag indicates that the interpolation filtering mode is allowed to be used, continuing to decode the interpolation filtering mode flag of the current block, and if the value of the sequence-level flag indicates that the interpolation filtering mode is not allowed to be used, skipping decoding the interpolation filtering mode flag of the current block; Decoding an image level flag indicating whether the interpolation filter mode is allowed to be used, and if the value of the image level flag indicates that the interpolation filter mode is allowed to be used, continuing to decode the interpolation filter mode flag of the current block, and if the value of the image level flag indicates that the interpolation filter mode is not allowed to be used, skipping decoding the interpolation filter mode flag of the current block; Decoding a slice-level flag indicating whether the interpolation filter mode is allowed to be used, and continuing to decode the interpolation filter mode flag of the current block only when the value of the slice-level flag indicates that the interpolation filter mode is allowed to be used, and skipping decoding the interpolation filter mode flag of the current block when the value of the slice-level flag indicates that the interpolation filter mode is not allowed to be used; Decode the general restriction information GCI identifier indicating whether the use of the interpolation filtering mode is prohibited. When the value of the GCI identifier indicates that the use of the interpolation filtering mode is not prohibited, continue to decode the sequence-level flag indicating whether the use of the interpolation filtering mode is allowed. When the value of the GCI identifier indicates that the use of the interpolation filtering mode is prohibited, skip decoding the sequence-level flag indicating whether the use of the interpolation filtering mode is allowed. By default, the sequence-level flag indicates that the use of the interpolation filtering mode is not allowed.

26. The video decoding method according to claim 19, wherein: The current block is a luminance block; after performing intra-frame prediction on the current block, the method further includes: When the chroma block at the position where the current block is located adopts the DM mode, determining that the mode for performing intra-frame prediction on the chroma block is the planar mode; or When the chroma block at the position where the current block is located adopts the DM mode, determining that the mode for intra-frame prediction of the chroma block is the interpolation filtering mode; or When the chroma block at the position of the current block adopts the DM mode, the interpolation filter mode flag of the chroma block continues to be decoded, and whether the mode of intra-frame prediction of the chroma block is the interpolation filter mode is determined according to the value of the interpolation filter mode flag.

27. The video decoding method according to claim 19, wherein: Before decoding the interpolation filter mode flag of the current block, the method further includes any one or more of the following processes: Determine whether the size of the current block satisfies the maximum block size and / or the minimum block size for enabling the interpolation filtering mode, and if so, continue decoding the interpolation filtering mode flag of the current block, and if not, skip decoding the interpolation filtering mode flag of the current block; the size includes at least one of size, width, and height; Determine whether the difference between the horizontal coordinate of the upper left corner of the current block and the horizontal coordinate of the upper left corner of the current image is greater than or equal to a set first threshold, and if it is greater than or equal to the set first threshold, continue to decode the interpolation filter mode flag of the current block; if it is less than the first threshold, skip decoding the interpolation filter mode flag of the current block; Determine whether the difference between the vertical coordinate of the upper left corner of the current block and the vertical coordinate of the upper left corner of the current image is greater than or equal to a set second threshold. If it is greater than or equal to the set second threshold, continue decoding the interpolation filter mode flag of the current block. If it is less than the second threshold, skip decoding the interpolation filter mode flag of the current block.

28. The video decoding method according to claim 19, wherein: The method further comprises: decoding an intra-frame sub-block partitioning ISP mode flag; When the ISP mode flag indicates that the current coding unit uses the ISP mode, the current block for interpolation filtering is a sub-block obtained by dividing the current coding unit; When the ISP mode flag indicates that the current coding unit does not use the ISP mode, the current block on which the interpolation filtering is performed is the current coding unit.

29. The video decoding method according to claim 19, wherein: After performing intra-frame prediction on the current block, the method further includes: decoding a combined prediction PDCP mode flag related to the position; when the PDCP mode flag indicates that the current block uses the PDCP mode, using the unsmoothed filtered reconstruction values ​​of the pixel points in at least one row and one column adjacent to the current block to weight the prediction block of the current block obtained by interpolation filtering.

30. The video decoding method according to claim 19, wherein: After performing intra-frame prediction on the current block, the method further includes: performing intra-frame prediction on the current block according to one or more intra-frame prediction modes different from the interpolation filtering mode, and weighting the prediction block of the current block obtained based on the one or more intra-frame prediction modes and the prediction block of the current block obtained based on the interpolation filtering mode to obtain a final prediction block of the current block.

31. The video decoding method of claim 19, wherein: After performing intra-frame prediction on the current block, the method further includes: determining a residual of the current block according to an original value and a predicted value of the current block, and performing transformation and inverse transformation on the residual; wherein: When transforming and inversely transforming the residual, the transform kernel used is a transform kernel obtained by training based on residual samples in an interpolation filtering mode; or When transforming and inversely transforming the residual, limiting the number of combinations of primary transform kernels and secondary transform kernels; or When transforming and inversely transforming the residual, the number of combinations of primary transform kernels and secondary transform kernels is limited according to the size and / or shape of the current block; or When transforming and inversely transforming the residual, the transform kernel adopted when performing intra-frame prediction based on the planar mode is used.

32. The video decoding method of claim 19, wherein: After performing intra-frame prediction on the current block, the method further includes: Determine a residual of the current block according to an original value and a predicted value of the current block, use a transform skip mode to skip transforming the residual, and perform block-wise quantization, dequantization, and reconstruction on the current block; or A residual of the current block is determined according to an original value and a predicted value of the current block, a transform skip mode is used to skip transforming the residual, and pixel points of the current block are quantized, dequantized, and reconstructed point by point.

33. The video decoding method of claim 19, wherein: After performing intra-frame prediction on the current block, the method further includes: When the current block is used as a neighbor block of other coding blocks to construct a most probable mode list, the planar mode is used as the intra prediction mode used by the current block.

34. A video encoding method, comprising: Performing intra-frame prediction on the current block based on multiple intra-frame prediction modes, wherein, when it is determined that the current block allows the use of the interpolation filtering mode, performing intra-frame prediction of the current block in the interpolation filtering mode according to the intra-frame prediction method according to any one of claims 1 to 18; When it is determined according to the rate-distortion costs of the multiple intra-frame prediction modes that the current block uses an interpolation filter mode for intra-frame prediction, an interpolation filter mode flag of the current block is encoded to indicate that the current block uses the interpolation filter mode for intra-frame prediction.

35. The video encoding method of claim 34, wherein: The determining that the current block is allowed to use the interpolation filtering mode includes: when none of the conditions for prohibiting the use of the interpolation filtering mode is satisfied, determining that the current block is allowed to use the interpolation filtering mode; wherein the conditions for prohibiting the use of the interpolation filtering mode include any one or more of the following conditions: The current block is a chroma block; There is a sequence-level flag indicating that interpolation filter mode is not allowed; The presence of an image level flag indicating that the interpolation filter mode is not allowed; There is a slice-level flag indicating that the interpolation filter mode is not allowed; There is a GCI flag indicating that the interpolation filter mode is prohibited; The size of the current block does not satisfy the maximum block size and / or the minimum block size for enabling the interpolation filtering mode; The difference between the horizontal coordinate of the upper left corner of the current block and the horizontal coordinate of the upper left corner of the current image is less than a set first threshold; The difference between the ordinate of the upper left corner of the current block and the ordinate of the upper left corner of the current image is less than a set second threshold.

36. The video encoding method of claim 34, wherein: The intra-frame prediction of the interpolation filtering mode on the current block includes: determining a combination of a filter window of the current block and a reconstruction area adjacent to the current block used when the intra-frame prediction of the interpolation filtering mode is performed on the current block, K combinations correspond to K sub-modes of the interpolation filtering mode, and based on the filter window of the current block and the reconstruction area adjacent to the current block in the combination corresponding to each sub-mode, performing intra-frame prediction based on interpolation filtering on the current block, K is the number of combinations, K≥2; When determining, according to the rate-distortion costs of the multiple intra-frame prediction modes, that the current block uses the interpolation filtering mode for intra-frame prediction, a minimum value among the rate-distortion costs of the K sub-modes is used as the rate-distortion cost of the interpolation filtering mode, and is compared with the rate-distortion costs of other intra-frame prediction modes in the multiple intra-frame prediction modes; When the rate-distortion cost of the interpolation filtering mode is the smallest, the method further comprises: encoding index information for indicating the sub-mode with the smallest rate-distortion cost among the K sub-modes.

37. The video encoding method of claim 36, wherein: The index information representing the sub-mode with the smallest rate-distortion cost among the K sub-modes includes any one or more of the following indexes: Sub-mode index, each sub-mode index corresponds to a combination of a filter window of a current block and a reconstruction area adjacent to the current block; A reconstruction region index of the current block, each reconstruction region index corresponds to a reconstruction region; a parameter index of a reconstruction area of ​​the current block, wherein the parameter index of the reconstruction area includes at least one of a size index and a type index, and the size of the reconstruction area includes at least one of the number of rows of the reconstruction area above the current block and the number of columns of the reconstruction area on the left side of the current block; The filter window index of the current block, each filter window index corresponds to a filter window; A parameter index of a filter window of a current block, wherein the parameter of the filter window includes at least one of a shape, a size, and a number of neighboring positions.

38. A code stream, wherein The code stream is generated according to the video encoding method as described in any one of claims 34 to 37.

39. An intra-frame prediction device based on interpolation filtering, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, the intra-frame prediction method based on interpolation filtering as claimed in any one of claims 1 to 18 can be implemented.

40. A video decoding device, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the video decoding method as described in any one of claims 19 to 33.

41. A video encoding device, comprising a processor and a memory storing a computer program, wherein: When the processor executes the computer program, it can implement the video encoding method as described in any one of claims 34 to 37.

42. A video encoding and decoding system, wherein: It comprises the video encoding device as claimed in claim 41 and the video decoding device as claimed in claim 40.

43. A non-transitory computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, it can implement the intra-frame prediction method based on interpolation filtering as described in any one of claims 1 to 18, or implement the video decoding method as described in any one of claims 19 to 33, or implement the video encoding method as described in any one of claims 34 to 37.

44. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, it can implement the intra-frame prediction method based on interpolation filtering as described in any one of claims 1 to 18, or implement the video decoding method as described in any one of claims 19 to 33, or implement the video encoding method as described in any one of claims 34 to 37.