Self-guided intra interpolation filter

Abstract

The present disclosure includes a method and apparatus that includes processing circuitry that receives encoding information of a code stream from the code stream including a current block in a picture, the encoding information indicating that the current block is encoded in an angular intra prediction mode using an intra interpolation filter. The processing circuitry applies each of a predefined set of intra-frame interpolation filters to adjacent reconstructed samples within N adjacent rows from the boundary of the current block. The processing circuitry selects one interpolation filter from the predefined set of interpolation filters based on the prediction error associated with each of the predefined set of interpolation filters, and predicts samples in the current block with an angular intra prediction mode using the selected one interpolation filter. The processing circuitry reconstructs the current block based on the predicted samples.

Description

[0001] Incorporation by Reference

[0002] This application claims priority to U.S. Patent Application No. 18 / 384,759, filed on October 27, 2023, entitled "Self-guided Intra interpolation filter," which claims priority to U.S. Provisional Patent Application No. 63 / 444,877, filed on February 10, 2023, entitled "Self-guided Intra interpolation filter." The disclosures of the two prior applications are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure describes aspects generally related to video encoding and decoding. Background Art

[0004] The background description provided herein is intended to generally present the context of the present application. To the extent described in this background section, the work of the inventors currently named, as well as various aspects of the description that may not have the qualification of being prior art at the time of filing this application, are neither explicitly nor implicitly considered to be prior art of the present application.

[0005] Image / video compression can help transmit image / video data between different devices, storages, and networks with minimal degradation. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can predict samples in the current picture based on previously reconstructed pictures through motion compensation. Motion compensation can be represented by a motion vector (MV). Summary of the invention

[0006] Aspects of the present disclosure include methods and apparatus for video encoding / decoding. In some examples, a video decoding apparatus includes a processing circuit. In the example, the processing circuit receives encoding information of a code stream from a code stream including a current block in a picture, and the encoding information indicates that the current block is encoded using an intra-frame interpolation filter in an angular intra-frame prediction mode. The processing circuit applies each of a predefined set of intra-frame interpolation filters to adjacent reconstructed samples within N adjacent rows from a boundary of the current block. N is a positive integer. The processing circuit selects an intra-frame interpolation filter from a predefined set of intra-frame interpolation filters based on the prediction error associated with each of the predefined set of intra-frame interpolation filters, and uses the selected intra-frame interpolation filter to predict samples in the current block using the angular intra-frame prediction mode.

[0007] In an example, a processing circuit receives a code stream of a current block in a picture. The coding information of the code stream indicates that the current block is encoded using an intra-frame interpolation filter in an angular intra-frame prediction mode. The processing circuit selects an intra-frame interpolation filter from a predefined set of intra-frame interpolation filters based on neighboring reconstructed samples of the current block. The neighboring reconstructed samples include reconstructed samples within N rows from one or more boundaries of the current block. The processing circuit applies the selected intra-frame interpolation filter to reference samples in a reference row in the picture to predict samples in the current block using the angular intra-frame prediction mode, and reconstructs the current block based on the predicted samples.

[0008] In an example, the processing circuit selects one of: (i) a type of intra-frame interpolation filter, or (ii) a number of taps in the intra-frame interpolation filter from a predefined set of intra-frame interpolation filters based on neighboring reconstructed samples of the current block.

[0009] In an example, a predefined set of intraframe interpolation filters includes different types of intraframe interpolation filters. The processing circuit selects the type of the intraframe interpolation filter from different types of intraframe interpolation filters based on adjacent reconstructed samples of the current block. The selected intraframe interpolation filter is one of a bilinear interpolation filter, a cubic interpolation filter, a spline interpolation filter, a DCT-based interpolation filter, or a DST-based interpolation filter.

[0010] In an example, a set of intraframe interpolation filters are predefined to include different numbers of taps. The processing circuit selects the number of taps of the intraframe interpolation filter based on neighboring reconstructed samples of the current block. The number of taps of the intraframe interpolation filter is one of 2 taps, 4 taps, 6 taps or 8 taps.

[0011] In an example, N is greater than 1, and the adjacent reconstructed samples include a plurality of rows of adjacent reconstructed samples. For each combination of an intraframe interpolation filter in a predefined set of intraframe interpolation filters and a row of adjacent reconstructed samples in the plurality of rows of adjacent reconstructed samples, the processing circuit uses the corresponding intraframe interpolation filter to predict the adjacent reconstructed samples of the corresponding row based on one or more remaining rows in the plurality of rows of adjacent reconstructed samples, and obtains at least one prediction error, and selects the intraframe interpolation filter corresponding to the minimum prediction error among the obtained prediction errors.

[0012] In an example, whether the neighboring reconstructed samples include (i) a left neighboring reconstructed sample row to the left of the current block, (ii) a top neighboring reconstructed sample row above the current block, or (iii) a left neighboring reconstructed sample row to the left of the current block and a top neighboring reconstructed sample row above the current block depends on the intra prediction direction of the angular intra prediction mode.

[0013] In the example, the value of N depends on the block size.

[0014] In an example, the processing circuit predicts each row of neighboring reconstructed samples based on an intra prediction direction of an angular intra prediction mode.

[0015] In an example, the processing circuit predicts each row of neighboring reconstructed samples based on a direction opposite to the intra prediction direction of the angular intra prediction mode.

[0016] In an example, the processing circuit selects an intra interpolation filter from a predefined set of intra interpolation filters based on neighboring reconstructed samples of the current block, and selects an angular intra prediction mode from a plurality of angular intra prediction modes.

[0017] In the example, the processing circuit derives N1 angular intra prediction modes from a plurality of angular intra prediction modes, the N1 angular intra prediction modes being associated with the N1 lowest cost values ​​from the cost values ​​of the angular intra prediction modes. Each cost value is determined based on adjacent reconstructed samples of the current block, the corresponding angular intra prediction mode, and a default intra interpolation filter from a predefined set of intra interpolation filters. For each of the N1 angular intra prediction modes, the processing circuit selects M1 intra interpolation filters from a predefined set of intra interpolation filters, the M1 intra interpolation filters being associated with the M1 lowest cost values ​​from the cost values ​​of the predefined set of intra interpolation filters. Each cost value is determined based on adjacent reconstructed samples of the current block, the corresponding intra interpolation filter, and the corresponding angular intra prediction mode from the N1 angular intra prediction modes. The processing circuit selects an intra interpolation filter and an angular intra prediction mode from an N1×M1 combination. Each of the N1×M1 combinations includes an intra interpolation filter from among the M1 intra interpolation filters and an angular intra prediction mode from among the N1 angular intra prediction modes.

[0018] In an example, the processing circuit selects N2 angular intra prediction modes from a first plurality of angular intra prediction modes based on neighboring reconstructed samples of the current block and an intra interpolation filter. The processing circuit selects a first updated intra interpolation filter based on neighboring reconstructed samples of the current block and one of the N2 angular intra prediction modes. The processing circuit selects N3 angular intra prediction modes from a second plurality of angular intra prediction modes based on neighboring reconstructed samples of the current block and the first updated intra interpolation filter. The processing circuit selects a second updated intra interpolation filter based on neighboring reconstructed samples of the current block and one of the N3 angular intra prediction modes. The processing circuit selects a second updated intra-frame interpolation filter as the selected intra-frame interpolation filter and selects one of the N3 angular intra-frame prediction modes as the selected angular intra-frame prediction mode based on (i) a difference between a first prediction error associated with a first updated intra-frame interpolation filter and an angular intra-frame prediction mode among the N2 directional intra-frame prediction modes and a second prediction error associated with a second updated intra-frame interpolation filter and an angular intra-frame prediction mode among the N3 angular intra-frame prediction modes, or (ii) the second prediction error.

[0019] In the example, for each combination of an angular intra prediction mode among multiple angular intra prediction modes and an intra interpolation filter from a predefined set of intra interpolation filters, the processing circuit determines a cost value for each combination based on adjacent reconstructed samples of the current block, selects K combinations based on the determined cost values, and selects the intra interpolation filter and the angular intra prediction mode from the K combinations based on index information signaled in the bitstream.

[0020] In an example, the processing circuit calculates a feature value based on neighboring reconstructed samples of the current block, and selects an intra-frame interpolation filter based on the calculated feature value.

[0021] In an example, the processing circuit calculates the feature value as one of: (i) an absolute gradient value associated with neighboring reconstructed samples of the current block, or (ii) a difference between a minimum value of the neighboring reconstructed samples of the current block and a maximum value of the neighboring reconstructed samples of the current block.

[0022] In an example, the processing circuit obtains an output of a neural network, the input of the neural network includes neighboring reconstructed samples of the current block, and the output indicates the selected intra-frame interpolation filter. The input of the neural network also includes an intra-frame prediction mode index indicating an angular intra-frame prediction mode.

[0023] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer, cause the computer to perform a video encoding / decoding method. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Further features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0025] Figure 1 is a schematic diagram of an exemplary block diagram of a communication system (100).

[0026] Figure 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0027] Figure 3 is a schematic diagram of an exemplary block diagram of an encoder.

[0028] Figure 4 Nine prediction sub-directions from possible prediction sub-directions are depicted according to an aspect of the present disclosure.

[0029] Figure 5 Intra prediction directions according to an aspect of the present disclosure are shown.

[0030] Figure 6 Eight nominal angles are shown in accordance with an aspect of the present disclosure: V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED.

[0031] Figure 7 An example of a non-directional intra predictor according to an aspect of the present disclosure is shown.

[0032] Figure 8 An example of a recursive intra filtering mode in accordance with aspects of the present disclosure is shown.

[0033] Fig. 9 An example of multiple reference line (MRL) intra prediction is shown.

[0034] Fig.10 An exemplary angular intra prediction using reference line 0 neighboring a block is shown.

[0035] Fig.11 An example of a current block and neighboring reconstructed samples of the current block encoded using an angular intra prediction mode according to an aspect of the present disclosure is shown.

[0036] Fig.12 A flow chart outlining a decoding process according to some aspects of the present disclosure is shown.

[0037] Fig.13 A flow chart outlining an encoding process according to some aspects of the present disclosure is shown.

[0038] Fig.14 A flow chart outlining a decoding process according to some aspects of the present disclosure is shown.

[0039] Fig.15 is a schematic diagram of a computer system according to one aspect. DETAILED DESCRIPTION

[0040] Figure 1 A block diagram of a video processing system (100) in some examples is shown. The video processing system (100) is an example of the application of the disclosed subject matter, video encoders and video decoders in a streaming environment. The disclosed subject matter of the present application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0041] The video processing system (100) may include an acquisition subsystem (113), which may include a video source (101) such as a digital camera, which creates, for example, an uncompressed video picture stream (102). In one example, the video picture stream (102) includes samples captured by the digital camera. The video picture stream (102) is depicted as a thick line to emphasize that it has a higher data volume than the encoded video data (104) (or the encoded video bitstream), and the video picture stream (102) can be processed by an electronic device (120), which includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video data (104) (or the encoded video bitstream) is depicted as a thin line to emphasize that it has a lower data volume than the video picture stream (102), which can be stored on a streaming server (105) for future use. One or more streaming client subsystems, e.g. Figure 1A client subsystem (106) and a client subsystem (108) in a streaming server (105) may access a streaming server (105) to retrieve a copy (107) and a copy (109) of encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and generates an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T Recommendation H.265. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the subject matter disclosed in the present application may be used in the context of the VVC standard.

[0042] It should be noted that the electronic device (120) and the electronic device (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0043] Figure 2 is an exemplary block diagram of a video decoder (210). The video decoder (210) may be provided in an electronic device (230). The electronic device (230) may include a receiver (231) (eg, a receiving circuit). The video decoder (210) may be used to replace Figure 1 A video decoder (110) is shown in an example.

[0044] The receiver (231) can receive one or more encoded video sequences in a bitstream to be decoded by the video decoder (210). In one aspect, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequence can be received from a channel (201), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective use entities (not depicted). The receiver (231) can separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) can be set external to the video decoder (210) (not depicted). In other cases, a buffer memory (not depicted) is provided outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be configured inside the video decoder (210) to, for example, handle broadcast timing. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability, or from an isochronous network, it may not be necessary to configure the buffer memory (215), or the buffer memory may be made smaller. For use on a best-effort service packet network such as the Internet, a buffer memory (215) may also be required, and the buffer memory may be relatively large and may have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not depicted) outside the video decoder (210).

[0045] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potential information for controlling a display device such as a rendering device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown. The control information for the display device may be a parameter set fragment (not depicted) of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set for at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (220) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0046] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0047] Depending on the type of coded video picture or coded video picture portion (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (220) and the multiple units below is not described.

[0048] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0049] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into an aggregator (255).

[0050] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) uses surrounding reconstructed information extracted from a current picture buffer (258) to generate a block of the same size and shape as the block being reconstructed. For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0051] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (221) belonging to the block, these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The acquisition of the predicted samples by the motion compensated prediction unit (253) from the address in the reference picture memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbols (221), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0052] The output samples of the aggregator (255) may be used by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques that are controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) and made available to the loop filter unit (256) as symbols (221) from the parser (220), but may also be responsive to meta-information obtained during decoding of a coded picture or a previous (in decoding order) portion of the coded video sequence, and to previously reconstructed and loop filtered sample values.

[0053] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0054] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0055] The video decoder (210) may perform decoding operations according to a standard or predetermined video compression technology, such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the encoded video sequence follows the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0056] In one aspect, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0057] Figure 3 is an example block diagram of a video encoder (303). The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (eg, a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an example.

[0058] The video encoder (303) can be used to obtain the video source (301) (not Figure 3 In another embodiment, the video source (301) is a part of the electronic device (320) in the example, and receives the video samples. The video source can collect video images to be encoded by the video encoder (303). In another embodiment, the video source (301) is a part of the electronic device (320).

[0059] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The following description focuses on samples.

[0060] According to one aspect, the video encoder (303) can encode and compress pictures of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints required. Implementing the appropriate encoding speed is a function of the controller (350). In some aspects, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, the coupling is not shown in the figure. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, group of pictures (group of pictures, GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0061] In some aspects, the video encoder (303) operates in a coding loop. As a simplified description, in one example, the coding loop may include a source encoder (330) (e.g., responsible for creating symbols, e.g., a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0062] The operation of the "local" decoder (333) may be combined with, for example, Figure 2 The "remote" decoder described in detail for the video decoder (210) is identical. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into the encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0063] In one aspect, any decoder technology except the parsing / entropy decoding present in the decoder is also present in the corresponding encoder in the same or substantially the same functional form. Accordingly, the application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is mutually inverse to the decoder technology described comprehensively. A more detailed description is only needed in certain areas and is provided below.

[0064] During operation, in some examples, the source encoder (330) may perform motion compensated predictive coding to predictively encode an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the difference between a pixel block of the input picture and a pixel block of a reference picture that may be selected as a prediction reference for the input picture.

[0065] The local video decoder (333) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When the video encoder (303) is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0066] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (335) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (334).

[0067] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0068] The outputs of all the above functional units may be entropy encoded in an entropy encoder (345). The entropy encoder (345) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0069] The transmitter (340) may buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0070] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain coded picture type to each coded picture, which may affect the coding techniques that may be applied to the corresponding picture. For example, a picture may generally be assigned any of the following picture types:

[0071] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0072] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction, which uses a motion vector and a reference index to predict sample values ​​of each block.

[0073] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0074] The source picture may typically be spatially subdivided into a number of blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or the blocks may be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0075] The video encoder (303) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0076] In one aspect, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0077] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0078] In some aspects, bidirectional prediction techniques may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future, respectively, in display order). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block may be predicted by a combination of the first reference block and the second reference block.

[0079] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0080] According to some aspects disclosed in the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, for example, 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine the prediction type for the CU, for example, an inter-frame prediction type or an intra-frame prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one aspect, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0081] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors that execute software instructions.

[0082] Video codec techniques may include intra-frame coding. In intra-frame coding, sample values ​​may be represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, pictures may be spatially divided into blocks of samples. When all blocks of samples are encoded in intra-frame mode, the picture may be an intra-frame picture (e.g., an I-picture).

[0083] In some aspects, prediction may be performed based on surrounding sample data and / or metadata obtained during encoding and / or decoding of the data block. Such techniques are referred to as "intra-frame prediction" techniques. In one aspect, intra-frame prediction (e.g., intra-picture prediction) may use only reference data from the current picture being reconstructed, rather than from reference pictures.

[0084] In intra prediction, the predicted subblock can be formed using the values ​​of neighboring samples of the already available samples. Sample values ​​of neighboring samples can be copied to the predicted subblock depending on the direction. The reference to the direction in use can be encoded in the bitstream, or it can be predicted itself.

[0085] Various intra-frame prediction coding tools can be used, such as Figures 4 to 6 and Fig.10 The angle intra prediction shown in Fig. 9 Multiple Reference Line (MRL) prediction as shown etc. Non-directional intra prediction modes may also be used.

[0086] Figure 4 Nine prediction sub-directions from possible prediction sub-directions (e.g., 33 prediction sub-directions, corresponding to 33 angular modes such as the 35 intra modes defined in H.265) are depicted according to an aspect. Point (401) represents a sample being predicted. Arrows represent directions in which sample (401) can be predicted. For example, arrow (402) indicates that sample (401) is predicted from one or more samples to the upper right at an angle of 45° to the horizontal direction (e.g., the X dimension). Arrow (403) indicates that sample (401) is predicted from one or more samples to the lower left of sample (401) at an angle of 22.5° to the horizontal direction.

[0087] Figure 4A block (e.g., a square block of 4×4 samples indicated by a dashed bold line) (404) is also depicted. The block (404) may include 16 samples, each sample labeled with an "S", its position in the Y dimension (e.g., a row index), and its position in the X dimension (e.g., a column index). For example, sample S21 is the second sample in the Y dimension (counting from top to bottom) and the first sample in the X dimension (counting from left to right). Figure 4 Reference samples are also shown in a similar numbering scheme. Reference samples are labeled with R, their Y position (e.g., row index) and X position (column index) relative to the block (404). In some aspects (e.g., in H.264 and H.265), the prediction samples may be adjacent to the block being reconstructed. In some aspects, multiple reference rows may be used, such as Fig. 9 as shown in .

[0088] Intra-frame prediction may be performed by copying reference sample values ​​from neighboring samples indicated by a prediction direction (e.g., a signaled prediction direction). For example, if a prediction direction indicated by arrow (402) is signaled in the encoded video bitstream, the sample is predicted based on the upper right sample at a 45° angle to the horizontal. For example, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Sample S44 may be predicted based on reference sample R08.

[0089] In some cases, the values ​​of multiple reference samples may be combined, for example by interpolation (eg, intra-frame interpolation filters), in order to compute a reference sample such as Fig.10 For example, when the direction is not divisible by 45°, the values ​​of multiple reference samples are combined by interpolation.

[0090] Any suitable number of possible directions may be used in intra prediction. Figure 4 In examples such as H.264, 9 different directions are used. In examples such as H.265, 33 different directions are used.

[0091] In an example of finer granularity angular prediction such as in VVC, 65 different directions are used. 65 angular prediction directions are available for block sizes, and the set of angles may depend on the block size. For square blocks, in an example, 65 angular prediction directions are defined in a clockwise direction from 45° to -135° for a square coding block. Figure 5 A schematic diagram (510) depicting 65 intra prediction directions according to one aspect of the present disclosure (e.g., in JEM) is shown. In addition to the 65 intra prediction directions (corresponding to the 65 intra prediction modes 2-66), the intra prediction modes may include a planar mode ("0") and a DC mode ("1").

[0092] In some examples, such as in VVC, wide angle intra prediction (WAIP) can be used. In WAIP, for non-square blocks, the 14 angles using prediction of the shorter side of the block can be replaced with more extreme angles using prediction of the longer side, bringing the total number of angles supported in WAIP to 93, and the number of angle modes signaled for a particular block size can remain at 65.

[0093] In an example, such as in VP9, ​​8 directional modes are supported, corresponding to angles from 45° to 207°. Figure 6 An example of directional intra prediction such as used in Alliance for Open Media (AOMedia) Video 1 (AV1) is shown in . To exploit a wider variety of spatial redundancies in directional textures, such as in AV1, the directional intra mode can be extended to a finer-grained set of angles. The original 8 angles can be slightly modified as nominal angles. Figure 6 The 8 nominal angles are shown: V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED. Each nominal angle can have 7 finer angles, so for example, 56 directional angles are used in AV1. The prediction angle can be represented by the nominal intra angle (e.g., V_PRED) plus an angle increment (which is a step size of -3 to 3 times 3°). On the one hand, in order to implement the directional prediction mode in a common way in AV1, the directional intra prediction modes in AV1 (e.g., all 56 directional intra prediction modes) are implemented using a unified directional predictor that projects each pixel to a reference sub-pixel location and interpolates the reference pixels through an intra interpolation filter (e.g., a 2-tap bilinear filter), such as Fig.10 as shown in .

[0094] Figure 7 An example of a non-directional intra predictor such as used in AV1 according to an aspect of the present disclosure is shown. Figure 7The positions of the top, left, and upper left samples of a pixel (701) in a current block (710) to be predicted are shown. In an example, such as in AV1, five non-directional smooth intra prediction modes are used, including DC, PAETH, SMOOTH, SMOOTH_V, and SMOOTH_H. For DC prediction, the average of the left neighboring samples and the upper neighboring samples may be used as a predictor for the pixel (701) in the current block (710). For the PAETH predictor, the top, left, and upper left reference samples may be first extracted, and then the value closest to (top + left - upper left) may be set as the predictor for the pixel (701). The SMOOTH, SMOOTH_V, and SMOOTH_H modes may predict the block (710) using quadratic interpolation in the vertical or horizontal direction, or an average in both directions.

[0095] use Figure 8 An intra predictor based on recursive filtering is described. Figure 8 Examples of recursive intra-frame filtering modes in accordance with aspects of the present disclosure are shown. In order to capture the attenuated spatial correlation with the reference on the edge, the FILTER INTRA mode can be used for the block (e.g., luma block). Five filter intra-frame modes can be used, such as in AV1. Each filter intra-frame mode can be represented by a set of eight 7-tap filters that reflect the correlation between the pixel in the 4×2 patch and the 7 neighbors adjacent to the 4×2 patch. The weighting factors of the 7-tap filters can be position-dependent. Figure 8 An example of an 8×8 block (810) is shown. The block (810) can be divided into eight 4×2 patches B0, B1, B2, B3, B4, B5, B6, and B7. The 7 neighborhoods of each patch (e.g., represented by R0 to R7) can be used to predict pixels in the current patch. For patch B0, all neighbors (e.g., represented by R0 to R7 of patch B0) have been reconstructed. For other patches, not all neighbors may have been reconstructed, and the predicted values ​​of the direct neighbors can be used as a reference. For example, the neighborhood of patch B7 (e.g., all neighbors) is not reconstructed, and the predicted samples of the neighbors (e.g., patches B5 and B6) are used.

[0096] MRL intra prediction can use multiple reference lines for intra prediction. Fig. 9 An example of MRL intra prediction is shown. Fig. 9, four reference rows 0 to 3 of the current block (901) are shown. Reference row i may include reference samples that are i rows away from the current block (901), for example, i rows away from a boundary of the current block (901) (e.g., i rows away from the top boundary and / or i columns away from the left boundary), where i is 0, 1, 2, or 3. For example, reference row i includes reference samples that are i rows above the top boundary of the current block (901) and / or i columns to the left of the left boundary of the current block (901). In an example, reference row 0 includes reference samples adjacent to the current block (901), such as reconstructed adjacent samples including a top adjacent sample above the current block (901) and a left adjacent sample to the left of the block (901). In an example, reference row 0 may include an upper left reconstructed adjacent sample.

[0097] Reference lines 0 to 3 may include multiple segments, such as segments A to F. In an example, samples of segments A and F are not extracted from reconstructed adjacent samples. Samples of segments A and F may be filled (or filled) with the nearest samples in segments B and E, respectively.

[0098] In an example, such as in HEVC, the nearest reference row (i.e., reference row 0) is used in intra prediction (or intra picture prediction). In an example, a reference row other than the nearest reference row (e.g., reference row 1) is used in intra prediction. In an example, multiple reference rows may be used in MRL intra prediction.

[0099] MRL intra prediction can be extended, for example in Enhanced Compression Model 5 (ECM5), to include more reference lines for intra prediction.

[0100] In angular intra prediction, a current sample in a current block may be predicted using, for example, reference samples (eg, prediction samples) in a reference row or interpolated reference samples.

[0101] Fig.10 An example of angular intra prediction using reference line 0 adjacent to block (1000) is shown. The description may be appropriately applied to using reference lines that are not adjacent to block (1000) (e.g., reference line 1). A portion of reference line 0 is shown. Sample (1001) in block (1000) may be predicted using one or more samples of reference line 0 in an intra prediction sub-direction (or angular direction) (1022). Sample (1001) may be projected onto reference line 0 along angular direction (1022).

[0102] exist Fig.10In the example shown, the projected position (1013) of the sample (1001) is located between two reference samples (1011) to (1012) of reference row 0, referred to as a projected fractional position or fractional sample position. Reference samples around the projected position (1013) can be used to predict the sample (1001), for example, by using an interpolation filter (e.g., an intra-frame interpolation filter). In one aspect, when the projected position (1013) of the sample (1001) is a fractional sample position between two adjacent reference samples, an interpolation filter is applied. The intra-frame interpolation filter can be any suitable filter, such as a bilinear filter, a cubic filter, a spline interpolation filter, a DCT-based interpolation filter, a DST-based interpolation filter, etc. In one aspect, a sample to be intra-predicted using an angular intra-prediction mode (e.g., sample (1001)) is projected to a fractional position (e.g., projection position (1013)) between two adjacent reference samples (e.g., (1011) to (1012)), and a filter used to generate a predicted sample value for the sample is referred to as an intra-interpolation filter. In an example, the predicted sample value is based on an interpolation of sample values ​​of two adjacent reference samples (e.g., (1011) to (1012)), for example, the predicted sample value is a weighted average of sample values ​​of the two adjacent reference samples, and the weights correspond to filter coefficients of the intra-prediction filter. In an example, the weights are corresponding filter coefficients of the intra-prediction filter. In an example, the predicted sample value is based on an interpolation of sample values ​​of two adjacent reference samples (e.g., (1011) to (1012)) and an additional reference sample.

[0103] The two reference samples (1011) to (1012) of reference row 0 may be used to predict sample (1001) using a two-tap intraframe interpolation filter (e.g., a bilinear filter). In an example, the filter coefficients in the 2-tap intraframe interpolation filter may be based on two distances between the projected position (1013) and two adjacent integer positions (represented by black dots) of the two reference samples (1011) to (1012), respectively. For example, if the distance between the projected fractional position (1013) and the reference sample (1011) is less than the distance between the projected fractional position (1013) and the reference sample (1012), the filter coefficient of the reference sample (1011) is greater than the filter coefficient of the reference sample (1012).

[0104] In an example, 4 reference samples (1011), (1012), (1014), and (1015) of reference row 0 may be used to predict sample (1001) using a 4-tap intra-frame interpolation filter (e.g., a 4-tap linear interpolation filter) such as a DCT-based interpolation filter (DCTIF).

[0105] The filter type, number of filter coefficients (also referred to as filter taps or taps), etc. of the intra-frame interpolation filter may vary. The filter type may include different types of intra-frame interpolation filters, such as a bilinear filter, a cubic filter, a spline filter, a DCT-based interpolation filter, a DST-based interpolation filter, etc. In an example, the filter type includes intra-frame interpolation filters associated with different directions, such as a horizontal filter (e.g., applied to samples in a row, such as Fig.10 ) and vertical filters (e.g., applied to samples in a column). Fig.10 , the 2-tap intra-frame interpolation filter applied to the two reference samples (1011) to (1012) can be a horizontal filter.

[0106] The number of filter taps of the intra-frame interpolation filter may vary, for example, different intra-frame interpolation filters may include intra-frame interpolation filters with different taps, such as 2-tap, 4-tap, 6-tap, and 8-tap filters.

[0107] The filter type and filter taps of the intra-frame interpolation filter may vary.

[0108] In the related art, a known intra-frame interpolation filter is a fixed interpolation filter for a block (e.g., each block). In an example, when the filter type and the number of taps of the intra-frame interpolation filter are fixed, the intra-frame interpolation filter applied to the block is considered to be fixed. When the filter type and the number of taps of the intra-frame interpolation filter are fixed, the filter coefficients of the intra-frame interpolation filter can vary based on the samples to be predicted in the block and the corresponding intra-frame prediction mode used to predict these samples.

[0109] Since the best intra-frame interpolation filter may depend on the statistics of samples in an image or in a video (such as in a picture or in a frame), using a fixed interpolation filter (eg, a fixed intra-frame interpolation filter) may be suboptimal.

[0110] A current block in a current picture may be encoded using a directional intra prediction mode, intra prediction. A directional intra prediction mode may also be referred to as an angular intra prediction mode, an angular mode, a directional mode, or a directional intra mode. A directional intra prediction mode or an angular intra prediction mode is an intra prediction mode that may predict samples in the current block along a prediction direction associated with the angular intra prediction mode, such as Figure 4 and Fig.10 Described in .

[0111] According to an aspect of the present disclosure, an intra-frame interpolation filter (e.g., such as a filter such as a filter element) used in an angular intra-frame prediction mode may be determined (e.g., derived or selected) based on neighboring reconstructed samples (also referred to as neighboring reconstructed samples) of a current block. Fig.10). Determining the intra-frame interpolation filter may include determining one or more characteristics of the intra-frame interpolation filter. For example, determining the intra-frame interpolation filter includes determining the type of the intra-frame interpolation filter and / or determining the number of taps used in the intra-frame interpolation filter. The determined intra-frame interpolation filter may depend on the neighboring reconstructed samples of the current block and may therefore be considered self-guided, e.g., the determination of the intra-frame interpolation filter is guided by the neighboring reconstructed samples of the current block. The determined intra-frame interpolation filter may be referred to as a self-guided intra-frame interpolation filter and may be applied to predict samples in the current block.

[0112] In one aspect, an intra-frame interpolation filter for a current block is determined based on neighboring reconstructed samples of the current block and an angular intra-frame prediction mode used to predict the current block. In an example, a first portion of neighboring reconstructed samples (e.g., Fig.11 The reference line 0 in FIG. 1 is a predefined set of intraframe interpolation filters, which are used to obtain the second part of the adjacent reconstructed samples (e.g., Fig.11 Reference row 1 in ), predicts using an angular intra prediction mode, and thus generates a predicted first part associated with the predefined set of intra interpolation filters. One of the predefined set of intra interpolation filters can be selected based on a comparison of the predicted first part with the first part. For example, based on the predicted first part and the first part, a prediction error is generated, and one of the predefined set of intra interpolation filters can be selected as the intra interpolation filter associated with the minimum prediction error. In the example, the predefined set of intra interpolation filters includes a first filter and a second filter. The predicted first part includes a predicted first part I predicted using the first filter and a predicted first part II predicted using the second filter. The prediction error includes a first prediction error between the predicted first part I and the first part and a second prediction error between the predicted first part II and the second part. If the first prediction error is the minimum prediction error (for example, the first prediction error is less than the second prediction error), the first filter can be selected as the intra prediction filter of the current block. If multiple parts (for example, reference rows 0 to 3) are used (rather than only the first part described above) to determine the intra prediction filter, the above description can be appropriately modified.

[0113] In one aspect, an intra interpolation filter and an angular intra prediction mode for predicting the current block are determined for the current block based on neighboring reconstructed samples of the current block.

[0114] Using a self-guided intra-frame interpolation filter can improve codec efficiency and thus can be more efficient than using a fixed interpolation filter in the related art. In an example, the fixed interpolation filter is predefined and does not depend on neighboring reconstructed samples of the current block. In an example, the current block is a luminance block. In an example, the current block is a chrominance block. MRL intra-frame prediction can be used to predict the current block, where a reference line other than reference line 0 (e.g., Fig. 9 Reference row 1) in , predict the samples in the current block.

[0115] In the present disclosure, the term "block" may be interpreted as a prediction block, a coding block, a coding unit (CU), etc. The term "current block" may be interpreted as a prediction block being encoded or decoded (e.g., in reconstruction), a coding block being encoded or decoded (e.g., in reconstruction), a coding unit (CU) being encoded or decoded (e.g., in reconstruction), etc. The method described in the present disclosure may be applicable to a variety of different video coding standards, including but not limited to AV1, AOMedia Video Model (AVM), AOMedia Video 2 (AV2), Versatile Video Coding (VVC), Enhanced Compression Model (ECM), H.267, etc.

[0116] The intra-frame interpolation filter used in the directional (or angular intra-frame prediction mode) intra-frame prediction mode can be derived or selected from the neighboring reconstructed samples of the current block. For example, the current block in the picture is encoded in the angular intra-frame prediction mode using the intra-frame interpolation filter. The intra-frame interpolation filter used in the angular intra-frame prediction mode can be determined (e.g., derived or selected) from a predefined set of intra-frame interpolation filters based on the neighboring samples of the current block. The neighboring samples may include, for example, neighboring reconstructed samples within N rows (e.g., N neighboring rows, N>1) from one or more boundaries of the current block. The determined intra-frame interpolation filter (e.g., the selected or derived intra-frame interpolation filter) can be applied to the reference samples in the reference row in the picture to predict the samples in the current block using the angular intra-frame prediction mode.

[0117] In one aspect, the value of N depends on the block size. N indicates the number of rows and / or columns in the plurality of reference rows. In one example, how many rows and / or columns are used depends on the block size. Whether to use adjacent reconstructed samples to derive an interpolation filter (e.g., an intra-frame interpolation filter) may depend on the block area. For example, when the block area is less than a threshold, the cost (e.g., prediction error) of each filter (e.g., each intra-frame interpolation filter in a predefined set of intra-frame interpolation filters) is derived using adjacent pixels (e.g., adjacent reconstructed samples) to save (e.g., reduce) signaling overhead.

[0118] Fig.11An example of a current block (1101) and neighboring reconstructed samples (1110) of the current block (1101) encoded using an angular intra prediction mode according to an aspect of the present disclosure is shown. In the example, the current block (1101) is being reconstructed. The prediction direction of the angular intra prediction mode may be referred to as an intra prediction direction. Fig.11 In the example shown, arrows (1121) to (1123) indicate the same direction, such as the prediction direction of the angular intra prediction mode. Arrows (1124) to (1126) may indicate the same direction, such as the direction opposite to the prediction direction of the angular intra prediction mode. The opposite direction (e.g., pointing downward, indicated by arrows (1124) to (1126)) may be directly opposite to the prediction direction of the angular intra prediction mode (e.g., pointing upward, indicated by arrows (1121) to (1123)).

[0119] The neighboring reconstructed samples (1110) of the current block (1101) may be referred to as neighboring reconstructed samples of the current block (1101). The neighboring reconstructed samples (1110) of the current block (1101) may include samples in a plurality of reference lines (eg, two or more reference lines). Fig.11 In the example shown, the neighboring reconstructed samples (1110) of the current block (1101) include neighboring reconstructed samples in reference rows 0 to (N-1), where N is 4. Fig.11 The described method can be applied to any suitable N greater than 1. The reference row i may include reference samples that are i rows away from the current block (1101), for example, i rows away from the boundary of the current block (1101) (e.g., i rows away from the top boundary and / or i columns away from the left boundary), where i is 0, 1, 2 or 3, such as Fig.11 As shown. For example, reference row i includes reference samples that are i rows above the top boundary of the current block (1101) and / or i columns to the left of the left boundary of the current block (1101). In the example, reference row 0 includes reference samples that are adjacent to the current block (1101), such as reconstructed adjacent samples including the top adjacent samples above the current block (1101) and the left adjacent samples to the left of the block (1101).

[0120] In an example, the neighboring reconstructed samples (1110) of the current block (1101) include a top reference line (1111), which is a reference line located above a top boundary of the current block (1101). In an example, the neighboring reconstructed samples (1110) of the current block (1101) include a left reference line (1112), which is a reference line located to the left of a left boundary of the current block (1101). In an example, the neighboring reconstructed samples (1110) of the current block (1101) include a top reference line (1111) and a left reference line (1112).

[0121] In one aspect, N is greater than 1, and an angular intra prediction mode has been determined (eg, associated with the prediction direction indicated by arrows (1121) to (1123)). The adjacent reconstructed samples may include multiple rows of adjacent reconstructed samples (eg, Fig.11 0 to 3 in the reference column).

[0122] Each intra-frame interpolation filter in a predefined set of intra-frame interpolation filters can be applied to each row of adjacent reconstructed samples in multiple rows of adjacent reconstructed samples to predict the adjacent reconstructed samples of the corresponding row using an angular intra-frame prediction mode, and a corresponding prediction error can be obtained. In the example, for each combination of an intra-frame interpolation filter in a predefined set of intra-frame interpolation filters and a row of adjacent reconstructed samples in multiple rows of adjacent reconstructed samples, the corresponding row of adjacent reconstructed samples can be predicted based on one or more remaining rows in the multiple rows of adjacent reconstructed samples using the corresponding intra-frame interpolation filter and the angular intra-frame prediction mode, and at least one prediction error can be obtained.

[0123] In the example, see Fig.11 , N is 4, multiple rows of adjacent reconstructed samples include reference rows 0 to 3, one of the intraframe interpolation filters in the predefined set of intraframe interpolation filters is a bilinear filter, and one row of adjacent reconstructed samples in the multiple rows of adjacent reconstructed samples is reference row 0. One or more remaining rows in the multiple rows of adjacent reconstructed samples include reference rows 1 to 3. The row of adjacent reconstructed samples (e.g., reference row 0) can be predicted based on reference rows 1 to 3, and the at least one prediction error includes 3 prediction errors associated with reference rows 1 to 3, respectively. For example, the first prediction error is based on the reconstructed samples in reference row 0 and the prediction samples of reference row 0, and the prediction samples of reference row 0 are based on reference row 1 and the angular intra-frame prediction mode associated with the prediction direction indicated by arrow (1121). Similarly, reference rows 1 to 3 can be predicted, and corresponding prediction errors are obtained for the bilinear filter. For example, reference row 1 is predicted based on reference rows 0, 2 and 3, the bilinear filter and the angular intra-frame prediction mode associated with the prediction directions indicated by arrows (1122) and (1124), respectively, and 3 prediction errors are obtained.

[0124] The above process for one intra-frame interpolation filter (e.g., a bilinear filter) in a predefined set of intra-frame interpolation filters may be performed or repeated for other intra-frame interpolation filters (e.g., a spline filter, a cubic filter, etc.) in a predefined set of intra-frame interpolation filters. The intra-frame interpolation filter corresponding to the minimum prediction error among the obtained prediction errors may be selected. For example, if the minimum prediction error is a prediction error obtained using a specific intra-frame interpolation filter (e.g., a bilinear filter), then for an angular intra-frame prediction mode (e.g., associated with a prediction direction indicated by arrows (1121) to (1123)), the specific intra-frame interpolation filter is selected as the intra-frame interpolation filter to be applied to the current block (1101).

[0125] In one aspect, given an angular intra prediction mode (also referred to as a directional intra prediction mode) and multiple rows of neighboring reconstructed samples (e.g., Fig.11 ), applying a predefined set of intraframe interpolation filters to the reference lines (e.g., Fig.11 Reference rows 0 to 3 in ( ) to predict each other along an intra prediction direction (e.g., pointing upward, the prediction direction indicated by arrows (1121) to (1123)) and / or an opposite direction (e.g., pointing downward, the opposite direction indicated by arrows (1124) to (1126)), and select an intra interpolation filter candidate (also referred to as a candidate interpolation filter) with the smallest prediction error as the intra interpolation filter of the current block (1101). Reference Fig.11 , reference row 0 is predicted by other reference rows such as reference rows 1, 2, 3, ... and (N-1) (indicated by arrow (1121)). Reference row 1 can be predicted by other reference rows such as reference rows 0, 2, 3, ... and (N-1) (indicated by arrows (1122) and (1124)). Reference row 2 can be predicted by other reference rows such as reference rows 0, 1, 3, ... and (N-1) (indicated by arrows (1123) and (1125)). Reference row 3 can be predicted by other reference rows such as reference rows 0, 1, 2, ... and (N-1) (indicated by arrow (1126)). Each candidate interpolation filter is applied to perform mutual prediction between reference rows, as described above, and one of the candidate interpolation filters with the smallest prediction error is used as the prediction filter (e.g., the determined intra-frame interpolation filter).

[0126] In one aspect, one of: (i) the type of intra interpolation filter, or (ii) the number of taps in the intra interpolation filter is selected from a predefined set of intra interpolation filters based on neighboring reconstructed samples of the current block.

[0127] In an example, a predefined set of intra-frame interpolation filters includes different types of intra-frame interpolation filters, such as one or more of a bilinear interpolation filter, a cubic interpolation filter, a spline interpolation filter, a DCT-based interpolation filter, or a DST-based interpolation filter. The type of intra-frame interpolation filter can be selected from different types of intra-frame interpolation filters based on adjacent reconstructed samples of the current block. The selected intra-frame interpolation filter can be one of a bilinear interpolation filter, a cubic interpolation filter, a spline interpolation filter, a DCT-based interpolation filter, or a DST-based interpolation filter.

[0128] For example, a predefined set of intra-frame interpolation filters (e.g., candidate intra-frame interpolation filters) may include but are not limited to bilinear filters, cubic filters, spline interpolation filters, DCT / DST-based interpolation filters (e.g., DCT or DST-based interpolation filters).

[0129] In an example, a set of intra-frame interpolation filters are predefined to include different numbers of taps, such as one or more of a 2-tap interpolation filter, a 4-tap interpolation filter, a 6-tap interpolation filter, or an 8-tap interpolation filter. The number of taps of the intra-frame interpolation filter can be selected based on the adjacent reconstructed samples of the current block. The number of taps of the intra-frame interpolation filter can be one of 2 taps, 4 taps, 6 taps, or 8 taps.

[0130] In an example, a predefined set of intra-frame interpolation filters (eg, candidate intra-frame interpolation filters) may have different numbers of filter taps. For example, the candidate intra-frame interpolation filters may be a set of 2-tap, 4-tap, 6-tap, and 8-tap filters.

[0131] In the example, whether the plurality of rows of adjacent reconstructed samples (e.g., reference rows 0 to (N-1)) include (i) a left reference row (1112) (e.g., a left row of adjacent reconstructed samples to the left of the current block (1101)), or (ii) a top reference row (1111) (e.g., a top row of adjacent reconstructed samples above the current block (1101)), or (iii) a left reference row (1112) (e.g., a left row of adjacent reconstructed samples to the left of the current block (1101)) and a top reference row (1111) (e.g., a top row of adjacent reconstructed samples above the current block (1101)) depends on the intra prediction direction of the angular intra prediction mode. Fig.11In the example shown, the intra prediction direction of the angular intra prediction mode points upward, as indicated by arrows (1121) to (1123). For example, whether to use a left reference line (e.g., left reference line (1112)), or an upper reference line (also referred to as a top reference line, such as top reference line (1111)), or both a left reference line and an upper reference line to derive an intra interpolation filter may depend on a given intra prediction direction, such as indicated by Fig.11 In the example, if the prediction direction is horizontal, the left reference row is used. In the example, if the prediction direction is vertical, the top reference row is used.

[0132] In one aspect, selectable filters are derived using only intra prediction directions. For example, each row of neighboring reconstructed samples in a plurality of rows is predicted based on an intra prediction direction of an angular intra prediction mode (such as the prediction direction indicated by arrows (1121) to (1123)). Fig.11 In this case, reference row 0 can be predicted from reference rows 1 to 3 based on the prediction direction indicated by arrow (1121); reference row 1 can be predicted from reference rows 2 to 3 based on the prediction direction indicated by arrow (1122); reference row 2 can be predicted from reference row 3 based on the prediction direction indicated by arrow (1123); and no prediction is performed on reference row 3.

[0133] In one aspect, selectable filters are derived using only directions opposite to the intra prediction direction. For example, each row of neighboring reconstructed samples in the plurality of rows (e.g., reference row 1) is predicted based on a direction opposite to the intra prediction direction of the angular intra prediction mode (such as the opposite direction indicated by arrows (1124) to (1126)). Fig.11 In this case, reference row 1 can be predicted from reference row 0 based on the opposite direction indicated by arrow (1124); reference row 2 can be predicted from reference rows 0 to 1 based on the opposite direction indicated by arrow (1125). Reference row 3 can be predicted from reference rows 0 to 2, respectively, based on the opposite direction indicated by arrow (1126); no prediction is performed on reference row 0.

[0134] According to an aspect of the present disclosure, selecting an intra interpolation filter from a predefined set of intra interpolation filters and selecting an angular intra prediction mode from a plurality of angular intra prediction modes (e.g., a predefined set of angular intra prediction modes) can be performed based on neighboring reconstructed samples of a current block (1101). Both the intra interpolation filter and the angular intra prediction mode can be selected based on neighboring reconstructed samples of the current block (1101).

[0135] In the example, the selection of both the directional intra prediction mode (or angular intra prediction mode) and the intra interpolation filter can be evaluated by a cost value calculated as the prediction error between the prediction reference lines, and the best L intra modes (e.g., one or more intra modes from a predefined set of angular intra prediction modes) and filters (e.g., one or more filters from a predefined set of intra prediction filters) are selected as candidates. The actual angular intra prediction mode (e.g., associated with the intra prediction angle) and the interpolation filter (e.g., the intra interpolation filter) used in the intra prediction process can be obtained from the best L intra modes and filter combinations.

[0136] In the example, the selection of intra-frame interpolation filters and angular intra-frame prediction modes includes the following. N1 angular intra-frame prediction modes from a plurality of angular intra-frame prediction modes (e.g., a predefined set of angular intra-frame prediction modes) can be derived, wherein the N1 angular intra-frame prediction modes are associated with the N1 lowest cost values ​​from the cost values ​​of the angular intra-frame prediction modes. Each cost value (e.g., the cost value of the corresponding angular intra-frame prediction mode) can be determined based on adjacent reconstructed samples of the current block (1101), the corresponding angular intra-frame prediction mode, and a default intra-frame interpolation filter (e.g., a filter from a predefined set of intra-frame interpolation filters). For each of the N1 angular intra-frame prediction modes, M1 intra-frame interpolation filters can be selected from a predefined set of intra-frame interpolation filters, and the M1 intra-frame interpolation filters are associated with the M1 lowest cost values ​​from the cost values ​​of the predefined set of intra-frame interpolation filters. Each cost value (e.g., the cost value of a corresponding filter in a predefined set of intra-frame interpolation filters) can be determined based on neighboring reconstructed samples of the current block, a corresponding intra-frame interpolation filter, and a corresponding angular intra-frame prediction mode in the N1 angular intra-frame prediction modes. The intra-frame interpolation filter and the angular intra-frame prediction mode can be selected from N1×M1 combinations, where each of the N1×M1 combinations can include one intra-frame interpolation filter in the M1 intra-frame interpolation filters and one angular intra-frame prediction mode in the N1 angular intra-frame prediction modes.

[0137] For example, a default intra-frame interpolation filter (which may be predefined) is applied, and the best N1 intra-frame prediction modes (e.g., the above-mentioned N1 angle intra-frame prediction modes) are derived using adjacent reconstructed samples. After selecting the best N1 intra-frame prediction modes, the best M1 interpolation filters may be further selected, for example, from a predefined set of intra-frame interpolation filters. Example values ​​of M1 and N1 may include, but are not limited to, 1, 2, 3, 4, etc. M1 and N1 may be positive integers.

[0138] In one aspect, the intra-frame prediction mode (e.g., the angular intra-frame prediction mode) and the intra-frame interpolation filter can be determined in an iterative manner by adjacent reconstructed samples. That is, the intra-frame prediction filter can be first selected using the above method, for example, the intra-frame prediction filter is selected as one of the M1 intra-frame interpolation filters. Then, by trying each intra-frame prediction mode in the adjacent reconstructed samples and selecting the L1 intra-frame prediction mode associated with the smallest L1 prediction errors using the intra-frame prediction filter, the best L1 intra-frame prediction mode is further selected. Subsequently, the intra-frame interpolation filter can be further modified using a similar method. Using the modified intra-frame interpolation filter, the intra-frame prediction mode can be further modified.

[0139] In an example, the iterative search for the intra-frame prediction mode (e.g., angular intra-frame prediction mode) and the intra-frame interpolation filter can be terminated when the prediction error no longer decreases, the prediction error is within a predefined threshold (e.g., a first predefined threshold), or the degree of reduction of the prediction error during the search process is within a predefined threshold (e.g., a second predefined threshold) (e.g., the difference in errors between consecutive iterations is within the second predefined threshold).

[0140] An example of the last two iterations (e.g., the first iteration and the second iteration) in the iterative process is described below. In the first iteration, N2 angular intra prediction modes may be selected from a first plurality of angular intra prediction modes based on neighboring reconstructed samples of the current block and the intra interpolation filter. A first updated intra interpolation filter may be selected based on neighboring reconstructed samples of the current block and one of the N2 directional intra prediction modes, such as Fig.11As described in . In a second iteration, N3 angular intra prediction modes may be selected from a second plurality of angular intra prediction modes based on adjacent reconstructed samples of the current block and the first updated intra interpolation filter. A second updated intra interpolation filter may be selected based on adjacent reconstructed samples of the current block and one of the N3 angular intra prediction modes. The iterative process may be terminated and the intra interpolation filter and the angular intra prediction mode may be selected based on one of the following: (i) a first prediction error of the first iteration and a second prediction error of the second iteration, and (ii) a second prediction error of the second iteration. The first prediction error may be associated with the first updated intra interpolation filter and one of the N2 directional intra prediction modes. The second prediction error may be associated with the second updated intra interpolation filter and one of the N3 angular intra prediction modes. In an example, based on a difference between the first prediction error and the second prediction error (such as an absolute difference between the first prediction error and the second prediction error being within a second predefined threshold), a second updated intra-frame interpolation filter is selected as the selected intra-frame interpolation filter, and one of the N3 angular intra-frame prediction modes is selected as the selected angular intra-frame prediction mode. In an example, based on a second prediction error (such as a second prediction error being within a first predefined threshold), a second updated intra-frame interpolation filter is selected as the selected intra-frame interpolation filter, and one of the N3 angular intra-frame prediction modes is selected as the selected angular intra-frame prediction mode.

[0141] In an example, the first plurality of modes and the second plurality of modes are the same as the plurality of angular intra prediction modes. In an example, the first plurality of modes and the second plurality of modes are subsets of the plurality of angular intra prediction modes. The subset of the plurality of angular intra prediction modes may be obtained from a previous iteration. For example, the second plurality of modes is N2 angular intra prediction modes.

[0142] In one aspect, the angular intra prediction mode and the intra interpolation filter can be determined based on adjacent reconstructed samples in another iterative manner: first, the angular intra prediction mode is selected, then the best intra prediction filter can be obtained, and then the angular intra prediction mode is modified using a similar method (e.g., based on one of the best intra prediction filters). Using the modified angular intra prediction mode, the intra prediction filter can be further modified.

[0143] According to one aspect of the present disclosure, for each combination of an angular intra prediction mode in a plurality of angular intra prediction modes and an intra interpolation filter from a predefined set of intra interpolation filters, for example, using Fig.11The method described in the embodiment determines the cost value of the corresponding combination based on the neighboring reconstructed samples of the current block. In the example, the cost value is determined based on the reconstructed samples in the reference row (e.g., reference row 0) and the predicted samples of the reference row (e.g., predicted by the reference row 1 using the angle intra prediction mode and the intra interpolation filter in the combination), such as Fig.11 In an example, based on the reconstructed samples in a reference row (e.g., reference row 0) and the predicted samples of the reference row predicted using the reference rows (e.g., predicted by reference rows 1 to 3 using the angle intra prediction mode and the intra interpolation filter in the combination), a plurality of cost values ​​are determined, and the cost value is one of the plurality of cost values ​​(e.g., the minimum cost value), such as Fig.11 As shown in . K combinations can be selected based on the determined multiple cost values. The intra-frame interpolation filter and the angular intra-frame prediction mode can be selected from the K combinations based on the index information signaled in the code stream. In the example, the combinations of the angular intra-frame prediction mode and the intra-frame interpolation filter are, for example, sorted in ascending order of the cost value, and the K combinations are associated with the K minimum cost values ​​among the multiple cost values.

[0144] In one example, the selection of directional intra prediction modes and intra interpolation filters by using adjacent reconstructed samples can be performed at both the encoder and decoder sides. Multiple combinations of intra prediction modes (e.g., angular intra prediction modes) and intra interpolation filters can be selected. In one example, the order of multiple combinations of directional intra prediction modes and intra interpolation filters is determined based on the calculated cost value of each combination. Fig.11 An example of calculating the cost value of each combination is shown. For example, a syntax element may be signaled in a bitstream to indicate which combination is used for the current block. In an example, multiple combinations of intra prediction modes (e.g., angular intra prediction modes) and intra interpolation filters may be sorted, for example, based on the calculated cost values ​​of the combinations.

[0145] According to one aspect of the present disclosure, a feature value can be calculated based on adjacent reconstructed samples of a current block, and an intra-frame interpolation filter can be selected from a predefined set of intra-frame interpolation filters based on the calculated feature value. The feature value can be calculated as an absolute gradient value associated with the adjacent reconstructed samples of the current block, or the difference between the minimum value of the adjacent reconstructed samples of the current block and the maximum value of the adjacent reconstructed samples of the current block. For example, given adjacent reconstructed samples, feature values ​​such as an absolute gradient value and a difference between the minimum and maximum values ​​(e.g., the difference between the minimum and the maximum value) are calculated. Subsequently, based on the feature value, an intra-frame interpolation filter (e.g., an intra-frame interpolation filter to be used in an angular intra-frame prediction mode to predict the current block) can be selected from a set of predefined intra-frame interpolation filters (e.g., the predefined set of intra-frame interpolation filters). In the example, the intra-frame interpolation filter with the maximum gradient value or the minimum gradient value is selected. In the example, the absolute gradient value is calculated based on one or more predefined directions.

[0146] According to one aspect of the present disclosure, an output may be obtained from a neural network (e.g., a convolutional neural network (CNN)). The input of the neural network may include neighboring reconstructed samples of a current block, and the output (e.g., an index) indicates a selected intra-frame interpolation filter, which is one of a predefined set of intra-frame interpolation filters. In an example, the input of the neural network also includes an intra-frame prediction mode index indicating an angular intra-frame prediction mode.

[0147] In an example, adjacent reconstructed samples are fed into a convolutional neural network (CNN), and the output includes an index of an intra-frame interpolation filter selected from a set of predefined intra-frame interpolation filters (e.g., the predefined set of intra-frame interpolation filters). Network parameters (e.g., parameters of the CNN) can be pre-trained and evaluated on both the encoder side and the decoder side, or trained online using reconstructed blocks. In one example, the intra-frame prediction mode index is fed into the CNN together with the adjacent reconstructed samples to derive the intra-frame interpolation filter.

[0148] The description of (i) determining (e.g., selecting) an intra-frame interpolation filter or (ii) determining (e.g., selecting) an intra-frame interpolation filter and an angular intra-frame prediction mode for a current block (1101) can be based on any suitable neighboring reconstructed samples of the current block (1101). The size of the neighboring reconstructed samples of the current block (1101) can depend on (e.g., increase with) the block size of the current block (1101). Fig.11, the top reference line (1111) and the left reference line (1112) may include the area (1113) as the upper left neighbor of the current block (1101). In some examples, the top reference line (1111) and / or the left reference line (1112) may include a portion of the area (1113). In some examples, the top reference line (1111) and / or the left reference line (1112) may not include the area (1113).

[0149] Fig.12 A flow chart outlining a process (1200) according to an aspect of the present disclosure is shown. The process (1200) may be used in a video decoder. In various aspects, the process (1200) is performed by a processing circuit, such as a processing circuit that performs the functions of a video decoder (110), a processing circuit that performs the functions of a video decoder (210), etc. In some aspects, the process (1200) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (1200). The process (1200) begins at (S1201) and proceeds to (S1210).

[0150] At (S1210), a code stream of a current block in a picture is received. The encoding information of the code stream may indicate that the current block is encoded in an angular intra prediction mode using an intra interpolation filter.

[0151] At (S1220), an intra-frame interpolation filter may be determined (e.g., selected) from a predefined set of intra-frame interpolation filters based on neighboring reconstructed samples of the current block. The neighboring reconstructed samples may include reconstructed samples within N rows from one or more boundaries of the current block, such as Fig.11 shown.

[0152] In one aspect, one of: (i) the type of intra interpolation filter, or (ii) the number of taps in the intra interpolation filter is selected from a predefined set of intra interpolation filters based on neighboring reconstructed samples of the current block.

[0153] In an example, a predefined set of intraframe interpolation filters includes different types of intraframe interpolation filters. The type of the intraframe interpolation filter can be selected from different types of intraframe interpolation filters based on adjacent reconstructed samples of the current block. The selected intraframe interpolation filter can be one of a bilinear interpolation filter, a cubic interpolation filter, a spline interpolation filter, a DCT-based interpolation filter, or a DST-based interpolation filter.

[0154] In an example, a set of intra-frame interpolation filters are predefined to include different numbers of taps. The number of taps of the intra-frame interpolation filter can be selected based on the neighboring reconstructed samples of the current block. The number of taps of the intra-frame interpolation filter can be 2 taps, 4 taps, 6 taps or 8 taps.

[0155] In an example, whether the neighboring reconstructed samples include (i) a left neighboring reconstructed sample row to the left of the current block, (ii) a top neighboring reconstructed sample row above the current block, or (iii) a left neighboring reconstructed sample row to the left of the current block and a top neighboring reconstructed sample row above the current block depends on the intra prediction direction of the angular intra prediction mode.

[0156] In the example, the value of N depends on the block size.

[0157] In one aspect, N is greater than 1, and the adjacent reconstructed samples include multiple rows of adjacent reconstructed samples. For each combination of an intraframe interpolation filter in a predefined set of intraframe interpolation filters and a row of adjacent samples in the multiple rows of adjacent reconstructed samples, the corresponding intraframe interpolation filter may be used to predict the adjacent reconstructed samples of the corresponding row based on one or more remaining rows in the multiple rows of adjacent reconstructed samples, and at least one prediction error may be obtained. The intraframe interpolation filter corresponding to the minimum prediction error among the obtained prediction errors may be selected.

[0158] In an example, each row of neighboring reconstructed samples is predicted based on an intra prediction direction of an angular intra prediction mode. In an example, each row of neighboring reconstructed samples is predicted based on a direction opposite to the intra prediction direction of the angular intra prediction mode.

[0159] In one aspect, based on neighboring reconstructed samples of the current block, (i) an intra interpolation filter is selected from a predefined set of intra interpolation filters, and (ii) an angular intra prediction mode is selected from a plurality of angular intra prediction modes.

[0160] In the example, N1 angular intra prediction modes are derived from a plurality of angular intra prediction modes, and the N1 angular intra prediction modes are associated with the N1 lowest cost values ​​among the cost values ​​of the angular intra prediction modes. Each cost value can be determined based on the adjacent reconstructed samples of the current block, the corresponding angular intra prediction mode, and the default intra interpolation filter in a predefined set of intra interpolation filters. For each of the N1 angular intra prediction modes, M1 intra interpolation filters can be selected from a predefined set of intra interpolation filters, and the M1 intra interpolation filters are associated with the M1 lowest cost values ​​among the cost values ​​of the predefined set of intra interpolation filters. Each cost value can be determined based on the adjacent reconstructed samples of the current block, the corresponding intra interpolation filter, and the corresponding angular intra prediction mode in the N1 angular intra prediction modes. The intra interpolation filter and the angular intra prediction mode can be selected from the N1×M1 combination. Each of the N1×M1 combinations may include an intra interpolation filter from among the M1 intra interpolation filters and an angular intra prediction mode from among the N1 angular intra prediction modes.

[0161] In the example, the selection of the angular intra prediction mode and the intra interpolation filter is performed iteratively in an iterative process. The iterative process may include: selecting N2 angular intra prediction modes from a first plurality of angular intra prediction modes based on the neighboring reconstructed samples of the current block and the intra interpolation filter; selecting a first updated intra interpolation filter from a second plurality of angular intra prediction modes based on the neighboring reconstructed samples of the current block and one of the N2 directional intra prediction modes; selecting N3 angular intra prediction modes from a second plurality of angular intra prediction modes based on the neighboring reconstructed samples of the current block and the first updated intra interpolation filter; selecting N4 angular intra prediction modes from a second plurality of angular intra prediction modes based on the neighboring reconstructed samples of the current block and the N4 angular intra prediction modes. An angular intra-frame prediction mode, selecting a second updated intra-frame interpolation filter; and based on (i) a difference between a first prediction error associated with the first updated intra-frame interpolation filter and one of the N2 directional intra-frame prediction modes and a second prediction error associated with the second updated intra-frame interpolation filter and one of the N3 angular intra-frame prediction modes, or (ii) a second prediction error, selecting the second updated intra-frame interpolation filter as the selected intra-frame interpolation filter, and selecting one of the N3 angular intra-frame prediction modes as the selected angular intra-frame prediction mode. In the example, the first multiple modes and the second multiple modes are the same as the multiple angular intra-frame prediction modes. In the example, the first multiple modes and the second multiple modes are subsets of the multiple angular intra-frame prediction modes. The subset of the multiple angular intra-frame prediction modes can be obtained from a previous iteration. For example, the second multiple modes are N2 angular intra-frame prediction modes.

[0162] In an example, for each combination of an angular intra prediction mode in a plurality of angular intra prediction modes and an intra interpolation filter from a predefined set of intra interpolation filters, a cost value of each combination is determined based on neighboring reconstructed samples of a current block. K combinations may be selected based on the determined cost values. The intra interpolation filter and the angular intra prediction mode may be selected from the K combinations based on index information signaled in a bitstream.

[0163] In one aspect, a feature value is calculated based on neighboring reconstructed samples of the current block, and an intra-frame interpolation filter may be selected based on the calculated feature value.

[0164] In one aspect, a neural network (such as a CNN) may be used to determine an intra-frame interpolation filter for a current block. An input to the neural network includes neighboring reconstructed samples of the current block, and an output is obtained from the neural network based on the input. The output may indicate the selected intra-frame interpolation filter. In an example, the input to the neural network also includes an intra-frame prediction mode index indicating an angular intra-frame prediction mode.

[0165] At (S1230), the selected intra interpolation filter may be applied to reference samples in a reference row in the picture to predict samples in the current block using an angular intra prediction mode, such as Fig.10 shown.

[0166] At (S1240), the current block may be reconstructed based on the predicted samples.

[0167] Then, the process proceeds to (S1299) and terminates.

[0168] The process (1200) may be adapted as appropriate. One or more steps in the process (1200) may be modified and / or omitted. One or more additional steps may be added. Any suitable implementation order may be used. Fig.14 A variation of process (1200) is shown.

[0169] Fig.13 A flow chart outlining a process (1300) according to an aspect of the present disclosure is shown. The process (1300) may be used in a video encoder. In various aspects, the process (1300) is performed by a processing circuit, such as a processing circuit that performs the functions of the video encoder (103), a processing circuit that performs the functions of the video encoder (303), etc. In some aspects, the process (1300) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (1300). The process begins at (S1301) and proceeds to (S1310).

[0170] At (S1310), an intra-frame interpolation filter may be selected from a predefined set of intra-frame interpolation filters based on neighboring reconstructed samples of the current block. The neighboring reconstructed samples may include reconstructed samples within N rows from one or more boundaries of the current block. The current block is encoded in an angular intra-frame prediction mode by the selected intra-frame interpolation filter.

[0171] At (S1320), the selected intra interpolation filter may be applied to reference samples in a reference row in the picture to predict samples in the current block using an angular intra prediction mode.

[0172] At (S1330), the current block is encoded based on the predicted samples.

[0173] Then, the process proceeds to (S1399) and terminates.

[0174] The process (1300) may be adapted as appropriate. One or more steps in the process (1300) may be modified and / or omitted. One or more additional steps may be added. Any suitable order of implementation may be used.

[0175] Fig.14 A flow chart outlining a process (1400) according to an aspect of the present disclosure is shown. The process (1400) may be used in a video decoder. In various aspects, the process (1400) is performed by a processing circuit, such as a processing circuit that performs the functions of a video decoder (110), a processing circuit that performs the functions of a video decoder (210), etc. In some aspects, the process (1400) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (1400). The process (1400) begins at (S1401) and proceeds to (S1410).

[0176] At (S1410), encoding information of a code stream is received from a code stream including a current block in a picture, the encoding information indicating that the current block is encoded in an angular intra prediction mode using an intra interpolation filter.

[0177] At (S1420), each of a predefined set of intraframe interpolation filters is applied to adjacent reconstructed samples within N adjacent rows from the boundary of the current block. N is a positive integer, such as Fig.11 Described in .

[0178] At (S1430), based on the prediction error associated with each of the predefined set of intraframe interpolation filters, an intraframe interpolation filter is selected from the predefined set of intraframe interpolation filters, such as Fig.11 and Fig.12 Described in .

[0179] At (S1440), using the selected one intra-frame interpolation filter, the samples in the current block are predicted using the angular intra-frame prediction mode, such as Fig.12 Described in .

[0180] At (S1450), based on the predicted samples, the current block is reconstructed.

[0181] Then, the process proceeds to (S1499) and terminates.

[0182] The process (1400) may be adapted as appropriate. One or more steps in the process (1400) may be modified and / or omitted. One or more additional steps may be added. Any suitable order of implementation may be used.

[0183] The aspects and examples in the present disclosure may be used alone or in any order. In addition, each of the method (or aspect, example), encoder, and decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-volatile computer-readable medium.

[0184] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Fig.15 A computer system (1500) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0185] The computer software may be encoded using any suitable machine code or computer language, which may be subjected to assembly, compilation, linking or similar mechanisms to create code comprising instructions that may be executed directly or through interpretation, microcode execution, etc., by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0186] The instructions may be executed on various types of computers or computer components, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0187] Fig.15 The components shown for the computer system (1500) are exemplary in nature and are not intended to imply any limitation on the scope of use or functionality of the computer software implementing aspects of the application. Nor should the configuration of components be interpreted as having any dependency or requirement on any one or combination of components shown in the exemplary aspects of the computer system (1500).

[0188] The computer system (1500) may include certain human-machine interface input devices. Such human-machine interface input devices may be responsive to input from one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, taps), visual input (e.g., gestures), olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that may not be directly related to a person's conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0189] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard (1501), mouse (1502), track pad (1503), touch screen (1510), data gloves (not shown), joystick (1505), microphone (1506), scanner (1507), camera (1508).

[0190] The computer system (1500) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (1510), a data glove (not shown), or a joystick (1505), although there may also be tactile feedback devices that do not serve as input devices), audio output devices (e.g., speakers (1509), headphones (not depicted)), visual output devices (e.g., screens (1510), including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light emitting diode (OLED) screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which are capable of outputting two-dimensional visual output or output greater than three dimensions through, for example, stereographic output; virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted), and printers (not depicted).

[0191] The computer system (1500) may also include human-accessible storage devices and associated media for the storage devices, such as optical media including CD / DVD ROM / RW (1520) with CD / DVD etc. media (1521), thumb drives (1522), removable hard drives or solid-state drives (1523), legacy magnetic media such as tapes and floppy disks (not depicted), ROM / Application Specific Integrated Circuit (ASIC) / Programmable Logic Device (PLD) based specialized devices such as security devices (not depicted), and the like.

[0192] Those skilled in the art will also understand that the term "computer-readable medium" used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transient signals.

[0193] The computer system (1500) may also include an interface (1554) to one or more communication networks (1555). The network may be, for example, wireless, wired, optical. The network may also be local, wide area, metropolitan, vehicle-mounted and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including Global System for Mobile Communications (GSM), third generation (3G), fourth generation (4G), fifth generation (5G), long term evolution (LTE), etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicle-mounted networks and industrial networks including controller area network bus (CANBus), and the like. Some networks typically require an external network interface adapter attached to some general data port or peripheral bus (1549) (e.g., a universal serial bus (USB) port of the computer system (1500)); other networks are typically integrated into the core of the computer system (1500) by attaching to a system bus as described below (e.g., integrated into a PC computer system through an Ethernet interface, or integrated into a smart phone computer system through a cellular network interface). Using any of these networks, the computer system (1500) can communicate with other entities. Such communications can be one-way receive only (e.g., broadcast TV), one-way send only (e.g., CANBus connected to some CANBus devices), or bidirectional, for example, using a local area digital network or a wide area digital network to connect to other computer systems. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

[0194] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the core (1540) of the computer system (1500).

[0195] The core (1540) may include one or more central processing units (CPUs) (1541), graphics processing units (GPUs) (1542), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1543), hardware accelerators for certain tasks (1544), graphics adapters (1550), and the like. These devices, along with read-only memory (ROM) (1545), random access memory (1546), and internal mass storage devices (1547) such as internal non-user accessible hard drives, solid-state drives (SSDs), etc., may be connected via a system bus (1548). In some computer systems, the system bus (1548) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, and the like. Peripheral devices may be attached to the core's system bus (1548) directly or via a peripheral bus (1549). In an example, a screen (1510) may be connected to a graphics adapter (1550). Architectures for peripheral buses include peripheral component interconnect (PCI), USB, and the like.

[0196] The CPU (1541), GPU (1542), FPGA (1543) and accelerator (1544) can execute certain instructions, which in combination can constitute the above-mentioned computer code. The computer code can be stored in ROM (1545) or RAM (1546). Transient data can also be stored in RAM (1546), while permanent data can be stored, for example, in an internal mass storage device (1547). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1541), GPUs (1542), mass storage devices (1547), ROMs (1545), RAMs (1546), etc.

[0197] The computer readable medium may have computer codes for performing various computer-implemented operations. The media and computer codes may be media and computer codes designed and constructed specifically for the purpose of this application, or may be of a type well known and available to technicians in the computer software field.

[0198] By way of example and not limitation, a computer system having the architecture (1500) and in particular the core (1540) may provide functionality resulting from the execution of software embodied in one or more tangible computer-readable media by a processor (including a CPU, GPU, FPGA, accelerator, etc.). Such computer-readable media may be media associated with the user-accessible mass storage device introduced above and certain storage devices of a non-temporary nature of the core (1540) (e.g., a core internal mass storage device (1547) or ROM (1545)). Software implementing various aspects of the present application may be stored in such devices and executed by the core (1540). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software may enable the core (1540) and, in particular, the processor therein (including a CPU, GPU, FPGA, etc.) to perform specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1546) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality produced by logic hardwired or otherwise embodied in circuits (e.g., accelerators (1544)) that may operate in place of or in conjunction with software to perform specific processes or specific portions of specific processes described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits embodying logic for execution, or both, where appropriate. The present application encompasses any suitable combination of hardware and software.

[0199] As used in this disclosure, "at least one" or "one of" is intended to include any one or combination of the recited elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; at least one of A through C is intended to include only A, only B, only C, or any combination thereof. References to "one of A or B" and "one of A and B" are intended to include A or B or (A and B). The use of "one of" does not exclude any combination of the recited elements where applicable, such as when the elements are not mutually exclusive.

[0200] Although the present application describes several exemplary aspects, various modifications, permutations and combinations, and various alternative equivalents are possible within the scope of the present application. Therefore, it should be understood that within the spirit and scope of the application, those skilled in the art can design various systems and methods that are not explicitly shown or described herein but can embody the principles of the present application.

Claims

1. A video decoding method, characterized in that: include: Receiving encoding information of a code stream from a code stream including a current block in a picture, the encoding information indicating that the current block is encoded in an angular intra prediction mode using an intra interpolation filter; Applying each of a predefined set of intraframe interpolation filters to adjacent reconstructed samples within N adjacent rows from a boundary of the current block, where N is a positive integer; selecting an intraframe interpolation filter from the predefined set of intraframe interpolation filters based on a prediction error associated with each of the predefined set of intraframe interpolation filters; Using the selected one intra-frame interpolation filter, predicting samples in the current block using the angular intra-frame prediction mode; and Based on the predicted samples, the current block is reconstructed.

2. The method according to claim 1, characterized in that The step of selecting an intra-frame interpolation filter comprises: Based on the neighboring reconstructed samples of the current block, one of the following is selected from the predefined set of intra-frame interpolation filters: (i) a type of the one intra-frame interpolation filter, or (ii) a number of taps in the one intra-frame interpolation filter.

3. The method according to claim 2, characterized in that The predefined set of intraframe interpolation filters includes different types of intraframe interpolation filters; The selecting of an intraframe interpolation filter comprises: based on the adjacent reconstructed samples of the current block, selecting the type of the intraframe interpolation filter from the different types of intraframe interpolation filters, the selected intraframe interpolation filter being one of a bilinear interpolation filter, a cubic interpolation filter, a spline interpolation filter, a DCT-based interpolation filter or a DST-based interpolation filter.

4. The method according to claim 2, characterized in that: The predefined set of intraframe interpolation filters includes different numbers of taps; The selecting an intra-frame interpolation filter comprises: selecting the number of taps of the intra-frame interpolation filter based on the adjacent reconstructed samples of the current block, wherein the number of taps of the intra-frame interpolation filter is one of 2 taps, 4 taps, 6 taps or 8 taps.

5. The method according to claim 1, characterized in that N is greater than 1, and the adjacent reconstructed samples include a plurality of rows of adjacent reconstructed samples; The selecting an intraframe interpolation filter comprises: for each combination of an intraframe interpolation filter in the predefined set of intraframe interpolation filters and a row of adjacent reconstructed samples in the plurality of rows of adjacent reconstructed samples, Using a corresponding intra-frame interpolation filter, predicting the adjacent reconstructed samples of the corresponding row based on one or more remaining rows of the plurality of rows of adjacent reconstructed samples, and obtaining at least one prediction error; and An intra-frame interpolation filter corresponding to the minimum prediction error among the obtained prediction errors is selected.

6. The method according to claim 1, characterized in that Whether the adjacent reconstructed samples include (i) a left adjacent reconstructed sample row to the left of the current block, or (ii) a top adjacent reconstructed sample row above the current block, or (iii) a left adjacent reconstructed sample row to the left of the current block and a top adjacent reconstructed sample row above the current block depends on the intra prediction direction of the angular intra prediction mode.

7. The method according to claim 1, characterized in that The value of N depends on the block size.

8. The method according to claim 5, characterized in that The predicting of the adjacent reconstructed samples of the corresponding row includes: predicting each row of adjacent reconstructed samples based on the intra prediction direction of the angular intra prediction mode.

9. The method according to claim 5, characterized in that The predicting of the adjacent reconstructed samples of the corresponding row includes: predicting each row of adjacent reconstructed samples based on a direction opposite to the intra prediction direction of the angular intra prediction mode.

10. The method according to claim 1, characterized in that The selecting an intra-frame interpolation filter comprises: selecting the intra-frame interpolation filter from the predefined set of intra-frame interpolation filters based on the adjacent reconstructed samples of the current block, and selecting the angular intra-frame prediction mode from multiple angular intra-frame prediction modes.

11. The method according to claim 10, characterized in that Selecting the one intra-frame interpolation filter and the angular intra-frame prediction mode comprises: deriving N1 angular intra prediction modes from the plurality of angular intra prediction modes, the N1 angular intra prediction modes being associated with N1 lowest cost values ​​among the cost values ​​of the angular intra prediction modes, wherein each cost value is determined based on the neighboring reconstructed samples of the current block, the corresponding angular intra prediction mode, and a default intra interpolation filter from the predefined set of intra interpolation filters; for each of the N1 angular intra prediction modes, selecting M1 intra interpolation filters from the predefined set of intra interpolation filters, the M1 intra interpolation filters being associated with M1 lowest cost values ​​among the cost values ​​of the predefined set of intra interpolation filters, wherein each cost value is determined based on the neighboring reconstructed samples of the current block, a corresponding intra interpolation filter, and the corresponding angular intra prediction mode of the N1 angular intra prediction modes; and The one intra-frame interpolation filter and the angular intra-frame prediction mode are selected from N1×M1 combinations, each of which includes an intra-frame interpolation filter among the M1 intra-frame interpolation filters and an angular intra-frame prediction mode among the N1 angular intra-frame prediction modes.

12. The method according to claim 10, characterized in that Selecting the one intra-frame interpolation filter and the angular intra-frame prediction mode comprises: Selecting N2 angular intra prediction modes from a first plurality of the plurality of angular intra prediction modes based on the neighboring reconstructed samples of the current block and an intra interpolation filter; Selecting a first updated intra interpolation filter based on the neighboring reconstructed samples of the current block and one of the N2 angular intra prediction modes; selecting, based on the neighboring reconstructed samples of the current block and the first updated intra interpolation filter, N3 angular intra prediction modes from a second plurality of the plurality of angular intra prediction modes; Selecting a second updated intra interpolation filter based on the neighboring reconstructed samples of the current block and one of the N3 angular intra prediction modes; and Based on (i) the difference between a first prediction error associated with the first updated intra-frame interpolation filter and the one angular intra-frame prediction mode among the N2 angular intra-frame prediction modes and a second prediction error associated with the second updated intra-frame interpolation filter and the one angular intra-frame prediction mode among the N3 angular intra-frame prediction modes, or (ii) the second prediction error, select the second updated intra-frame interpolation filter as the selected one intra-frame interpolation filter, and select the one angular intra-frame prediction mode among the N3 angular intra-frame prediction modes as the selected angular intra-frame prediction mode.

13. The method according to claim 10, characterized in that Selecting the one intra-frame interpolation filter and the angular intra-frame prediction mode comprises: for each combination of an angular intra prediction mode from the plurality of angular intra prediction modes and an intra interpolation filter from the predefined set of intra interpolation filters, Determining a cost value for each combination based on the neighboring reconstructed samples of the current block; Based on the determined cost values, selecting K combinations; and The one intra-frame interpolation filter and the angular intra-frame prediction mode are selected from the K combinations based on index information signaled in the code stream.

14. The method according to claim 1, characterized in that Selecting the one intra-frame interpolation filter comprises: Calculating a feature value based on the adjacent reconstructed samples of the current block; and Based on the calculated feature value, the one intraframe interpolation filter is selected.

15. The method according to claim 14, characterized in that Calculating the characteristic value includes: The feature value is calculated as one of: (i) an absolute gradient value associated with the neighboring reconstructed samples of the current block, or (ii) a difference between a minimum value of the neighboring reconstructed samples of the current block and a maximum value of the neighboring reconstructed samples of the current block.

16. The method according to claim 1, characterized in that Selecting the one intra-frame interpolation filter comprises: An output of a neural network is obtained, wherein the input of the neural network includes the neighboring reconstructed samples of the current block, and the output indicates a selected intra-frame interpolation filter.

17. The method according to claim 16, characterized in that The input to the neural network also includes an intra-prediction mode index indicating the angular intra-prediction mode.

18. A video decoding device, characterized in that: include: A processing circuit configured to: Receiving encoding information of a code stream from a code stream including a current block in a picture, the encoding information indicating that the current block is encoded in an angular intra prediction mode using an intra interpolation filter; Applying each of a predefined set of intraframe interpolation filters to adjacent reconstructed samples within N adjacent rows from a boundary of the current block, where N is a positive integer; selecting an intraframe interpolation filter from the predefined set of intraframe interpolation filters based on a prediction error associated with each of the predefined set of intraframe interpolation filters; Predicting samples in the current block using the angular intra prediction mode using the selected one intra interpolation filter; and Based on the predicted samples, the current block is reconstructed.

19. The device according to claim 18, characterized in that The processing circuit is configured to: Based on the neighboring reconstructed samples of the current block, one of the following is selected from the predefined set of intra-frame interpolation filters: (i) a type of the one intra-frame interpolation filter, or (ii) a number of taps in the one intra-frame interpolation filter.

20. The device according to claim 18, characterized in that N is greater than 1, the adjacent reconstructed samples include a plurality of rows of adjacent reconstructed samples; and The processing circuit is configured to, for each combination of an intraframe interpolation filter in the predefined set of intraframe interpolation filters and a row of adjacent reconstructed samples in the plurality of rows of adjacent reconstructed samples, Using a corresponding intra-frame interpolation filter, predicting the adjacent reconstructed samples of the corresponding row based on one or more remaining rows of the plurality of rows of adjacent reconstructed samples, and obtaining at least one prediction error; and An intra-frame interpolation filter corresponding to the minimum prediction error among the obtained prediction errors is selected.

Images