Method and apparatus for intra prediction mode

Abstract

The current block is encoded using intra-frame prediction based on an extrapolation filter, chosen from a pool of candidate models. Each candidate model is associated with a corresponding filter and includes multiple filter coefficients for that filter. Reference values ​​are determined based on neighboring samples in the template region of the current block. A candidate model for predicting the current block is selected from the pool of candidate models based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model chosen from the pool of candidate models. The current block is reconstructed based on the portion of the predicted sample values ​​that falls within the limiting range.

Description

[0001] Incorporation This application claims priority to U.S. Patent Application No. 19 / 266,130, filed July 10, 2025, entitled "Method and Apparatus for Intra Prediction Mode," which claims priority to U.S. Provisional Application No. 63 / 671,180, filed July 13, 2024, entitled "Multi-model Extrapolation Filter-based Intra Prediction with Clipping," U.S. Provisional Application No. 63 / 671,265, filed July 14, 2024, entitled "Mulfti-Hypothesis Intra Prediction," and U.S. Provisional Application No. 63 / 671,265, filed July 16, 2024, entitled "Directional Padding on PictureBoundary." U.S. Provisional Application No. 63 / 672,208 entitled "Reference Sample Shifting for Intra Prediction" (filed October 19, 2024), U.S. Provisional Application No. 63 / 709,441 entitled "Reference Sample Shifting for Intra Prediction" (filed November 5, 2024), U.S. Provisional Application No. 63 / 716,717 entitled "Intra-mode Derivation with Signaled Syntax and Most Probable Mode Adaptation" (filed November 5, 2024), and U.S. Provisional Application No. 63 / 717,865 entitled "Histogram of Gradients-Based Directional Planar" (filed November 7, 2024), are hereby granted priority to this application. The entire disclosure of these prior applications is incorporated herein by reference. Technical Field

[0002] This invention describes various aspects generally related to video encoding and decoding. Background Technology

[0003] The background description provided in this application is intended to generally present the context of the invention. To the extent that the work described in this background section is intended, the work of the currently named inventors and aspects of the description that may not have otherwise qualified as prior art at the time of filing are neither explicitly nor implicitly considered as prior art to the invention.

[0004] Image / video compression facilitates the transfer of image / video data across different devices, storage devices, and networks with minimal quality loss. In some examples, video codecs can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which compresses images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current image being reconstructed to predict samples. In another example, a video codec can use a technique called inter-frame prediction, which compresses images based on temporal redundancy. For example, inter-frame prediction can utilize motion compensation (MC) to predict samples in the current image based on previously reconstructed images. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Various aspects of the present invention include bitstreams, methods, and apparatus for video encoding / decoding. In some examples, the apparatus for video encoding / decoding includes processing circuitry.

[0006] According to one aspect of the present invention, a video decoding method is provided. In this method, a video stream is received, the video stream including encoded information of a current block in a current image. The encoded information indicates that the current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients of the corresponding filter. A reference value is determined based on neighboring samples in a template region of the current block. Based on a comparison between the reference value and a threshold, a candidate model from the plurality of candidate models for predicting the current block is determined. A limiting range for the predicted sample values ​​of the current block is determined. Based on the candidate model from the plurality of candidate models, predicted sample values ​​of the current block are obtained. The current block is reconstructed based on the portion of the predicted sample values ​​that falls within the limiting range.

[0007] According to another aspect of the present invention, a video decoding method is provided. In this method, a video stream is received, the video stream including encoded information of a current image and a reference image of the current image. It is determined whether to apply directional padding to a filling region. The filling region is located outside and adjacent to the boundary of the current image. A filling direction for the directional padding is determined based on the direction of an intra-frame prediction mode, the intra-frame prediction mode being derived from one of a boundary block in the current image and a motion compensation block in the filling region. The boundary block is located inside and adjacent to the boundary of the current image. A portion of the filling region is filled based on multiple samples along the filling direction in the current image.

[0008] According to another aspect of the present invention, a video decoding method is provided. In this method, a video stream is received, the video stream including encoded information of a current block in a current image. The encoded information indicates a prediction mode of the current block, the prediction mode being associated with a list of most probable modes (MPMs) generated based on a first type of prediction information and a list of most dominant modes (MDMs) generated based on a second type of prediction information. The list of MDMs, including multiple candidate MDMs, is determined based on a reference region of the current block. The current block is reconstructed based on at least one of the following: (i) one candidate MDM from the multiple candidate MDMs in the list of MDMs, and (ii) one candidate MPM from the list of MPMs.

[0009] Aspects of the present invention include a video coding method. In this method, a reference value is determined based on neighboring samples in a template region of a current block in a current image. The current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients of that corresponding filter. A candidate model for predicting the current block is determined based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. The current block is encoded based on the portion of the predicted sample values ​​that falls within the limiting range.

[0010] Aspects of the present invention include a video coding method. In this method, it is determined whether to apply directional padding to a padding region. The padding region is located outside and adjacent to the boundary of a current image. A padding direction for the directional padding is determined based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block within and adjacent to the boundary of the current image and a motion compensation block in the padding region. A portion of the padding region is encoded by padding it based on multiple samples in the current image along the padding direction.

[0011] Aspects of the present invention include a video coding method. In this method, a list of MPMs for a current block in a current image is determined. A list of MDMs for the current block is determined. The list of MDMs includes a plurality of candidate MDMs based on a reference region of the current block. The current block is encoded into a bitstream based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0012] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media bitstream encoded by an encoding method. In this encoding method, a reference value is determined based on neighboring samples in a template region of a current block in a current image. The current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients for that corresponding filter. A candidate model for predicting the current block is determined based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. The current block is encoded based on the portion of the predicted sample values ​​that falls within the limiting range.

[0013] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media stream encoded by an encoding method. In this encoding method, it is determined whether directional padding is applied to a padding region. The padding region is located outside and adjacent to the boundary of a current image. A padding direction for the directional padding is determined based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block within and adjacent to the boundary of the current image and a motion compensation block in the padding region. A portion of the padding region is encoded by padding a portion of the padding region based on multiple samples in the current image along the padding direction.

[0014] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media bitstream encoded by an encoding method. In this encoding method, a list of MPMs for a current block in a current image is determined. A list of MDMs for the current block is determined. The list of MDMs includes a plurality of candidate MDMs based on a reference region of the current block. The current block is encoded into the bitstream based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0015] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current block in a current image. The encoded information indicates that the current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients for that corresponding filter. Format rules specify that a reference value is determined based on neighboring samples in a template region of the current block. Format rules specify that a candidate model from the plurality of candidate models used to predict the current block is determined based on a comparison between the reference value and a threshold. Format rules specify that a limiting range of predicted sample values ​​for the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. Format rules specify that the current block is processed based on the portion of the predicted sample values ​​that falls within the limiting range.

[0016] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current image and a reference image of the current image. The format rules specify whether to apply directional padding to a padding region located outside and adjacent to a boundary of the current image. The format rules specify determining a padding direction for directional padding based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block and a motion compensation block within and adjacent to the boundary of the current image. The format rules specify padding a portion of the padding region based on multiple samples along the padding direction in the current image.

[0017] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current block in a current image. The encoded information indicates a prediction mode for the current block, which is associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information. The format rules specify that the list of MDMs, including a plurality of candidate MDMs, is determined based on a reference region of the current block. The format rules specify that the current block is processed based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0018] The present invention also provides an apparatus for video decoding. The apparatus for video decoding includes processing circuitry configured to implement any of the described methods for video decoding.

[0019] Aspects of the present invention also provide an apparatus for video encoding. The apparatus for video encoding includes processing circuitry configured to implement any of the described methods for video encoding.

[0020] Aspects of the present invention also provide a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform any of the described methods.

[0021] Aspects of the present invention also provide a non-transitory computer-readable storage medium storing a video media stream that, when processed by at least one processor, causes the at least one processor to perform any of the described methods for video decoding.

[0022] The technical solution of the present invention includes a method and apparatus for improving intra-frame predictive coding. The method includes: (i) intra-frame prediction based on a multi-model extrapolation filter, (ii) intra-frame prediction using a multi-hypothesis model, (iii) directional padding for image boundary padding, (iv) reference shifting for intra-frame prediction, (v) intra-frame mode derivation using signaling indication syntax and MPM adaptation, and (vi) directional plane mode based on histogram of gradients (HoG).

[0023] In one example, a video stream is received, which includes encoded information for the current block in the current image. The encoded information indicates that the current block is encoded based on intra-frame prediction using an extrapolation filter, according to one of a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes multiple filter coefficients for that corresponding filter. Reference values ​​are determined based on neighboring samples in the template region of the current block. One of the plurality of candidate models is determined for predicting the current block, based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Based on the one of the plurality of candidate models, the predicted sample values ​​of the current block are obtained. The current block is reconstructed based on the portion of the predicted sample values ​​that falls within the limiting range.

[0024] In one example, a video stream is received, which includes encoded information of the current image and a reference image of the current image. It is determined whether directional padding should be applied to a fill region. The fill region is located outside and adjacent to the boundary of the current image. The fill direction of the directional padding is determined based on the direction of an intra-prediction mode derived from one of the boundary blocks in the current image and the motion-compensated blocks in the fill region. The boundary blocks are located inside and adjacent to the boundary of the current image. A portion of the fill region is filled based on multiple samples along the fill direction in the current image.

[0025] In one example, a video stream is received, which includes encoded information of the current block in the current image. The encoded information indicates the prediction mode of the current block, which is associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information. The list of MDMs, which includes multiple candidate MDMs, is determined based on a reference region of the current block. The current block is reconstructed based on at least one of the following: (i) one candidate MDM from the list of multiple candidate MDMs, and (ii) one candidate MPM from the list of MPMs.

[0026] Therefore, the coding efficiency and accuracy of intra-frame prediction are improved based on at least one of the following methods: (i) intra-frame prediction based on multi-model extrapolation filters, (ii) intra-frame prediction using multiple hypothesis models, (iii) directional padding for image boundary padding, (iv) reference shifting for intra-frame prediction, (v) intra-frame mode derivation using signaling indication syntax and MPM adaptation, and (vi) HoG-based directional plane mode. Attached Figure Description

[0027] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which: Figure 1An example block diagram of a communication system (100) is shown schematically.

[0028] Figure 2 An example block diagram of a decoder is shown schematically.

[0029] Figure 3 An example block diagram of an encoder is shown schematically.

[0030] Figure 4 Various examples of extrapolation filter shapes according to some aspects of the invention are shown.

[0031] Figure 5 Various examples of template types according to some aspects of the present invention are shown.

[0032] Figure 6 This shows an example of predictions generated diagonally for different positions within the current block.

[0033] Figure 7 An example of threshold calculation based on the average value in the reconstructed region is shown.

[0034] Figure 8 Various examples of reference value calculations are shown.

[0035] Figure 9 An example of deriving the amplitude limit range in a reconstructed area is shown.

[0036] Figure 10 A flowchart outlining the decoding process of multi-hypothesis intra-frame prediction, indicated by flag bits, is shown.

[0037] Figure 11 A flowchart outlining the decoding process of multi-hypothesis intra-frame prediction determined by the multi-hypothesis data size is shown.

[0038] Figure 12 A flowchart outlining the decoding process of multi-hypothesis intra-prediction, including implicit intra-prediction mode assignment, is shown.

[0039] Figure 13 Examples of repeating padding and motion-compensated (MC) boundary padding are shown.

[0040] Figure 14 An example of filling block derivation is shown.

[0041] Figure 15 Examples of repeated filling according to some aspects of the present invention are shown.

[0042] Figure 16 An example of directional filling according to some aspects of the present invention is shown.

[0043] Figure 17 An example of vertical intra-frame prediction according to some aspects of the present invention is shown.

[0044] Figure 18 An example of reference sample shifting according to some aspects of the present invention is shown.

[0045] Figure 19 Various examples of reference sample shifting according to some aspects of the invention are shown.

[0046] Figure 20 An example of intra-frame mode derivation based on decoder gradients is shown.

[0047] Figure 21 An example is shown that uses reconstructed samples in a template to derive inter-frame prediction patterns.

[0048] Figure 22 An example of constructing termination conditions based on a threshold-based histogram is shown.

[0049] Figure 23 A reference sample example is shown, taking into account the top template.

[0050] Figure 24 A reference sample example is shown when the left-hand template is considered.

[0051] Figure 25 An example of histogram construction based on a block vector (BV) guided template is shown.

[0052] Figure 26 A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0053] Figure 27 A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0054] Figure 28 A flowchart outlining the decoding process according to some aspects of the present invention is shown.

[0055] Figure 29 It is a schematic diagram of a computer system based on one aspect. Detailed Implementation

[0056] Figure 1 Block diagrams of some example video processing systems (100) are shown. The video processing system (100) is an example application of the disclosed subject matter, namely, video encoders and video decoders in a streaming media environment. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming media services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0057] The video processing system (100) includes an acquisition subsystem (113), which may include a video source (101), such as a digital camera, for creating, for example, an uncompressed video image stream (102). In one example, the video image stream (102) includes samples captured by a digital camera. The video image stream (102) is depicted as a thick line compared to the encoded video data (104) (or encoded video stream) to emphasize the high data volume of the video image stream, which may be processed by an electronic device (120) including a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to enable or implement aspects of the disclosed subject matter described in more detail below. The encoded video data (104) (or encoded video stream) is depicted as a thin line compared to the video image stream (102) to emphasize the low data volume of the encoded video data, which may be stored on a streaming media server (105) for future use. At least one streaming media client subsystem, such as Figure 1 Client subsystems (106) and (108) can access a streaming media server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and creates an outgoing video image stream (111) that can be presented on a display (112) (e.g., a screen) or other display device (not depicted). In some streaming media systems, the encoded video data (104), (107), and (109) (e.g., a video stream) may be encoded according to certain video coding / compression standards. For example, those standards include the ITU-T H.265 Recommendation. In one example, the video coding standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics are applicable in the context of VVC.

[0058] It should be noted that electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may also include a video decoder (not shown), while electronic device (130) may also include a video encoder (not shown).

[0059] Figure 2 An example block diagram of a video decoder (210) is shown. The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may replace... Figure 1The video decoder (110) in the example is used.

[0060] The receiver (231) can receive, for example, at least one encoded video sequence included in the bitstream, which is decoded by the video decoder (210). In one aspect, one encoded video sequence is received at a time, and the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences can be received from a channel (201), which can be a hardware / software link connected to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data and other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). The receiver (231) can separate the encoded video sequences from other data. To combat / mitigate network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) may be external to the video decoder (210) (not depicted). In other applications, a buffer memory (not depicted) may exist external to the video decoder (210), for example, to combat / mitigate network jitter, and another buffer memory (215) may exist internally to the video decoder (210), for example, to handle playback timing. The buffer memory (215) may not be necessary or may be made smaller when the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network. For use on best-effort packet networks such as the Internet, a buffer memory (215) may be required. The buffer memory (215) may be relatively large and may advantageously have an adaptive size, and may be implemented at least partially in a similar component (not depicted) external to the operating system or the video decoder (210).

[0061] The video decoder (210) may include a parser (220) for reconstructing symbols (221) from the encoded video sequence. Categories of those symbols include information for managing the operation of the video decoder (210) and may include information for controlling a display device, such as a component of a non-electronic device (230) but which may be coupled to an electronic device (230) (e.g., a display screen). Figure 2As shown in the diagram. Control information for one or more display devices may be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (220) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup / subcluster in the video decoder based on at least one parameter corresponding to a group. Subgroups may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (220) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0062] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) in order to create symbols (221).

[0063] Depending on the type of the encoded video image or its portions (e.g., inter-frame and intra-frame images, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (220). For clarity, the flow of such subgroup control information between the parser (220) and the multiple units described below is not depicted.

[0064] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In practical implementations under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0065] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives from the parser (220) quantization transform coefficients as one or more symbols (221) and control information, including the transform to be used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output blocks containing sample values, which can be input into the aggregator (255).

[0066] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks do not use predictive information from previously reconstructed images, but instead can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-image prediction unit (252). In some cases, the intra-image prediction unit (252) uses surrounding reconstructed information obtained from the current image buffer (258) to generate blocks of the same size and shape as the blocks in the reconstruction. For example, the current image buffer (258) is used to buffer partially reconstructed current images and / or fully reconstructed current images. In some cases, the aggregator (255) adds the predictive information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0067] In other cases, the output samples of the scaler / inverse transform unit (251) may involve inter-frame coded and possibly motion-compensated blocks. In this case, the motion compensation prediction unit (253) can access the reference image memory (257) to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols (221), these samples can be added to the output of the scaler / inverse transform unit (251) (referred to in this case as residual samples or residual signals) by the aggregator (255) to generate output sample information. The address of the predicted samples obtained by the motion compensation prediction unit (253) in the reference image memory (257) can be controlled by motion vectors used by the motion compensation prediction unit (253) in the form of symbols (221), which may have, for example, X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0068] The output samples of the aggregator (255) can be employed by various loop filtering techniques in the loop filter unit (256). Video compression techniques may include loop filtering techniques controlled by parameters included in the encoded video sequence (also known as the encoded video stream) and available as symbols (221) from the parser (220) for use in the loop filter unit (256). Video compression may also respond to metadata obtained during the decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0069] The output of the loop filter unit (256) can be a sample stream, which can be output to the display device (212) and stored in the reference image memory (257) for future inter-frame image prediction.

[0070] Some encoded images can be used as reference images for future predictions after they have been fully reconstructed. For example, after the encoded image corresponding to the current image has been fully reconstructed and the encoded image has been identified as a reference image (by, for example, a parser (220)), the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0071] The video decoder (210) can perform decoding operations according to a predetermined video compression technology or standard, such as ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax of the video compression technology or standard and the configuration file documented therein if the encoded video sequence follows both the syntax of the video compression technology or standard and the configuration file documented therein. Specifically, the configuration file may select certain tools from all available tools in the video compression technology or standard as the only tools available under that configuration file. For compliance, the complexity of the encoded video sequence must also be within the limits defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer indicated by signaling in the encoded video sequence.

[0072] In one aspect, the receiver (231) may receive supplemental (redundant) data along with the encoded video. This supplemental data may be part of one or more encoded video sequences. The supplemental data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0073] Figure 3 An example block diagram of a video encoder (303) is shown. The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., transmission circuitry). The video encoder (303) may replace Figure 1 The video encoder (103) in the example is used.

[0074] The video encoder (303) can obtain data from the video source (301) (non-video source). Figure 3 In one example, an electronic device (320) receives video samples, and a video source (301) can capture one or more video images to be encoded by a video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0075] A video source (301) can provide a sequence of source video samples in the form of a digital video sample stream to be encoded by a video encoder (303). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual images that, when viewed sequentially, produce an animated effect. The images themselves can be organized as spatial pixel arrays, where each pixel can include at least one sample, depending on the sampling structure, color space, etc., used. Samples are described in detail below.

[0076] According to one aspect, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or as needed, according to any other time constraints. Implementing an appropriate encoding rate is a function of the controller (350). In some aspects, the controller (350) controls and is functionally coupled to other functional units described below. For clarity, the coupling is not depicted. Parameters set by the controller (350) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, GOP layout, maximum motion vector search range, etc. The controller (350) can be configured to have other suitable functions related to the video encoder (303) optimized for a specific system design.

[0077] In some respects, the video encoder (303) is configured to operate in an encoding loop. Simply put, in one example, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and one or more reference images), and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since the decoding of the symbol stream produces a bit-by-bit accurate result unaffected by the decoder's location (local or remote), the contents of the reference image memory (334) are also bit-by-bit accurate between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same as the sample values ​​"seen" by the decoder during decoding when using the prediction. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related fields.

[0078] The operation of the “local” decoder (333) can be the same as that of a “remote” decoder such as a video decoder (210), which has already been described above. Figure 2 A detailed description has been provided. However, a brief additional reference is also available. Figure 2 When symbols are available and the entropy encoder (345) and parser (220) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (210), including the buffer (215) and parser (220), may not be fully implemented in the local decoder (333).

[0079] In one respect, decoder techniques other than parsing / entropy decoding, which exist in the decoder, exist in the corresponding encoder with the same or substantially the same functional form. Therefore, the disclosed subject focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are the reverse operation of the comprehensively described decoder techniques. More detailed descriptions are only required in certain areas, and these descriptions are provided below.

[0080] In some examples, during operation, the source encoder (330) may perform motion-compensated predictive coding, i.e., predictively coding the input image against at least one previously encoded image designated as a “reference image” in the reference video sequence. In this way, the coding engine (332) encodes the differences between pixel blocks of the input image and pixel blocks of one or more reference images, which may be selected as one or more predictive references for the input image.

[0081] The local video decoder (333) can decode encoded video data of an image that can be designated as a reference image, based on symbols created by the source encoder (330). Advantageously, the operation of the encoding engine (332) can be a lossy process. When the encoded video data can be decoded by the video decoder (333), Figure 3 During decoding at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process, which can be performed by the video decoder on the reference image, and can store the reconstructed reference image in the reference image memory (334). In this way, the video encoder (303) can store a copy of the reconstructed reference image locally, which shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0082] The predictor (335) can perform a prediction search against the encoding engine (332). That is, for a new image to be encoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as suitable prediction references for the new image. The predictor (335) can operate on a pixel-by-pixel basis based on sample blocks to find suitable prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input image may have prediction references obtained from multiple reference images stored in the reference image memory (334).

[0083] The controller (350) can manage the encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for video data encoding.

[0084] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder (345). The entropy encoder (345) converts these symbols into an encoded video sequence by applying lossless compression to the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding.

[0085] The transmitter (340) may buffer one or more encoded video sequences created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which may be a hardware / software link connected to a storage device that will store the encoded video data. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0086] The controller (350) can manage the operation of the video encoder (303). During encoding, the controller (350) can assign a specific encoded image type to each encoded image, which may affect the encoding techniques applicable to the corresponding image. For example, one of the following image types can typically be assigned to an image.

[0087] Intraframe images (I-images) are images that are encoded and decoded without using any other images in the sequence as prediction sources. Some video codecs allow different types of intraframe images, including, for example, Independent Decoder Refresh (IDR) images.

[0088] Predictive images (P-images) can be images that are encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using motion vectors and reference indices.

[0089] A bidirectional predictive image (B-image) can be an image encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using two motion vectors and a reference index. Similarly, a multi-predictive image can reconstruct a single block using more than two reference images and associated metadata.

[0090] The source image can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignment applied to the corresponding images of the blocks. For example, blocks of an I-image can be non-predictively coded, or these blocks can be predictively coded with reference to already coded blocks of the same image (spatial prediction or intra-frame prediction). Pixel blocks of a P-image can be predictively coded via spatial prediction or via temporal prediction with reference to a previously coded reference image. Blocks of a B-image can be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference images.

[0091] The video encoder (303) can perform encoding operations according to predetermined video coding techniques or standards, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0092] In one respect, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may transmit such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0093] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, the specific image being encoded / decoded is called the current image, which is divided into blocks. Where a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image and, in the case of multiple reference images, may have a third dimension identifying the reference images.

[0094] In some aspects, inter-frame image prediction can utilize bidirectional prediction techniques. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly representing past and future images in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. A block can be predicted using a combination of the first and second reference blocks.

[0095] Furthermore, merging mode techniques can be used to improve coding efficiency in inter-frame image prediction.

[0096] According to some aspects of the invention, prediction is performed on a block-by-block basis, such as inter-frame image prediction and intra-frame image prediction. For example, according to the HEVC standard, images in a video image sequence are divided into coding tree units (CTUs) for compression. A single CTU in an image has the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU comprises three coding tree blocks (CTBs): one luma coding tree block and two chroma coding tree blocks. Each CTU can be recursively quadtree-based into one or more coding units. For example, a 64×64 pixel CTU can be split into one 64×64 pixel coding unit, or four 32×32 pixel coding units, or sixteen 16×16 pixel coding units. In one example, each coding unit is analyzed to determine its prediction type, such as inter-frame prediction or intra-frame prediction. Coding units are split into at least one prediction unit based on temporal and / or spatial predictability. Typically, each prediction unit comprises one luma prediction block (PB) and two chroma prediction blocks. In one aspect, prediction operations in encoding / decoding are performed on a block-by-block basis. Using the luma prediction block as an example, a prediction block comprises a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0097] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one integrated circuit (IC). In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using at least one processor that executes software instructions.

[0098] Various aspects of the present invention include methods and systems for band-limited intra-frame prediction based on multi-model extrapolation filters.

[0099] Video codecs are widely used in many applications. Various video codec standards, such as H.264, H.265, H.266 (VVC), AV1, and AVS, have been widely adopted.

[0100] Video codecs may include several modules, such as intra-frame prediction, inter-frame prediction, transform coding, quantization, entropy coding, and loop filtering. Aspects of the invention include a set of methods for video compression, such as methods related to template matching prediction.

[0101] The current coding block and its neighboring samples can share similar texture features. Based on this feature, neighboring reconstructed samples, known as templates, can be used to predict the current coding block. In one example, intra-frame prediction can be applied to predict the current coding block. Intra-frame prediction can include angular and non-angular intra-frame prediction modes that utilize directional or non-directional texture correlations between neighboring samples.

[0102] Intra-frame prediction based on extrapolation filters can leverage the correlation between neighboring samples by learning (or building) a model from them using filters such as 15-tap filters. In one example, the model includes a set of filter coefficients. The 15-tap filter can have different shapes. Figure 4 The shapes of three 15-tap filters are shown as (402), (404) and (406).

[0103] For a specific filter shape, for example, three different reference regions (also called templates) of the reconstructed sample can be selected to learn the model. Figure 5 Three example regions (or templates) (502), (504), and (506) are given. The template can be iterated (or traversed) using a square filter shape (508) to learn (or build) the model.

[0104] according to Figure 4 and 5The example shown allows for nine combinations of filter shapes (e.g., (402), (404), and (406)) and template types (e.g., (502), (504), and (506)). Given a specific filter shape and template type, the coefficients of the filter can be obtained by solving a linear function based on reconstructed samples in the template region. A flag bit can be signaled in the bitstream to indicate the activation (or enable) of intra-frame prediction based on the extrapolation filter. When this flag bit is enabled, related syntax elements that indicate the filter shape and template type are also signaled. The selected filter (indicated by the filter shape) can be iterated (or traversed by a sliding window) within the selected template region (indicated by the template type) with a single-pixel step size to construct the autocorrelation matrix and cross-correlation vector. The coefficients of the filter are calculated based on the autocorrelation matrix and cross-correlation vector. To generate the prediction signal for the current coding block, the filter is applied by constructing a predefined linear combination of the learned coefficients and reference samples.

[0105] In one example, such as Figure 6 As shown, when the learned model is applied, samples within the current block (602) can be generated sequentially from the top left to the bottom right position according to the prediction order, such as a diagonal prediction order. When the extrapolation filter (604) is applied to the boundary of the current block (602), reconstructed samples from the template (606) (e.g., (608)) and predicted samples from the current block (602) (e.g., (610)) within the same filter shape can be used as inputs to the filter (604).

[0106] Based on the determined filter and template types, a single model for intra-frame prediction based on extrapolation filters may generate suboptimal predictions because the training data in the template may be diverse, leading to inaccuracies in the trained model. Furthermore, the predicted signal may include some outliers, resulting in additional distortion.

[0107] Aspects of this invention include applying multiple models in intra-frame prediction based on extrapolation filters for a given filter type and template region (or template shape or template type). A threshold can be used as a classifier to determine whether a reference sample is suitable for one (or more) of the multiple models. When applying the learned multiple models, a limiting operation can be applied to the predicted samples using the minimum and maximum values ​​of each model.

[0108] In one aspect, two models are learned based on a given filter type and template region. In one example, these two models are represented in equations (1) and (2) as follows.

[0109] , If reference ≤ threshold Formula (1) , If reference > threshold Formula (2) Where C 1_i (i∈[0,14]) represents the coefficients learned from the first model, and C 2_i (i∈[0,14]) represents the coefficients learned from the second model. It is either a reconstructed value (e.g., (608) in the template) or a predicted value (e.g., (610) in the current block) used to predict the current location. By comparing the reference value with the threshold, it can be determined which model to train and apply.

[0110] In one aspect, the threshold can be set as the average of the samples in the reconstructed region. In one example, the average is the average of all samples in the reconstructed region, for example... Figure 7 The L-shaped reconstructed area (702) is shown.

[0111] In one example, the reference value is determined based on the filter shape. For example, as... Figure 8 As shown, a single sample, such as (802) or (804), can be identified in filter shape (806) or (808) and used as input for calculating a reference value. In one example, the single sample can be a sample in a reconstructed region (e.g., (702)). In one aspect, for certain filter shapes, such as filter shape (810), more than one sample, such as two samples (812) and (814), can be used as input for calculating a reference value, and the calculation can involve a fixed weighting or an adaptive weighting of the multiple sample values.

[0112] In one aspect, for the predicted sample limit, two ranges are derived for the two models proposed in formulas (1) and (2), respectively (e.g., Min1 , Max1 ) and (for example, Min2 , Max2 ).

[0113] In one example, when the reference value <= the threshold, a limiting range is applied to the first model shown in formula (1) (e.g., Min1=0, Max1=threshold+δ).

[0114] In one example, when the reference value <= the threshold, a limiting range (e.g., Min1=S) is applied to the first model. min Max1 = threshold + δ). Value S min Signaling can be used to indicate this from the bitstream.

[0115] In one example, when the reference value > the threshold, a limiting range is applied to the second model shown in formula (2) (e.g., Min2 = threshold - δ). ).

[0116] In one example, when the reference value > the threshold, a limiting range is applied to the second model (e.g., Min2 = threshold - δ, Max2 = S). max Value S max Signaling can be used to indicate this from the bitstream.

[0117] In one example, δ is a positive value and is used to control the amplitude limiting strength. A larger δ indicates a greater tolerance for outlier predicted samples. In one example, δ can be set independently for different models. δ can be a derived value or a predefined value.

[0118] In one example, the minimum and maximum values ​​used for amplitude limiting are derived from reference samples calculated during model derivation using tags (or annotations). Figure 9 An example of a labeled reference sample is shown.

[0119] like Figure 9 As shown, multiple reference values ​​can be determined when the filter (902) iterates (or slides through) the template region (904). An example of reference value calculation is shown below. Figure 8 As shown. Based on reference values ​​and their positions in the template region (904), a binary image (906) is constructed for different models, such as the two models shown in formulas (1) and (2). In one example of the binary image (906), positions with triangle markers are identified using the first model (e.g., the model shown in formula (1)), and positions without triangle markers are identified using the second model (e.g., the model shown in formula (2)). The binary image (906) may have the following dimensions: a width of (aboveSize–3), which is the size of the top row of the template region (904) (e.g., aboveSize) minus a constant (e.g., 3); and a height of (leftSize–3), which is the size of the left column of the template region (904) (e.g., leftSize) minus the constant.

[0120] Based on the binary graph (906), the reference value for each location can be statistically determined (or derived based on the filter). The minimum and maximum values ​​for each group (or model) are derived based on the reference values ​​associated with the corresponding group (or model).

[0121] Various aspects of the present invention include methods and systems for intra-frame prediction using a multi-hypothesis model.

[0122] Video codecs are widely used in various applications such as broadcasting, video recording, and video streaming. Emerging video codec standards such as H.264, H.265 / HEVC, H.266 / VVC, and AV1 have been released and are widely used in video applications. Hybrid video codecs can include multiple codec modules, such as intra-frame prediction, inter-frame prediction, transform coding, quantization, entropy coding, and post-loop filters.

[0123] Similar to conventional bidirectional prediction, a multi-hypothesis inter-frame prediction mode has been proposed to support at least one additional motion-compensated prediction sample. The overall prediction signal obtained from the multi-hypothesis inter-frame prediction mode can be obtained by weighting samples one by one. For example, based on the first additional inter-frame prediction signal (or hypothesis). h 3 and bidirectional prediction signals p bi The predicted signal can be obtained using the following formula (3). p 3: Formula (3) As shown in formula (3), the weighting factor can be derived from the lookup table using the index indicated by signaling in the code stream. α Similar to equation (3), more than one additional prediction signal can be used. The resulting overall prediction signal can be shown in equation (4), where each additional prediction signal is iteratively accumulated: Formula (4) As shown in Equation (4), the resulting overall prediction signal is determined to have the largest index. n The last predicted signal p n .

[0124] In one aspect, the multi-hypothesis intra-prediction signal can support at least one additional intra-prediction sample to obtain the final intra-prediction signal. In another aspect, for example, after the signaling-indicated intra-prediction mode, at least one syntax element can be signaled to indicate whether a multi-hypothesis intra-prediction mode is applied in addition to the signaling-indicated intra-prediction mode. This at least one syntax element may be or may include a flag bit. If the flag bit is, for example, true, the multi-hypothesis intra-prediction can be parsed or derived, and the multi-hypothesis model can be applied. Otherwise, if the flag bit is false, only the signaling-indicated intra-prediction mode is used to generate intra-prediction samples.

[0125] In one respect, the obtained intra-frame predicted samples p It is an intra-frame prediction mode indicated by signaling. p s and additional intra-frame prediction p hThe derived formula is shown in formula (5) below, where the data size is assumed to be 2.

[0126] Formula (5) In one aspect, the obtained intra-predicted samples are derived by iteratively accumulating each additional intra-predicted signal. Equation (6) shows an example when the multiple hypothesis data size is 3: , in Formula (6) As shown in formula (6), the obtained intra-frame predicted samples p It is an intra-frame prediction mode indicated by signaling. p s And two additional intra-frame predictions p h0 and p h1 This is derived. The overall prediction signal is obtained by iteratively accumulating the sums (or combinations) of each additional prediction signal and its corresponding weighting factor. In one example, the predicted sample... p h0 and p h1 It is generated based on additional intra-prediction modes, such as decoder-side intra-mode derivation (DIMD) or MPM. Due to the recursive nature of the multi-hypothesis model, the size of the multi-hypothesis model is... n The obtained intra-frame predicted samples can be derived from the general formula in formula (7) below.

[0127] , in Formula (7) in i Increase from 0 to n -2.

[0128] In one example, the decoding process for a multi-hypothesis intra-prediction mode is as follows: Figure 10-11 As shown. Figure 10 A flowchart (1000) outlining the decoding process of multi-hypothesis intra-frame prediction indicated by flag bits is shown. Figure 11 A flowchart (1100) outlining the decoding process for multi-hypothesis intra-frame prediction determined by the multi-hypothesis data size is shown. Figure 10 and 11 As shown, the weight factor (e.g., MH weight) for each additional intra-prediction mode is derived based on a lookup table and represented by the index indicated by the signaling. For each additional intra-prediction indicated by the signaling in the bitstream, the resulting intra-prediction samples are iteratively accumulated.

[0129] In one aspect, intra-frame prediction modes for multi-hypothesis prediction (e.g., p h0 and p h1 () is derived from adjacent encoded blocks (e.g., based on DIMD or MPM).

[0130] In one aspect, intra-prediction mode candidates are constructed from the intra-prediction modes of adjacent coded blocks.

[0131] In one respect, the weighting factor for each additional predicted sample is a predefined fixed weighting value, or a value indicated / derived by signaling. For example, the derived value may be indicated by an index, which may be indicated by signaling in the bitstream.

[0132] In one aspect, flags indicating the multi-hypothesis intra-prediction mode and the multi-hypothesis data size are signaled in the bitstream at each coding block level.

[0133] In one aspect, when the intra-prediction mode indicated by signaling satisfies at least one condition, a flag bit for a multiple hypothetical intra-prediction mode is indicated by signaling. For example, the intra-prediction mode includes an unconventional intra-prediction mode, a non-directional intra-prediction mode, a non-planar mode, and / or a non-DC mode. Otherwise, the flag bit for the multiple hypothetical intra-prediction mode may not be indicated by signaling, and the flag bit may be implicitly assigned to false.

[0134] In one respect, when selecting a multi-hypothesis intra-prediction mode from among multiple candidates, a signaling index can be used to indicate which candidate is used for multi-hypothesis intra-prediction.

[0135] In one aspect, the additional intra-prediction mode in multi-hypothesis prediction is explicitly indicated by signaling in the bitstream to indicate which intra-prediction mode has been selected.

[0136] In one aspect, the additional intra prediction modes in the multiple hypotheses are restricted to one of the predefined (or permitted) intra prediction modes. Permitted intra prediction modes can be based on block width and / or block height and / or block size information. An index is signaled in the bitstream indicating which permitted intra prediction mode from the predefined set has been selected.

[0137] In one aspect, the additional intra-prediction mode in the multiple hypotheses is implicitly assigned as the default intra-prediction mode. This default intra-prediction mode may include, but is not limited to, planar mode, DC mode, etc.

[0138] In one respect, the last additional intra-prediction mode is implicitly assigned as the default mode. When the multi-hypothesis data size is 2, the second additional intra-prediction mode is implicitly assigned as the default mode without any signaling. Figure 12 An example decoding flowchart (1200) for a multi-hypothesis intra-prediction mode is shown, where the last additional intra-prediction mode in the multi-hypothesis intra-prediction mode is implicitly assigned. Figure 12 As shown, predefined weights (e.g., MH weights) are also used as weighting factors for implicit default mode assignment (or the default mode of implicit assignment).

[0139] In one aspect, a default intra-prediction mode list is constructed to determine additional intra-prediction among the multi-hypothesis intra-prediction modes. The default intra-prediction mode list can be constructed from a predefined list. The first available intra-prediction mode in the predefined list can be added to the default intra-prediction mode list.

[0140] In one respect, the last additional intra-prediction mode is implicitly assigned as the default mode. This default mode is derived from a list of default modes, which is constructed in the same manner as the aforementioned list of default intra-prediction modes.

[0141] In one respect, the additional intra-prediction mode in the multiple hypothesis intra-prediction mode is a non-directional intra-prediction mode.

[0142] Aspects of the present invention include methods and systems for directional filling methods for image boundary filling.

[0143] Video codecs are widely used in various applications. Video codec standards, such as H.264, H.265, H.266 (VVC), AV1, and AVS, have been widely adopted.

[0144] Video codecs can include various modules, such as intra / inter-frame prediction, transform coding, quantization, entropy coding, and loop filtering. As one of the main modules, several intra-frame prediction techniques have been proposed. These techniques include both traditional signaling processing methods and recent neural network-based approaches.

[0145] Figure 13 The diagram illustrates repeating (or mirroring) padding (1300A) and motion-compensated boundary padding (1300B). As shown in repeating padding (1300A), repeating padding pixels from the image boundary can be used as reference pixels when the decoder performs motion compensation and the motion vector points to a block outside the frame boundary. As shown in motion-compensated boundary padding (1300B), motion-compensated boundary pixel padding (motion-compensated padding) can include a motion-compensated padding region (1320).

[0146] Figure 14An example of motion-compensated fill is shown, where an M×4 fill block (1402) is derived based on the left fill direction. For example... Figure 14 As shown, the MCP block (1410) can be identified as a reference block (not shown) of the current block (not shown) in the current image (not shown) by a motion vector (not shown). The MCP block (1410) can be located along the boundary (1406) of the reference image (1403). Figure 14 In this context, the reference image (1403) can also be referred to as the current image of the reference image (1401). The size of the MCP block (1410) can be 4×M or M×4, where M is the desired frame boundary extension. The motion vector (1412) is derived from the 4×4 block (1404) at the image boundary (1406). If the 4×4 block (1404) at the image boundary is intra-coded, a zero motion vector is used. If the 4×4 block at the image boundary is bidirectional inter-frame predictive coding, only the motion vector pointing to pixels farther from the frame boundary is used in the motion compensation for padding.

[0147] exist Figure 14 In one example, the motion vector (1412) points to a reference block (1408) in the reference image (1401). In another example, both the reference image (1403) and the reference image (1401) are reference images of the current image (not shown). In yet another example, the reference image (1401) is reconstructed prior to the reference image (1403). The MCP block (1410) can be filled (or padded) based on the MCP block (1402) located within the reference image (1401) according to motion-compensated boundary pixel padding. The MCP block (1402) is located within the reference image (1401) and is adjacent to the reference block (1408). In one aspect, the pixels in the motion-compensated padding block (1410) can be corrected using an offset equal to the difference between the DC value of the reconstructed boundary 4×4 block (1404) and the DC value of the corresponding 4×4 reference block (1408). Furthermore, when the motion vector (1412) points to a position within the boundary of the reference image (1401) (e.g., (1402)), M is set to at least 4. If M is less than 64, the remainder of the filled area can be filled with repeated fill samples.

[0148] In one example, regardless of whether motion-compensated fill is applied, when Figure 14 If the value of M is less than a preset value, such as less than 64, repeated padding will be applied. Figure 15 As shown, the repeated filling method can use the nearest sample, such as the sample closest to the filling region (1504) in the reference image (1502), to perform filling in the horizontal or vertical direction. Therefore, repeated filling may not be able to capture directional features from the actual content (1506).

[0149] Various aspects of the present invention relate to directional filling for acquiring directional features from actual content.

[0150] In one aspect, when the available motion compensation fill block (e.g., (1410)) is smaller than the maximum motion compensation fill width or height, directional fill is applied instead of repeated fill. The fill direction can be derived based on the texture of the motion compensation block (e.g., (1410)) initially used for motion compensation fill. The fill area is then filled with samples along the derived direction. Figure 16 An example of the proposed directional filling is shown. For example... Figure 16 As shown, the filling region (1602) is filled with samples (1604) within the image (e.g., reference image (1403)) along the derived direction.

[0151] In one aspect, when blocks at image boundaries (e.g., (1406)) such as 4×4 blocks (e.g., (1404)) are encoded as intra-frame blocks, the disclosed orientation padding is applied instead of motion-compensated padding.

[0152] In one aspect, the proposed directional filling is applied when the motion vector of the current block points outside the boundary of the reference image (e.g., (1406)). When the motion vector of the current block points outside the boundary of the reference image, this indicates that the current block is arranged across the boundary of the reconstructed region (e.g., the reconstructed region of the current image).

[0153] In one respect, the direction (or padding direction) is one of the intra-prediction modes. For example, the padding direction is the direction indicated by one of the intra-prediction modes.

[0154] In one respect, the direction (or fill direction) is derived from the motion compensation block (e.g., (1410)) originally used for motion compensation filling.

[0155] In one respect, this direction is derived on motion-compensated blocks using a gradient-based intra-frame mode derivation method.

[0156] In one example, gradient-based intra-frame mode derivation is applied to the motion compensation block (e.g., (1410)). An orientation is selected based on the HoG. For example, this orientation is indicated by an intra-frame prediction mode derived from the HoG. Orientation padding is performed using the selected orientation.

[0157] In one aspect, the filling region (e.g., (1602)) is filled with the nearest sample (e.g., (1604)) in the derived direction.

[0158] In one example, the filled region is filled (or padded) using regular intra-frame prediction (e.g., angular intra-frame prediction mode). For example, based on regular intra-frame prediction, the filled region is filled using regular interpolated reference samples.

[0159] In one example, when a region is filled (or padded) with regular intra-frame prediction, location-related prediction improvements can be applied. For instance, different weights can be applied to each sample in the filled region based on the sample's location.

[0160] In one aspect, the motion compensation block (e.g., (1410)) used for motion compensation filling is corrected using an offset.

[0161] In one example, the offset is the difference (e.g., average) between the DC value of the reconstructed boundary block (e.g., BBlK (1404)) and the DC value of the corresponding reference block (e.g., ref BBlK (1408)).

[0162] In one aspect, orientation fill is applied to motion compensation when the motion vector of the current block points outside the boundary of the reference image (e.g., (1406)).

[0163] In one example, the direction is derived by performing gradient-based intra-frame mode derivation on a portion of the boundary block that lies within the boundary. For example, when BBlK(1404) crosses the boundary (1406), the portion of the boundary block is the part of BBlK(1404) that lies within the boundary (1406).

[0164] In one example, gradient-based intra-frame mode derivation is applied to a portion of the reference block (e.g., (1408)). An orientation is selected based on HoG. Orientation padding is then performed using the selected orientation.

[0165] In one example, when the reference block (e.g., (1408)) is an intra-coded block, the encoded intra-mode is used as the direction. In another example, when the reference block is encoded with an intra-angle mode, such as a diagonal mode, the direction is set to a diagonal direction.

[0166] Aspects of the present invention include methods and systems relating to reference shifting methods for intra-frame prediction.

[0167] Based on the assumption that the current block is highly correlated with its neighboring samples, conventional intra-frame prediction uses reconstructed samples (i.e., reference samples) to generate the predicted block. Various intra-frame prediction modes, such as planar mode, DC mode, angle mode, matrix-based intra-frame prediction, and template-based intra-frame prediction, provide multiple options for approximating the original signal. However, block-by-block coding schemes have certain limitations. For example, when the minimum size of the current block is 4×4, it may not be possible to predict the current block using reference samples located between neighboring 4×4 blocks with the same prediction mode. Figure 17 In the example shown, the vertical prediction of block 0 (1702) can be performed using reference samples r0, r1, r2, and r3. However, due to the 4×4 block size constraint, it may not be possible to generate a vertical prediction signal for block 0 (1702) using r1, r2, r3, and r4.

[0168] Aspects of the present invention include methods and systems relating to reference-shifted intra-frame prediction methods, such as diversified intra-frame prediction using reference samples.

[0169] In this invention, the reference samples used for intra-frame prediction can be shifted to provide the same intra-frame prediction mode to different prediction blocks. Figure 18 An example is shown where a reference sample is shifted to perform a vertical prediction. For example... Figure 18 As shown, the current block (1802) uses the shifted reference sample for prediction. Figure 18 As shown, configuration (1800A) displays the original position of the reference sample for the current block (1802). This reference sample can be shifted to the right in configuration (1800B) or to the left in configuration (1800C). Therefore, different reference samples can be used to perform vertical prediction for the current block (1802).

[0170] In one respect, reference sample shifting is used for one of the possible intra-frame predictions.

[0171] In one example, reference sample shifting is used for regular intra-frame prediction modes, such as planar prediction, DC prediction, and angle prediction.

[0172] In one example, reference sample shifting is used for matrix-based intra-frame prediction modes.

[0173] In one example, reference sample shifting is used for template-based intra-frame prediction modes.

[0174] In one respect, the reference sample is shifted by N samples.

[0175] In one example, N is one of the integers greater than 0, such as 1, 2, 3, ...

[0176] In one example, N is a fractional value, such as 0.5, 1.5, 2.5, ...

[0177] In one example, when N is a score value, the reference sample is interpolated using multiple integer samples.

[0178] In one respect, the maximum N (or the maximum value of N) is equal to the minimum coding unit size minus 1. For example, when the minimum coding unit size is 4, the maximum N is 3.

[0179] In one respect, the maximum N of the luminance component or the chrominance component is different. In one example, when the minimum coding unit size of the luminance component and the chrominance component are different, the maximum N of each component is also different.

[0180] In one respect, reference sample shifting is applied based on at least one factor, such as block size, block shape, or block type.

[0181] In one respect, the top reference sample is shifted to the left or right, and the left reference sample is shifted to the up or down. Figure 19 An example of the shift direction is provided. For example... Figure 19 As shown, the top reference sample (1902) is shifted to the left or right. The left reference sample (1904) is shifted up or down.

[0182] In one aspect, template-based intra-frame prediction is used to apply reference sample shifting.

[0183] In one example, all possible reference sample shifts are applied to the reference samples of a template. The template can then be predicted using the reference samples shifted according to these possible reference sample shifts. By comparing the cost between the predicted template based on each possible reference sample shift and the reconstruction of that template, it can be determined whether to apply reference sample shifts, or which of the possible reference sample shifts to apply.

[0184] In one respect, reference sample shifting is performed only for the MPM.

[0185] In one example, when performing a reference sample shift, the intra-prediction mode is one of the MPM modes. In this case, the MPM flags can be inherited.

[0186] In one respect, the direction or total number of shifts associated with the reference sample shift is indicated by signaling.

[0187] In one example, a signaling instruction is used to indicate a syntax that specifies the direction in which the reference sample is shifted.

[0188] In one example, a signaling instruction is used to indicate a syntax that specifies how many reference samples are shifted.

[0189] Various aspects of the present invention include methods and systems for intra-frame mode derivation using signaling indication syntax and MPM adaptation.

[0190] Intra-frame prediction utilizes the spatial redundancy between the current block and its neighboring samples. Intra-frame prediction modes can be classified as directional or non-directional modes, indicating whether the current block is directionally or non-directionally correlated with a neighboring reference block. In such cases, the intra-frame mode information can be explicitly indicated in the bitstream via signaling. Based on the signaled intra-frame mode, corresponding prediction values ​​are generated based on the reference samples.

[0191] Intra-frame modes can also be derived in other ways. For example, intra-frame modes can be implicitly derived at the decoder. In such cases, intra-frame mode information can be derived from a predefined region with reconstructed samples. For example, in the first method, a template adjacent to the current block can be used, and the intra-frame mode can be derived based on gradients. Gradient-based intra-frame mode derivation (or method A) can generate a HoG using neighboring reconstructed samples of the current block. Based on the HoG, the first N gradients are mapped to at least one regular intra-frame mode, and the predictions based on this at least one regular intra-frame mode are combined into a final prediction.

[0192] Figure 20 An example of intra-frame mode derivation based on decoder gradients is shown. For example... Figure 20 As shown, HoG (2002) is derived based on the template (2004) of the current block (2006). The multiple intra-prediction modes can be derived by mapping the multiple gradients with the highest magnitudes in the histogram to multiple intra-prediction modes (e.g., modes 0 to 4). Each of these multiple intra-prediction modes can generate a prediction value for the current block (2006) (e.g., (2008)). The final prediction (2010) can be defined as a combination of these prediction values.

[0193] In one example, such as Figure 20 As shown, HoG can be constructed based on samples in a predefined region of a template (2004). In one example, the predefined region (2012) includes samples in the center row and center column of the template (2004). HoG can be determined based on the texture orientation of adjacent samples in the predefined region of the template. Figure 21 An example of HoG derivation is provided in the document.

[0194] like Figure 21As shown, the current coding block (2101) and its template (2103) are provided. A horizontal Sobel filter (2102) and a vertical Sobel filter (2104) can be applied to samples in a predefined region (2105) of the template (2103). In one example, the horizontal and vertical Sobel filters have predefined windows, such as 3×3 windows. By applying the horizontal and vertical Sobel filters to samples in the predefined region (2105), such as samples in the center row and center column of the template (2103), the horizontal gradient Gx and vertical gradient Gy are derived in each 3×3 window. The ratio of Gx to Gy within each 3×3 window of the template is calculated, which can provide intra-frame mode information of adjacent templates. In each 3×3 window, the ratio of Gx to Gy can be mapped to the closest regular intra-frame prediction mode, and the matching intra-frame modes are included in the histogram. The 3x3 window can slide through (or slide window through) the template and update the histogram based on the intra-frame pattern matched by each 3x3 window.

[0195] While Method A offers signaling savings (requiring only one flag / binary bit), the fusion of the final predictions does not always yield superior results. Furthermore, the fused prediction may differ from a single prediction derived by applying a single intra-frame mode. This difference can diminish potential bit savings compared to conventional MPM list signaling, which generates a single prediction from an intra-frame mode within the MPM list.

[0196] In one aspect, the first N modes derived in method A can form an MDM list. By managing the constraints between the MDM list and the MPM list, the signaling of intra-frame modes can be optimized. If an intra-frame mode appears in both the MDM and MPM lists, the encoder can use MDM syntax for signaling indication. Since MDM signaling precedes MPM signaling, MDM syntax may have lower bit cost. When intra-frame modes appearing in both the MDM and MPM lists are selected, the ordering of MDM and MPM can reduce signaling overhead.

[0197] Furthermore, if an MDM is not selected by the MDM syntax, it can be excluded from the MPM list. In one example, including such an MDM in the MPM list would be redundant, as the encoder would logically not choose an MPM syntax with higher bit cost to indicate the same prediction value with signaling. Excluding unselected MDMs from the MPM list allows for more efficient use of the MPM list. It increases the space for other candidate modes and improves overall coding efficiency.

[0198] Various aspects of this invention include a hybrid implicit pattern derivation combining MPM constraints and explicit pattern signaling indication. In one example, the maximum implicit pattern derivation is performed via method A. N There are several intra-frame modes. These modes are considered as MDMs, and an MDM list is constructed based on these MDMs. At least one syntax can be explicitly indicated by signaling to determine which MDM to apply. The final prediction is generated based on the determined MDM without fusing with other predictions. This at least one syntax can be indicated by signaling before the regular MPM list-related syntax (e.g., MPM flags, MPM indexes, etc.). Furthermore, depending on the requirements of constraints or limitations, it is ensured that the MDMs and intra-frame modes in the MPM list are included in the same set. In one example, the MPMs and MDMs are obtained in different ways. The MPM can rely on the similarity of neighboring blocks. The MDM can rely on the texture of neighboring samples. For example, the MDM can be based on... Figure 20-21 The derivation is based on gradient-based intra-frame mode derivation. MPM and MDM can be the same or different.

[0199] In one respect, the same set shared by the MDM and MPM lists is formed by intra-frame patterns that correspond to predictions generated through unconventional intra-frame prediction. Unconventional intra-frame prediction includes, but is not limited to, predictions generated by neural networks, matrix multiplication using predefined coefficients in a lookup table, and other similar methods.

[0200] In one example, the set shared between the MDM and MPM is the intra-frame modes numbered in the following two sets: (i) Mode 0, Mode 1, and Mode (2+2) (i) k), k belongs to 0 to 32, including end values ​​(set 1); and (ii) pattern 0, pattern 1 and pattern (2+4) k), k belongs to 0 to 16, including end values ​​(set 2).

[0201] In one example, if an MDM does not belong to set 1 or 2, the MDM is discarded. In another example, if an MDM does not belong to set 1 or 2, the MDM is quantized to the nearest pattern in set 1 or 2. In another example, if an MPM does not belong to set 1 or 2, the MPM is discarded. In yet another example, if an MPM does not belong to set 1 or 2, the MPM is quantized to the nearest pattern in set 1 or 2.

[0202] In one aspect, the MDM list is constructed by ranking. This ranking is based on the cost (e.g., sum of absolute differences (SAD) or sum of absolute transformed differences (SATD)) between reconstructed samples and predicted samples generated by MDM within a predefined template region.

[0203] In one example, signaling is used to indicate that the MDM requires a syntax. Flag This is used to indicate whether MDM signaling is activated, and another syntax is required. index This is used to indicate which intra-mode (or MDM) in the MDM list to use. In one example, only the intra-mode with the lowest cost is used, so there is no need to indicate the index with signaling. 。 In one example, the size of the MDM list is a predefined number, such as 8.

[0204] In one respect, the prediction patterns in the MDM list and the MPM list are mutually exclusive.

[0205] In one aspect, a threshold is introduced in method A. This threshold is used to terminate the construction of HoG based on a certain condition. For example, as... Figure 22 As shown, HoG(2202) is being constructed based on DIMD. The construction of the HoG is terminated when the accumulated HoG(2204) is greater than and / or equal to a given threshold.

[0206] In one aspect, method A is used to derive at most K (K > 0) modes. Intra-frame modes corresponding to the minimum S (S > 0) HoGs are excluded from intra-frame mode coding. In one example, S is 16. In another example, K is 67.

[0207] In one aspect, in method A, the MDM is derived using a top template region, a left template region, or both a top template region and a left template region. In one example, when the current block (2304) uses a single top template (2302), the reference sample (2306) is as follows: Figure 23 As shown. Figure 23 As shown, the reference sample (2306) includes the sample on the top side of the template (2302) and the sample on the left side of the current block (2304). In one example, a set of predefined matrix coefficients is applied to the reference sample. This set of predefined matrix coefficients can be multiplied with the reference sample to generate a prediction for the current block. A prediction pattern can be derived based on the prediction for the current block. This prediction pattern can be identified as an MDM in the MDM list.

[0208] In one respect, when the current block (2408) uses a single left-side template (2402), the reference sample (2408) is as follows: Figure 24 As shown. A set of predefined matrix coefficients can be further applied to the reference sample (2408). This set of predefined matrix coefficients can be multiplied with the reference sample (2408) to generate a prediction for the current block (2406). The MDM can then be derived (or indicated) based on this prediction for the current block.

[0209] In one aspect, the MDM is derived from a reference region guided by the corresponding block vector. For example... Figure 25 As shown, the current coding block (2502) corresponds to multiple reference blocks (2503)-(2505). Each reference block can be indicated by a corresponding block vector, such as BV1-BV3. In one example, the cost value between the template region of each reference block and the template region of the current block is calculated. The reference block corresponding to the minimum cost value is selected as the prediction block of the current block. The MDM can also be further derived based on the prediction block of the current block.

[0210] Various aspects of the present invention include methods and systems relating to HoG-based orientation plane patterns.

[0211] Several methods for derivation of intra-prediction modes at the decoder end have been proposed, such as template cost-based intra-prediction mode derivation and decoder gradient-based intra-prediction mode derivation. Gradient-based intra-prediction mode derivation can generate a HoG using the nearest neighbor samples of the current block. Based on this histogram, the first N gradients are mapped to each intra-prediction mode, and the predictions based on these intra-prediction modes are combined into the final prediction value. An example of decoder gradient-based intra-prediction mode derivation is shown below. Figure 20 As shown.

[0212] Simultaneously, directional plane modes that include only horizontal interpolation or only vertical interpolation can be used to obtain predicted samples. For the horizontal plane mode, horizontal linear interpolation is performed only based on the left and upper right reference samples to predict the current sample. For the vertical plane mode, vertical linear interpolation is performed only based on the upper and lower left reference samples to predict the current sample. Signaling indicator flags and indices can be used to indicate the use and direction of the directional plane mode, respectively.

[0213] Aspects of the present invention include a HoG-based orientation plane method. The orientation (e.g., vertical or horizontal) of a plane pattern (e.g., a vertical plane pattern or a horizontal plane pattern) can be determined based on a HoG in a template. Furthermore, the HoG can be used to determine the transformation index of the orientation plane pattern.

[0214] In one aspect, the template used to derive HoG includes spatially adjacent reconstructed samples.

[0215] In one respect, HoG is generated from isolated reconstructed samples.

[0216] In one example, the left HoG is generated from the reconstructed sample on the left.

[0217] In one example, the HoG above is generated from the reconstructed sample above.

[0218] In one respect, the orientation of a planar pattern is not limited to the horizontal and vertical directions.

[0219] In one example, the orientation of the planar mode is one of the intra-frame angle modes.

[0220] In one example, the orientation of the planar pattern is represented as a scalar in the range of 0 to 360 degrees.

[0221] In one respect, the orientation of the planar mode is determined according to HoG.

[0222] In one example, the orientation of the plane pattern is determined by the HoG values ​​of the horizontal and / or vertical plane patterns in the template. The orientation plane pattern with the largest HoG value is selected. Therefore, the orientation of the orientation plane pattern is inferred by using the HoG (or by the largest HoG value).

[0223] In one example, the HoG values ​​for the horizontal and vertical plane patterns in the template are calculated separately. These two HoG values ​​are then sorted in descending order. A signaling index is used to indicate which candidate to select from the sorted list.

[0224] In one aspect, the first N gradients are used (e.g., the first N largest values ​​in HoG).

[0225] In one example, when the optimal (or selected) gradient (e.g., the gradient corresponding to the maximum HoG value) is mapped to a near-horizontal mode, the orientation of the plane mode can be inferred as a horizontal plane mode.

[0226] In one example, when the optimal gradient is mapped to a near-vertical mode, the orientation of the planar mode can be inferred as a vertical planar mode.

[0227] In one example, when the optimal gradient is mapped to a near-vertical mode, the orientation of the planar mode can be inferred as a horizontal planar mode.

[0228] In one example, when the optimal gradient is mapped to a near-horizontal mode, the orientation of the planar mode can be inferred as a vertical planar mode.

[0229] In one example, when the current N gradients are similar to the horizontal pattern, the direction of the planar pattern can be inferred as the horizontal planar pattern.

[0230] In one respect, the HoG values ​​can be weighted and summed to define the HoG value.

[0231] In one example, a weighted sum is used to combine the HoG values ​​of the horizontal and near-horizontal modes based on the difference between the horizontal mode value and the near-horizontal mode value.

[0232] In one aspect, the first N gradients from each individual HoG (e.g., the vertical HoG derived from the top template and the horizontal HoG derived from the left template) are used.

[0233] In one example, the best gradients from each HoG (e.g., the gradient corresponding to the largest magnitude) are compared to determine the orientation of the planar pattern. When the difference between the best gradients is less than a threshold and close to a near-horizontal pattern, the orientation of the planar pattern can be inferred as a horizontal planar pattern.

[0234] In one aspect, HoG is used to determine the transformation index of the orientation plane mode.

[0235] In one example, the best gradient (e.g., the gradient corresponding to the largest magnitude) is used as the transformation index.

[0236] In one example, in addition to the optimal gradient (as a supplement to the optimal gradient), an offset increment value is indicated by signaling, which indicates the offset of the inferred transform index.

[0237] Figure 26 A flowchart of an overview process (2600) according to one aspect of the present invention is shown. The process (2600) can be used with a video decoder. In various aspects, the process (2600) is executed by processing circuitry, such as processing circuitry that performs the functions of the video decoder (110), processing circuitry that performs the functions of the video decoder (210), etc. In some aspects, the process (2600) is implemented in the form of software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes the process (2600). The process begins at (S2601) and continues to (S2610).

[0238] In (S2610), a video stream is received, which includes encoded information of the current block in the current image. The encoded information indicates that the current block is encoded based on intra-frame prediction using an extrapolation filter according to one of a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients of that corresponding filter.

[0239] In (S2620), a reference value is determined based on the adjacent samples in the template region of the current block.

[0240] In (S2630), based on the comparison between the reference value and the threshold, the candidate model used to predict the current block is determined from among the multiple candidate models.

[0241] In (S2640), the range of predicted sample values ​​for the current block is determined. Based on one of the multiple candidate models, the predicted sample values ​​for the current block are obtained.

[0242] In (S2650), the current block is reconstructed based on the portion of the predicted sample values ​​that are within the limit range.

[0243] In one aspect, when the reference value is equal to or less than a threshold, a first candidate model is determined from among the multiple candidate models for predicting the current block. When the reference value is greater than the threshold, a second candidate model is determined from among the multiple candidate models for predicting the current block.

[0244] In one respect, the reference value is determined as the average value of adjacent samples in the template region of the current block.

[0245] In one aspect, one or two neighboring samples in the template region of the current block are determined based on the filter shape of the filter associated with the first candidate model among the plurality of candidate models. By inputting the one or two neighboring samples into the first candidate model among the plurality of candidate models, a reference value is calculated as a weighted average of the one or two neighboring samples.

[0246] In one aspect, when the reference value is equal to or less than the threshold, the minimum value of the limiting range is determined to be either 0 or a first predefined value, and the maximum value of the limiting range is determined to be the sum of the threshold and a second predefined value. When the reference value is greater than the threshold, the minimum value of the limiting range is determined to be the threshold minus the second predefined value, and the maximum value of the limiting range is determined to be (i) a third predefined value and (ii) one of them.

[0247] In one aspect, a first filter associated with a first candidate model among the plurality of candidate models is applied to iteratively traverse neighboring samples in the template region to obtain a plurality of first reference values. A second filter associated with a second candidate model among the plurality of candidate models is applied to iteratively traverse neighboring samples in the template region to obtain a plurality of second reference values. Based on the minimum and maximum values ​​of the plurality of first reference values, a first limiting range associated with the first candidate model among the plurality of candidate models is determined. Based on the minimum and maximum values ​​of the plurality of second reference values, a second limiting range associated with the second candidate model among the plurality of candidate models is determined.

[0248] The process then continues to (S2699) and terminates.

[0249] The process (2600) can be adjusted as appropriate. One or more steps in the process (2600) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0250] Aspects of the present invention include a video coding method. In this method, a reference value is determined based on neighboring samples in a template region of a current block in a current image. The current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients of that corresponding filter. A candidate model for predicting the current block is determined based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. The current block is encoded based on the portion of the predicted sample values ​​that falls within the limiting range.

[0251] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media bitstream encoded by an encoding method. In this encoding method, a reference value is determined based on neighboring samples in a template region of a current block in a current image. The current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients for that corresponding filter. A candidate model for predicting the current block is determined based on a comparison between the reference value and a threshold. A limiting range for the predicted sample values ​​of the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. The current block is encoded based on the portion of the predicted sample values ​​that falls within the limiting range.

[0252] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current block in a current image. The encoded information indicates that the current block is encoded based on intra-frame prediction using an extrapolation filter according to a candidate model from a plurality of candidate models. Each of the plurality of candidate models is associated with a corresponding filter and includes a plurality of filter coefficients for that corresponding filter. Format rules specify that a reference value is determined based on neighboring samples in a template region of the current block. Format rules specify that a candidate model from the plurality of candidate models used to predict the current block is determined based on a comparison between the reference value and a threshold. Format rules specify that a limiting range of predicted sample values ​​for the current block is determined. Predicted sample values ​​for the current block are obtained based on the candidate model from the plurality of candidate models. Format rules specify that the current block is processed based on the portion of the predicted sample values ​​that falls within the limiting range.

[0253] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media stream that, when processed by at least one processor, causes the at least one processor to execute a process (2600).

[0254] Figure 27 A flowchart illustrating an overview process (2700) according to one aspect of the invention is shown. Process (2700) can be used with a video decoder. In various aspects, process (2700) is executed by processing circuitry, such as processing circuitry that performs the functions of video decoder (110), processing circuitry that performs the functions of video decoder (210), etc. In some aspects, process (2700) is implemented as software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (2700). The process begins at (S2701) and continues to (S2710).

[0255] In (S2710), a video stream is received, which includes encoded information of the current image and a reference image of the current image.

[0256] In (S2720), it is determined whether to apply directional fill to the fill region. The fill region is located outside and adjacent to the boundary of the current image.

[0257] In (S2730), the filling direction of the directional fill is determined based on the direction of the intra-frame prediction mode. The intra-frame prediction mode is derived from one of the boundary blocks located within and adjacent to the boundary of the current image and the motion compensation blocks in the filling region.

[0258] In (S2740), a portion of the filling region is filled based on multiple samples along the filling direction in the current image.

[0259] In one aspect, when the size of the motion compensation block is smaller than the maximum motion compensation width or the maximum motion compensation height, it is determined that directional padding will be applied to the fill region of the current image. When a boundary block is encoded as an intra-frame block, it is determined that directional padding will be applied to the fill region of the current image. When a boundary block is arranged across the boundary of the current image, it is determined that directional padding will be applied to the fill region of the current image.

[0260] In one aspect, the padding direction is determined as one of the intra-prediction modes applied to the boundary block.

[0261] In one aspect, HoG is determined based on neighboring samples of the motion-compensated block. The intra-prediction mode is then determined based on the HoG. The padding direction is determined to be the direction indicated by the intra-prediction mode.

[0262] In one aspect, the portion of the filling region is filled using multiple samples from the reference image that are closest to the filling region along the filling direction indicated by the intra-frame prediction mode.

[0263] In one aspect, multiple samples along the fill direction in the current image are identified. This fill direction is indicated by the intra-frame prediction mode. Weights are applied to each of these multiple samples to obtain weighted samples. The corresponding weights of these multiple samples are determined based on their positions in the current image. This portion of the fill region is filled based on the weighted samples.

[0264] In one aspect, the samples of the motion-compensated block are further adjusted with an offset. This offset is the difference between the DC value of the boundary block in the current image and the DC value of the reference block of the boundary block in the reference image.

[0265] In one aspect, when a boundary block is arranged across the boundary of the current image, the HoG (Homogeneous Image) is determined based on neighboring samples of the portion of the boundary block located within the current image. The intra-frame prediction mode is then determined based on the HoG. The padding direction is determined to be the direction indicated by the intra-frame prediction mode.

[0266] The process then continues to (S2799) and terminates.

[0267] The process (2700) can be adjusted as appropriate. One or more steps in the process (2700) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0268] Aspects of the present invention include a video coding method. In this method, it is determined whether to apply directional padding to a padding region. The padding region is located outside and adjacent to the boundary of a current image. A padding direction for the directional padding is determined based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block within and adjacent to the boundary of the current image and a motion compensation block in the padding region. A portion of the padding region is encoded by padding it based on multiple samples in the current image along the padding direction.

[0269] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media stream encoded by an encoding method. In this encoding method, it is determined whether directional padding is applied to a padding region. The padding region is located outside and adjacent to the boundary of a current image. A padding direction for the directional padding is determined based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block within and adjacent to the boundary of the current image and a motion compensation block in the padding region. A portion of the padding region is encoded by padding a portion of the padding region based on multiple samples in the current image along the padding direction.

[0270] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current image and a reference image of the current image. The format rules specify whether to apply directional padding to a padding region located outside and adjacent to a boundary of the current image. The format rules specify determining a padding direction for directional padding based on the direction of an intra-frame prediction mode, the direction of which is derived from one of a boundary block and a motion compensation block within and adjacent to the boundary of the current image. The format rules specify padding a portion of the padding region based on multiple samples along the padding direction in the current image.

[0271] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media stream that, when processed by at least one processor, causes the at least one processor to execute a process (2700).

[0272] Figure 28 A flowchart illustrating an overview process (2800) according to one aspect of the invention is shown. Process (2800) can be used with a video decoder. In various aspects, process (2800) is executed by processing circuitry, such as processing circuitry that performs the functions of video decoder (110), processing circuitry that performs the functions of video decoder (210), etc. In some aspects, process (2800) is implemented as software instructions, so that when processing circuitry executes software instructions, processing circuitry executes process (2800). The process begins at (S2801) and continues to (S2810).

[0273] At (S2810), a video stream is received, which includes encoded information of the current block in the current image. The encoded information indicates the prediction mode of the current block, which is associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information.

[0274] In (S2820), based on the reference area of ​​the current block, a list of MDMs including multiple candidate MDMs is determined.

[0275] In (S2830), the current block is reconstructed based on at least one of the following: (i) a candidate MDM from the plurality of candidate MDMs in the MDMs list, and (ii) a candidate MPM from the MPMs list.

[0276] In one aspect, the MPMs list and the MDMs list share at least one candidate mode. The at least one candidate mode includes one of the following: (i) an unconventional intra-prediction mode, (ii) an empty entry; (iii) a first subset including mode 0, mode 1 and mode (2+2×k1), where k1 is an integer from 0 to 32; and (iv) a second subset including mode 0, mode 1 and mode (2+4×k2), where k2 is an integer from 0 to 16.

[0277] In one aspect, when a candidate MDM in the list of multiple candidate MDMs is not included in either the first subset or the second subset, that candidate MDM in the list of multiple candidate MDMs is discarded or quantized as the nearest pattern in the first subset or the second subset. Similarly, when a candidate MPM in the list of multiple candidate MPMs is not included in either the first subset or the second subset, that candidate MPM in the list of multiple candidate MPMs is discarded or quantized as the nearest pattern in the first subset or the second subset.

[0278] In one aspect, the HoG (House of Gaussian) associated with samples in a reference region of the current block is determined. Samples in the reference region are adjacent to the current block. Multiple candidate MDMs are identified from the list of MDMs. These multiple candidate MDMs correspond to multiple maximum amplitude values ​​in the HoG. These multiple maximum amplitude values ​​are greater than a predefined threshold.

[0279] In one aspect, multiple sets of predefined matrix coefficients are received from the video stream. Multiple sets of predicted samples for a reference region are determined. The reference region is positioned on the top or left side of the current block. Each of these multiple sets of predicted samples is determined by multiplying a sample from the reference region with the corresponding set of predefined matrix coefficients. The multiple candidate MDMs in the MDMs list are determined based on these multiple sets of predicted samples from the reference region.

[0280] In one aspect, multiple sub-reference regions of a reference region are determined based on multiple block vectors. Each sub-reference region is indicated by a corresponding block vector. The cost value between the template region of each sub-reference region and the template region of the current block is calculated. A subset of the multiple sub-reference regions corresponding to the minimum cost value among the calculated cost values ​​is determined. This candidate MDM among the multiple candidate MDMs is based on this subset of the multiple sub-reference regions.

[0281] The process then continues to (S2899) and terminates.

[0282] The process (2800) can be adjusted as appropriate. One or more steps in the process (2800) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0283] Aspects of the present invention include a video coding method. In this method, a list of MPMs for a current block in a current image is determined. A list of MDMs for the current block is determined. The list of MDMs includes a plurality of candidate MDMs based on a reference region of the current block. The current block is encoded into a bitstream based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0284] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media bitstream encoded by an encoding method. In this encoding method, a list of MPMs for a current block in a current image is determined. A list of MDMs for the current block is determined. The list of MDMs includes a plurality of candidate MDMs based on a reference region of the current block. The current block is encoded into the bitstream based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0285] Aspects of the present invention include a method for processing visual media data. The method includes processing a bitstream of visual media data according to format rules. The bitstream includes encoded information of a current block in a current image. The encoded information indicates a prediction mode for the current block, which is associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information. The format rules specify that the list of MDMs, including a plurality of candidate MDMs, is determined based on a reference region of the current block. The format rules specify that the current block is processed based on at least one of: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

[0286] Aspects of the present invention include a non-transitory computer-readable storage medium storing a video media stream that, when processed by at least one processor, causes the at least one processor to execute a process (2800).

[0287] The above-described technologies can be implemented in the form of computer software using computer-readable instructions and physically stored in at least one computer-readable medium. For example, Figure 29 A computer system suitable for implementing certain aspects of the disclosed subject matter is shown (2900).

[0288] Computer software can be written using any suitable machine code or computer language. This machine code or computer language may be processed by mechanisms such as assembly, compilation, and linking to generate code containing instructions that can be executed directly by at least one computer central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, or other methods.

[0289] These instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, and so on.

[0290] Figure 29 The components of the computer system (2900) shown are merely examples and are not intended to imply any limitation on the scope or functionality of computer software used to implement aspects of the present invention. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination of components shown in the example aspects of the computer system (2900).

[0291] The computer system (2900) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user via, for example, tactile input (e.g., key presses, swipes, data glove movements), audio input (e.g., voice, clapping sounds), visual input (e.g., gestures), or olfactory input (not depicted). The human-machine interface device may also be used to acquire media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0292] Human-machine interface input devices may include at least one of the following (only one of each is depicted): keyboard (2901), mouse (2902), touchpad (2903), touch screen (2910), data glove (not shown), joystick (2905), microphone (2906), scanner (2907), and camera (2908).

[0293] The computer system (2900) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback provided by a touchscreen (2910), a data glove (not shown), or a joystick 2905, but there may also be tactile feedback devices that do not act as input devices), audio output devices (e.g., speakers (2909), headphones (not depicted)), visual output devices (e.g., screens (2910), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality—some of which may be able to output two-dimensional or more three-dimensional visual outputs through stereoscopic output, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted), etc.), and printers (not depicted).

[0294] The computer system (2900) may also include user-accessible storage devices and their associated media, such as optical media or similar media (2921) including CD / DVD ROM / RW (2920) equipped with CD / DVD, flash drives (2922), removable hard disk drives or solid-state drives (2923), conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and dedicated ROM / ASIC / PLD devices such as security dongles (not depicted).

[0295] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0296] The computer system (2900) may also include an interface (2954) connected to at least one communication network (2955). The network may be, for example, a wireless network, a wired network, or an optical network. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an in-vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of networks include: local area networks such as Ethernet and wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; in-vehicle and industrial networks including CAN buses, etc. Some networks typically require an external network interface adapter (e.g., a USB port of the computer system (2900)) attached to some general-purpose data port or peripheral bus (2949); other networks are typically integrated into the core of the computer system (2900) via a system bus described below (e.g., integrated into a PC computer system via an Ethernet interface, or integrated into a smartphone computer system via a cellular network interface). Using any of these networks, the computer system (2900) can communicate with other entities. This communication can be one-way receiving (e.g., broadcasting TV), one-way transmitting (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as connecting to other computer systems using a local area or wide area digital network. Certain protocols and protocol stacks can be used on each of the networks and network interfaces described above.

[0297] The aforementioned human-machine interface device, user-accessible storage device, and network interface can be attached to the core (2940) of the computer system (2900).

[0298] The core (2940) may include at least one CPU (2941), GPUs (2942), a dedicated programmable processing unit in the form of a field-programmable gate array (FPGA) (2943), a hardware accelerator (2944) for certain tasks, a graphics adapter (2950), etc. These devices, along with read-only memory (ROM) (2945), random-access memory (RAM) (2946), and internal mass storage devices (2947) such as internal non-user-accessible hard disk drives, SSDs, etc., can be connected via a system bus (2948). In some computer systems, the system bus (2948) may be accessed in the form of at least one physical connector, allowing for expansion via additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (2948) or connected to the core's system bus (2948) via a peripheral bus (2949). In one example, a screen (2910) may be connected to a graphics adapter (2950). Peripheral bus architectures include PCI, USB, etc.

[0299] CPUs (2941), GPUs (2942), FPGAs (2943), and accelerators (2944) can execute certain instructions that combine to form the aforementioned computer code. This computer code can be stored in ROM (2945) or RAM (2946). Temporary data can also be stored in RAM (2946), while permanent data can be stored, for example, in an internal mass storage device (2947). Fast storage and retrieval of any of the storage devices can be achieved by using a cache memory, which can be closely associated with at least one CPU (2941), GPU (2942), mass storage device (2947), ROM (2945), RAM (2946), etc.

[0300] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be designed and constructed specifically for the purposes of this invention, or may belong to a class well-known and available to those skilled in the art of computer software.

[0301] For example, but not as a limitation, a computer system having an architecture (2900), particularly a core (2940), can provide functionality generated when one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in at least one tangible computer-readable medium. Such a computer-readable medium can be a medium associated with certain storage devices (e.g., internal core mass storage (2947) or ROM (2945)) that are user-accessible mass storage devices as described above and with the non-volatile nature of the core (2940). Software implementing various aspects of the invention can be stored in such devices and executed by the core (2940). Depending on specific needs, the computer-readable medium may include at least one storage device or chip. The software can cause the core (2940), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described in this application, including defining data structures stored in RAM (2946) and modifying such data structures according to the processes defined by the software. Alternatively or as an alternative, the computer system may provide functionality generated by hard-wired logic or otherwise embodied in circuitry (e.g., an accelerator (2944)), which may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may cover logic, and vice versa. Where appropriate, references to computer-readable media may cover circuitry (e.g., integrated circuits) storing software for execution, circuitry embodying logic for execution, or both. This invention covers any suitable combination of hardware and software.

[0302] The terms “at least one of…” or “one of…” as used in this invention are intended to include any one or a combination of the listed elements. For example, mentioning at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A to C are intended to include only A, only B, only C, or any combination thereof. Mentioning one of A or B and one of A and B are intended to include either A or B or (A and B). Where applicable, such as when the elements are not mutually exclusive, the use of “one of…” does not exclude any combination of the listed elements.

[0303] Although several examples of various aspects have been described in this invention, modifications, substitutions, and various alternative equivalents exist that fall within the scope of this invention. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, while not explicitly stated or described herein, embody the principles of the invention and are therefore within its spirit and scope.

[0304] The aforementioned disclosure also includes the features mentioned below. These features can be combined in various ways, and are not limited to the combinations mentioned below.

[0305] (1) A video decoding method, the method comprising: receiving a video stream including encoded information of a current block in a current image, the encoded information indicating that the current block is encoded based on intra-frame prediction of an extrapolation filter according to a candidate model among a plurality of candidate models, each of the plurality of candidate models being associated with a corresponding filter and including a plurality of filter coefficients of the corresponding filter; determining a reference value based on neighboring samples in a template region of the current block; determining a candidate model among the plurality of candidate models for predicting the current block based on a comparison between the reference value and a threshold; determining a limiting range of predicted sample values ​​of the current block, the predicted sample values ​​of the current block being obtained based on the candidate model among the plurality of candidate models; and reconstructing the current block based on a portion of the predicted sample values ​​within the limiting range.

[0306] (2) According to the method of feature (1), wherein determining the candidate model among the plurality of candidate models further includes: when the reference value is equal to or less than the threshold, determining a first candidate model among the plurality of candidate models for predicting the current block; and when the reference value is greater than the threshold, determining a second candidate model among the plurality of candidate models for predicting the current block.

[0307] (3) According to the method of feature (1) or (2), wherein determining the reference value includes: determining the reference value as the average value of the adjacent samples in the template region of the current block.

[0308] (4) The method according to any one of features (1) to (3), wherein determining the reference value includes: determining one or two adjacent samples in the template region of the current block based on the filter shape of the filter associated with the first candidate model among the plurality of candidate models; and calculating the reference value as a weighted average of the one or two adjacent samples by inputting the one or two adjacent samples into the first candidate model among the plurality of candidate models.

[0309] (5) The method according to any one of features (1) to (4), wherein determining the limiting range of the predicted sample value further includes: when the reference value is equal to or less than the threshold, determining the minimum value of the limiting range as 0 and one of a first predefined value, and determining the maximum value of the limiting range as the sum of the threshold and a second predefined value; and when the reference value is greater than the threshold, determining the minimum value of the limiting range as the threshold minus the second predefined value, and determining the maximum value of the limiting range as (i) a third predefined value and (ii) one of them.

[0310] (6) The method according to any one of features (1) to (5), wherein determining the limiting range of the predicted sample value further includes: applying a first filter associated with a first candidate model among the plurality of candidate models to iteratively traverse the neighboring samples in the template region to obtain a plurality of first reference values; applying a second filter associated with a second candidate model among the plurality of candidate models to iteratively traverse the neighboring samples in the template region to obtain a plurality of second reference values; determining a first limiting range associated with the first candidate model among the plurality of candidate models based on the minimum and maximum values ​​of the plurality of first reference values; and determining a second limiting range associated with the second candidate model among the plurality of candidate models based on the minimum and maximum values ​​of the plurality of second reference values.

[0311] (7) A video decoding method, the method comprising: receiving a video stream including encoded information of a current image and a reference image of the current image; determining whether to apply directional filling to a filling region located outside and adjacent to a boundary of the current image; determining a filling direction of the directional filling based on the direction of an intra-frame prediction mode, the intra-frame prediction mode being derived from one of a boundary block located inside and adjacent to the boundary of the current image and a motion compensation block in the filling region; and filling a portion of the filling region based on a plurality of samples in the current image along the filling direction.

[0312] (8) The method according to feature (7), wherein determining whether to apply the directional fill further includes one of the following: when the size of the motion compensation block is less than the maximum motion compensation width or the maximum motion compensation height, determining whether to apply the directional fill to the fill region of the current image; when the boundary block is encoded as an intra-frame block, determining whether to apply the directional fill to the fill region of the current image; and when the boundary block is arranged across the boundary of the current image, determining whether to apply the directional fill to the fill region of the current image.

[0313] (9) According to the method of feature (7) or (8), wherein determining the filling direction further includes: determining the filling direction as the direction of the intra-prediction mode applied to the boundary block.

[0314] (10) The method according to any one of features (7) to (9), wherein determining the filling direction further includes: determining HoG based on neighboring samples of the motion compensation block; determining the intra-prediction mode based on the HoG; and determining the filling direction as the direction indicated by the intra-prediction mode.

[0315] (11) The method according to any one of features (7) to (10), wherein the filling further includes: filling the portion of the filling region using the plurality of samples in the reference image that are closest to the filling region along the filling direction indicated by the intra-frame prediction mode.

[0316] (12) The method according to any one of features (7) to (11), wherein the filling further includes: determining the plurality of samples in the current image along the filling direction indicated by the intra-prediction mode; applying weights to the plurality of samples respectively to obtain weighted samples, the corresponding weights of the plurality of samples being determined based on the positions of the plurality of samples in the current image; and filling the portion of the filling region based on the weighted samples.

[0317] (13) The method according to any one of features (7) to (12), wherein the sample of the motion compensation block is further adjusted by an offset, the offset being the difference between the DC value of the boundary block in the current image and the DC value of the reference block of the boundary block in the reference image.

[0318] (14) The method according to any one of features (7) to (13), wherein: when the boundary block is arranged across the boundary of the current image, the determination of the filling direction further includes: determining HoG based on neighboring samples of the portion of the boundary block located within the current image; determining the intra-frame prediction mode based on the HoG; and determining the filling direction as the direction indicated by the intra-frame prediction mode.

[0319] (15) A video decoding method, the method comprising: receiving a video stream including encoded information of a current block in a current image, the encoded information indicating a prediction mode of the current block, the prediction mode being associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information; determining a list of MDMs including a plurality of candidate MDMs based on a reference region of the current block; and reconstructing the current block based on at least one of: (i) a candidate MDM among the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM among the list of MPMs.

[0320] (16) The method according to feature (15), wherein: the list of MPMs and the list of MDMs share at least one candidate mode, and the at least one candidate mode includes one of the following: (i) an unconventional intra-prediction mode, (ii) an empty entry; (iii) a first subset including mode 0, mode 1 and mode (2+2×k1), where k1 is an integer from 0 to 32, and (iv) a second subset including mode 0, mode 1 and mode (2+4×k2), where k2 is an integer from 0 to 16.

[0321] (17) According to the method of feature (15) or (16), wherein: when one of the candidate MDMs in the list of MDMs is not included in either the first subset or the second subset, the candidate MDM in the list of MDMs is discarded or quantized as the nearest pattern in the first subset and the second subset, and when one of the candidate MPMs in the list of MPMs is not included in either the first subset or the second subset, the candidate MPM in the list of MPMs is discarded or quantized as the nearest pattern in the first subset and the second subset.

[0322] (18) The method according to any one of features (15) to (17), wherein determining the list of MDMs further includes: determining a HoG associated with a sample in the reference region of the current block, the sample in the reference region being adjacent to the current block; and determining a plurality of candidate MDMs in the list of MDMs corresponding to a plurality of maximum amplitude values ​​in the HoG, the plurality of maximum amplitude values ​​being greater than a predefined threshold.

[0323] (19) The method according to any one of features (15) to (18), wherein determining the list of MDMs further includes: receiving multiple sets of predefined matrix coefficients from the video stream; determining multiple sets of prediction samples of the reference region, the reference region being arranged at one of the top and left sides of the current block, each of the multiple sets of prediction samples being determined by multiplying a sample of the reference region with a corresponding set of the multiple sets of predefined matrix coefficients; and determining the multiple candidate MDMs in the list of MDMs based on the multiple sets of prediction samples of the reference region.

[0324] (20) The method according to any one of features (15) to (19), wherein the determination of the list of MDMs further includes: determining a plurality of sub-reference regions of the reference region based on a plurality of block vectors, each of the plurality of sub-reference regions being indicated by a corresponding block vector; calculating the cost value between the template region of each of the plurality of sub-reference regions and the template region of the current block; determining a subset of the plurality of sub-reference regions corresponding to the minimum cost value among the calculated cost values; and deriving one of the plurality of candidate MDMs based on the subset of the plurality of sub-reference regions.

[0325] (21) An apparatus for video decoding, comprising processing circuitry configured to perform a method according to any one of features (1) to (6).

[0326] (22) An apparatus for video decoding, comprising processing circuitry configured to perform a method according to any one of features (7) to (14).

[0327] (23) An apparatus for video decoding, comprising processing circuitry configured to perform a method according to any one of features (15) to (20).

[0328] (24) A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method according to any one of features (1) to (20).

[0329] (25) A non-transitory computer-readable storage medium storing a video media stream that, when processed by at least one processor, causes the at least one processor to perform the method according to any one of features (1) to (20).

Claims

1. A method of video decoding, the method comprising: The method includes: Receive a video stream, the video stream including encoded information of a current block in the current image, the encoded information indicating that the current block is encoded based on intra-frame prediction of an extrapolation filter according to one of a plurality of candidate models, each of the plurality of candidate models being associated with a corresponding filter and including a plurality of filter coefficients of the corresponding filter; A reference value is determined based on adjacent samples in the template region of the current block; Based on the comparison between the reference value and the threshold, one candidate model is determined from the plurality of candidate models to predict the current block; Determine a limiting range for the predicted sample values ​​of the current block, wherein the predicted sample values ​​of the current block are obtained based on one of the plurality of candidate models; and The current block is reconstructed based on a portion of the predicted sample values ​​within the specified limit.

2. The method of claim 1, wherein, The step of determining the candidate model among the plurality of candidate models further includes: When the reference value is equal to or less than the threshold, a first candidate model for predicting the current block is determined from the plurality of candidate models; and When the reference value is greater than the threshold, a second candidate model is determined from the plurality of candidate models to predict the current block.

3. The method according to claim 1 or 2, characterized in that, Determining the reference value includes: The reference value is determined as the average value of the neighboring samples in the template region of the current block.

4. The method according to any one of claims 1 to 3, characterized in that, Determining the reference value includes: The filter shape is based on the filter associated with the first candidate model among the plurality of candidate models. Determine one or two adjacent samples in the template region of the current block; and By inputting the one or two adjacent samples into the first candidate model among the plurality of candidate models, the reference value is calculated as the weighted average of the one or two adjacent samples.

5. The method according to any one of claims 1 to 4, characterized in that, The determination of the amplitude limit range for the predicted sample value also includes: When the reference value is equal to or less than the threshold, the minimum value of the limiting range is determined to be either 0 or a first predefined value, and the maximum value of the limiting range is determined to be the sum of the threshold and a second predefined value; and determining the minimum value of the clipping range as the threshold value minus the second predefined value and determining the maximum value of the clipping range as one of (i) a third predefined value and (ii) the threshold value plus the second predefined value when the reference value is greater than the threshold value. determining the minimum value of the clipping range as the threshold value minus the second predefined value and determining the maximum value of the clipping range as one of (i) a third predefined value and (ii) the threshold value plus the second predefined value when the reference value is greater than the threshold value.

6. A method of video decoding, comprising: The method includes: Receive a video stream, the video stream including encoded information of the current image and a reference image of the current image; Determine whether to apply directional fill to the fill region located outside and adjacent to the boundary of the current image; Based on the orientation of the intra-frame prediction mode, the filling orientation of the direction fill is determined, wherein the intra-frame prediction mode is derived from one of the boundary blocks located within and adjacent to the boundary of the current image and the motion compensation blocks in the filling region; and A portion of the filling region is filled based on multiple samples in the current image along the filling direction.

7. The method according to claim 6, characterized in that, The determination of whether to apply the directional fill also includes one of the following: When the size of the motion compensation block is smaller than the maximum motion compensation width or the maximum motion compensation height, it is determined that the directional fill will be applied to the fill area of ​​the current image; When the boundary block is encoded as an intra-frame block, it is determined that the directional fill will be applied to the fill region of the current image; as well as When the boundary block spans the boundary arrangement of the current image, it is determined that the directional fill will be applied to the fill area of ​​the current image.

8. The method according to claim 6 or 7, characterized in that, The determination of the filling direction further includes: The filling direction is determined as the direction of the intra-prediction mode applied to the boundary block.

9. The method according to any one of claims 6 to 8, characterized in that, The determination of the filling direction further includes: Based on the neighboring samples of the motion compensation block, HoG is determined; Based on the HoG, the intra-frame prediction mode is determined; and The filling direction is determined to be the direction indicated by the intra-frame prediction mode.

10. The method according to any one of claims 6 to 9, characterized in that, The filling also includes: The portion of the filling region is filled using the plurality of samples in the reference image that are closest to the filling region along the filling direction indicated by the intra-frame prediction mode.

11. A video decoding method, characterized in that, The method includes: Receive a video stream, the video stream including encoded information of the current block in the current image, the encoded information indicating the prediction mode of the current block, the prediction mode being associated with a list of MPMs generated based on a first type of prediction information and a list of MDMs generated based on a second type of prediction information; Based on the reference region of the current block, a list of MDMs including multiple candidate MDMs is determined; and The current block is reconstructed based on at least one of the following: (i) a candidate MDM from the plurality of candidate MDMs in the list of MDMs, and (ii) a candidate MPM from the list of MPMs.

12. The method according to claim 11, characterized in that: The MPMs list and the MDMs list share at least one candidate pattern, and The at least one candidate mode includes one of the following: (i) an unconventional intra-frame prediction mode, (ii) an empty entry; (iii) A first subset including pattern 0, pattern 1 and pattern (2+2×k1), where k1 is an integer from 0 to 32, and (iv) a second subset including pattern 0, pattern 1 and pattern (2+4×k2), where k2 is an integer from 0 to 16.

13. The method according to claim 12, characterized in that: When one of the candidate MDMs in the list of candidate MDMs is not included in either the first subset or the second subset, that candidate MDM is discarded or quantized as the nearest pattern in the first subset and the second subset. When one of the candidate MPMs in the list of candidate MPMs is not included in either the first subset or the second subset, the candidate MPM in the list of candidate MPMs is discarded or quantized as the most recent pattern in the first subset and the second subset.

14. The method according to any one of claims 11 to 13, characterized in that, Determining the list of MDMs also includes: Determine the HoG associated with a sample in the reference region of the current block, wherein the sample in the reference region is adjacent to the current block; and Identify a plurality of candidate MDMs in the list of MDMs that correspond to a plurality of maximum amplitude values ​​in the HoG, wherein the plurality of maximum amplitude values ​​are greater than a predefined threshold.

15. The method according to any one of claims 11 to 14, characterized in that, Determining the list of MDMs also includes: Receive multiple sets of predefined matrix coefficients from the video stream; Multiple sets of predicted samples are determined for the reference region, which is located at either the top or left side of the current block. Each set of predicted samples is determined by multiplying a sample from the reference region by a corresponding set of predefined matrix coefficients. Based on the multiple sets of predicted samples in the reference region, the multiple candidate MDMs in the MDMs list are determined.

Images