Fine-grained intra prediction fusion

Abstract

Aspects of the present disclosure include methods and apparatus of video decoding and encoding and a method of processing visual media data. A method for video decoding includes receiving encoded information in a code stream, the encoded information indicating that a current block is predicted based on a combination of a plurality of intra prediction modes. The method includes: determining a plurality of intra predictions of the current block based on corresponding intra prediction modes; determining a fusion prediction of the current block based on a weighted sum of the plurality of intra-prediction, wherein the weighted sum is in accordance with respective weights associated with the plurality of intra-prediction; and reconstructing the current block based on the fusion prediction. Each weight is based on one of a plurality of weighting functions that depends on the sample position (x, y) and the intra prediction mode of the intra prediction associated with the respective weight.

Description

Related applications

[0001] This application claims the benefit of priority to U.S. Patent Application No. 18 / 923,634, filed on October 22, 2024, which claims the benefit of priority to U.S. Provisional Application No. 63 / 601,689, filed on November 21, 2023. The entire disclosure of the prior application is hereby incorporated by reference herein. Technical Field

[0002] This disclosure describes various aspects generally related to video coding. Background Art

[0003] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. No admission is made, either explicitly or implicitly, that the work of the presently named inventors is prior art to the present disclosure to the extent that such work is described in this background section and to the extent that it may not qualify as prior art at the time of filing.

[0004] Image / video compression can help transmit image / video data between different devices, storage devices and networks while minimizing quality degradation. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In an example, a video codec can use a technique called intra-frame prediction, which can compress an image based on spatial redundancy. For example, intra-frame prediction can use reference data of the current picture being reconstructed to predict samples. In another example, a video codec can use a technique called inter-frame prediction, which can compress an image based on temporal redundancy. For example, inter-frame prediction can predict samples in the current picture based on a previously reconstructed picture through motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Aspects of the present disclosure include methods and apparatus for video encoding / decoding.

[0006] According to one aspect of the present disclosure, a method for video decoding includes receiving encoded information in a code stream. The encoded information indicates that a current block is predicted based on a combination of multiple intra-frame prediction modes. The method for video decoding includes: determining multiple intra-frame predictions of the current block based on corresponding intra-frame prediction modes; determining a fused prediction of the current block based on a weighted sum of the multiple intra-frame predictions of the current block, wherein the weighted sum is based on corresponding weights associated with the multiple intra-frame predictions; and reconstructing the current block based on the fused prediction. Each of the weights is based on a weighting function among multiple weighting functions, which depends on the sample position (x, y) and the intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0007] In one aspect, a method of video encoding includes determining a plurality of intra predictions for a current block based on a plurality of corresponding intra prediction modes; determining a fused prediction for the current block based on a weighted sum of the plurality of intra predictions for the current block, wherein the weighted sum is based on corresponding weights associated with the plurality of intra predictions; and encoding the current block based on the fused prediction. Each of the weights is based on one of a plurality of weighting functions that depends on a sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight.

[0008] In one aspect, a method for processing visual media data includes processing a codestream of the visual media data according to a format rule. The codestream includes a syntax element indicating that a current block is predicted based on a combination of multiple intra-frame prediction modes. The format rule specifies: determining multiple intra-frame predictions for the current block based on corresponding intra-frame prediction modes; determining a fused prediction for the current block based on a weighted sum of the multiple intra-frame predictions for the current block, wherein the weighted sum is based on corresponding weights associated with the multiple intra-frame predictions; reconstructing the current block based on the fused prediction; and each of the weights is based on a weighting function among multiple weighting functions, the weighting function being dependent on a sample position (x, y) and an intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0009] Aspects of the present disclosure further provide an apparatus for video encoding, wherein the apparatus comprises a processing circuit configured to implement any one of the methods for video encoding.

[0010] Aspects of the present disclosure further provide an apparatus for video decoding. The apparatus for video decoding includes a processing circuit configured to implement any one of the methods for video decoding.

[0011] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer, cause the computer to perform any of the methods for video decoding / encoding.

[0012] The technical solutions of the present disclosure include various aspects related to determining a fused prediction for a current block based on a weighted sum of multiple intra predictions for the current block. The weighted sum can be based on corresponding weights associated with the multiple intra predictions. In one aspect, the weighting function depends on the sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight. In an example, the weighting function is further based on subblock partition information of the current block, wherein the weighting function in different subblocks of the current block can be determined separately. Therefore, the weighting function used in the present disclosure processes heterogeneous region-related features of different prediction signals used in the fused intra prediction, and can generate a fused prediction that is a more accurate prediction than a fused prediction using other relevant weighting functions that do not take into account subblock partition information and / or intra prediction knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0014] Figure 1 is a schematic diagram of an example of a block diagram of a communication system (100).

[0015] Figure 2 is a diagram showing an example of a block diagram of a decoder.

[0016] Figure 3 is a diagram showing an example of a block diagram of an encoder.

[0017] Figure 4 An example of intra prediction according to an aspect of the present disclosure is shown.

[0018] Figure 5 An example of 67 intra prediction modes according to an aspect of the present disclosure is shown.

[0019] Figure 6 An example of a template-based intra mode derivation method according to an aspect of the present disclosure is shown.

[0020] Figure 7 An example of decoder-side intra mode derivation (DIMD) according to an aspect of the present disclosure is shown.

[0021] Figures 8A to 8D An example of sub-block partitioning of a current block according to an aspect of the present disclosure is shown.

[0022] Figure 9 A flowchart outlining a decoding method according to some aspects of the present disclosure is shown.

[0023] Figure 10 A flowchart outlining an encoding method according to some aspects of the present disclosure is shown.

[0024] Figure 11 is a schematic diagram of a computer system according to one aspect. DETAILED DESCRIPTION

[0025] Figure 1 A block diagram of a video processing system (100) is shown in some examples. The video processing system (100) is an example of an application of the disclosed subject matter, a video encoder and a video decoder in a streaming environment. The disclosed subject matter is also applicable to other video-enabled applications, including, for example, video conferencing, digital television, streaming services, and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0026] The video processing system (100) may include an acquisition subsystem (113), which may include a video source (101) such as a digital camera, which creates an uncompressed video picture stream (102). In an embodiment, the video picture stream (102) includes samples captured by the digital camera. The video picture stream (102) is depicted as a thick line to emphasize the high data volume of the video picture stream compared to the encoded video data (104) (or encoded video bitstream), and the video picture stream (102) may be processed by an electronic device (120) including a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or embody various aspects of the disclosed subject matter as described in more detail below. Compared to the video picture stream (102), the encoded video data (104) (or the encoded video code stream (104)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (104) (or the encoded video code stream), which can be stored on the streaming server (105) for future use. One or more streaming client subsystems, such as Figure 1The client subsystem (106) and the client subsystem (108) in the streaming server (105) can access the streaming server (105) to retrieve the copy (107) and the copy (109) of the encoded video data (104). The client subsystem (106) can include, for example, a video decoder (110) in the electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and produces an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) can be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application can be used in the context of the VVC standard.

[0027] It should be noted that the electronic devices (120) and (130) may include other components (not shown). For example, the electronic device (120) may integrate a video decoder (not shown), and the electronic device (130) may also integrate a video encoder (not shown).

[0028] Figure 2 is an example of a block diagram of a video decoder (210). The video decoder (210) may be provided in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuit). The video decoder (210) may be used to replace Figure 1 A video decoder (110) of an embodiment.

[0029] A receiver (231) may receive one or more encoded video sequences to be decoded by a video decoder (210), the one or more encoded video sequences being included in, for example, a bitstream. In one aspect, the encoded video sequences are received one at a time, wherein the decoding of each encoded video sequence is independent of the decoding of the other encoded video sequences. The encoded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective consuming entities (not shown). The receiver (231) may separate the encoded video sequence from the other data. To mitigate network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be provided outside the video decoder (210) (not shown). In other cases, a buffer memory (not shown) may be provided outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be provided inside the video decoder (210) to, for example, handle broadcast timing. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network, the buffer memory (215) may not be required, or the buffer memory may be made smaller. Of course, for use on a traffic packet network such as the Internet, a buffer memory (215) may also be required. The buffer memory may be relatively large and may have an adaptive size, and may be at least partially implemented in an operating system or similar component (not shown) outside the video decoder (210).

[0030] The video decoder (210) may include a parser (220) for reconstructing symbols (221) from an encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potentially information for controlling a display device, such as a display device (212) (e.g., a display screen), which is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown in . The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may be performed according to a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set for at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (220) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0031] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0032] Depending on the type of coded video picture or portion of a coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (220). For the sake of brevity, the flow of such subgroup control information between the parser (220) and the multiple units below is not described.

[0033] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually broken down into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and may be integrated with each other. However, for the purposes of describing the disclosed subject matter, the conceptual breakdown into the following functional units is appropriate.

[0034] The first unit is a scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) along with control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) may output a block comprising sample values, which may be input to an aggregator (255).

[0035] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates surrounding blocks of the same size and shape as the block being reconstructed using reconstructed information extracted from the current picture buffer (258). For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0036] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to inter-frame coded and potentially motion compensated blocks. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (221), these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signal) by the aggregator (255) to generate output sample information. The retrieval of the prediction samples by the motion compensated prediction unit (253) from the address in the reference picture memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbols (221), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values extracted from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0037] The output samples of the aggregator (255) may be used by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filtering techniques that are controlled by parameters included in the coded video sequence (also referred to as the coded video bitstream) and that are available to the loop filter unit (256) as symbols (221) from the parser (220). The video compression techniques may also be responsive to meta-information obtained during decoding of a coded picture or a previous (in decoding order) portion of the coded video sequence, and to previously reconstructed and loop filtered sample values.

[0038] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0039] Once fully reconstructed, certain coded pictures can be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257) and a new current picture buffer can be reallocated before starting to reconstruct a subsequent coded picture.

[0040] The video decoder (210) may perform decoding operations according to a predetermined video compression technique, such as that in the ITU-T H.265 standard. The coded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the coded video sequence follows the syntax of the video compression technique or standard and a profile recorded in the video compression technique or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technique or standard as the only tools available for use under the profile. For compliance, the complexity of the coded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the coded video sequence.

[0041] In one aspect, a receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, and the like.

[0042] Figure 3 is an example of a block diagram of a video encoder (303). The video encoder (303) is provided in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an embodiment.

[0043] The video encoder (303) can be used to generate a video from a video source (301) (not Figure 3 In another embodiment, the video source (301) is a part of the electronic device (320) to receive video samples, wherein the video source can capture video images to be encoded by the video encoder (303). In another embodiment, the video source (301) is a part of the electronic device (320).

[0044] The video source (301) may provide a source video sequence in the form of a stream of digital video samples to be encoded by the video encoder (303), wherein the stream of digital video samples may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that are imparted with motion when viewed sequentially. The pictures themselves may be constructed as a spatial array of pixels, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. used. The following description focuses on samples.

[0045] According to an embodiment, the video encoder (303) can encode and compress pictures of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints as required. Implementing an appropriate encoding speed is a function of the controller (350). In some embodiments, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of simplicity, the coupling is not shown in the figure. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate-distortion optimization technology, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0046] In some embodiments, the video encoder (303) operates in a coding loop. As a simplified description, in some embodiments, the coding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Because the decoding of the symbol stream produces bit-accurate results that are independent of the decoder's location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction portion of the encoder are exactly the same sample values that the decoder will "see" when using prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0047] The operation of the "local" decoder (333) can be combined with the operation of Figure 2 The "remote" decoder of the video decoder (210) is described in detail. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0048] In one aspect, decoder techniques other than parsing / entropy decoding present in the decoder are also present in the corresponding encoder in the same or substantially the same functional form. Therefore, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are mutually inverse to the decoder techniques described comprehensively. In some areas, a more detailed description is provided below.

[0049] During operation, in some embodiments, the source encoder (330) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes an input picture with reference to one or more previously encoded pictures in a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the differences between pixel blocks of the input picture and pixel blocks of a reference picture that may be selected as a prediction reference for the input picture.

[0050] The local video decoder (333) may decode the encoded video data of a picture that may be designated as a reference picture based on the symbols created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture that will be obtained by the remote video decoder.

[0051] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as suitable prediction references for the new picture. The predictor (335) may operate on a pixel block-by-pixel block basis based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory (334).

[0052] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0053] The outputs of all the above functional units may be entropy coded in an entropy coder (345). The entropy coder (345) losslessly compresses the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0054] The transmitter (340) can buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which can be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) can combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or an auxiliary data stream (source not shown).

[0055] The controller (350) can manage the operation of the video encoder (303). During encoding, the controller (350) can assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture can generally be assigned to any of the following picture types:

[0056] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0057] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses one motion vector and reference index to predict sample values for each block.

[0058] Bidirectionally predictive pictures (B pictures) can be encoded and decoded using intra prediction or inter prediction, which uses two motion vectors and reference indices to predict the sample values of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata to reconstruct a single block.

[0059] A source picture is typically spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples) and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or they may be predictively coded (spatial or intra-predicted) with reference to already coded blocks of the same picture. Pixel blocks of a P picture may be predictively coded using spatial prediction with reference to one previously coded reference picture or using temporal prediction. Blocks of a B picture may be predictively coded using spatial prediction with reference to one or two previously coded reference pictures or using temporal prediction.

[0060] The video encoder (303) may perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In operation, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video coding technique or standard used.

[0061] In an embodiment, the transmitter (340) may transmit additional data along with the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, redundant pictures and slices, and other forms of redundant data, SEI messages, VUI parameter set fragments, and the like.

[0062] The captured video may be presented as a temporal sequence of multiple source pictures (video pictures). Intra-picture prediction (often shortened to intra prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a previously encoded and buffered reference picture in the video, the block in the current picture can be encoded using a vector called a motion vector. The motion vector points to the reference block in a reference picture, and when multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0063] In some embodiments, bidirectional prediction techniques can be used for inter-picture prediction. According to bidirectional prediction techniques, two reference pictures are used, for example, a first reference picture and a second reference picture, both preceding the current picture in the video in decoding order (but potentially in the past and future, respectively, in display order). A block in the current picture can be encoded using a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0064] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0065] According to some embodiments disclosed in the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally speaking, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine the prediction type used for the CU, such as an inter prediction type or an intra prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example, the prediction block includes a matrix of pixel values (e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0066] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. In one aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In another aspect, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors executing software instructions.

[0067] In intra prediction or intra prediction mode, the sample values of a coding block in the current picture can be predicted based on already reconstructed samples in the current picture (called reference samples). These samples may be in one or more reference rows.

[0068] The prediction block can be formed using the values of adjacent samples of an existing sample. The sample values of adjacent samples can be copied into the prediction block according to a direction. The reference to the direction in use can be encoded in the codestream, or it can be predicted itself.

[0069] An example of intra prediction is planar mode, which can use bilinear interpolation. In this planar mode, one or more locations in the current block can be predicted using reference samples in a reference row. Other locations in the current block can be predicted as a linear combination of one or more samples at one or more locations and the reference samples. Weights can be determined based on the position of the current sample in the current block.

[0070] An example of intra prediction is DC mode. To predict samples in a block using DC mode, the average of the samples in the reference rows may be used as the prediction value.

[0071] An example of intra prediction is angular intra prediction. This angular intra prediction may also be referred to as an angular prediction method or an angular prediction mode. In angular intra prediction, a current sample in a current block may be predicted using reference samples (e.g., prediction samples) or interpolated reference samples (e.g., interpolated reference samples in a reference row), such as Figure 4 shown.

[0072] refer to Figure 4 , depicted in the lower right is a subset of nine prediction directions from a plurality of prediction directions (e.g., 33 angular modes out of the 35 intra modes in H.265). The point (401) where the arrows converge represents the sample being predicted. Each of these arrows may represent a respective direction in which the sample is being predicted. For example, arrow (402) indicates that sample (401) is predicted based on one or more samples to the upper right at a 45 degree angle from horizontal. Similarly, arrow (403) indicates that sample (401) is predicted based on one or more samples to the lower left of sample (401) at a 22.5 degree angle from horizontal.

[0073] Still refer to Figure 4 , at the top left, a square block (404) of 4×4 samples is depicted (indicated by the bold dashed line). The square block (404) includes 16 samples, each sample labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (404) in both the Y and X dimensions. Since the size of the block is 4×4 samples, S44 is at the bottom right. Reference samples are further shown that follow a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index), and X position (column index) relative to the block (404). In some examples, the prediction samples are adjacent to the block being reconstructed; therefore, negative values do not need to be used.

[0074] Intra-picture prediction can be performed by copying the reference sample values of adjacent samples indicated by the signaled prediction direction. For example, assume that the encoded video stream includes signaling that, for this block, the prediction direction is consistent with arrow (402), that is, the samples are predicted based on the samples to the upper right at a 45-degree angle from the horizontal direction. In this case, samples S41, S32, S23, and S14 are predicted based on the same reference sample R05. Sample S44 is then predicted based on reference sample R08.

[0075] In some cases, the values of multiple reference samples may be combined, for example by interpolation, in order to calculate the reference sample; in particular when the directions cannot be evenly divided by 45°.

[0076] As video coding techniques have advanced, the number of possible directions has increased. In examples, such as in H.264, nine different directions can be represented. In examples, such as in H.265, 33 different directions can be used. In examples, such as in JEM / VVC / BMS, up to 65 directions can be used. Experiments have been conducted to identify the most likely directions and use certain techniques in entropy coding to represent these possible directions with a small number of bits, with some penalty for less likely directions. In some examples, these directions can be predicted based on the neighboring directions used in adjacent, decoded blocks.

[0077] In order to improve coding efficiency, various intra-frame prediction coding tools are introduced, including but not limited to: angular intra-frame prediction with 65 angles and 4-tap interpolation filter, wide-angle intra-frame prediction (WAIP), position-dependent intra-frame prediction combination (PDPC), multiple reference line (MRL) prediction, intra-frame subpartitioning (ISP) mode, matrix-based intra-frame prediction (MIP), cross-component linear model (CCLM), intra-frame mode coding with 6 most probable modes (MPM), etc.

[0078] Figure 5 A schematic diagram (501) depicting 67 intra-frame prediction modes, for example, according to JEM is shown. The 67 intra-frame prediction modes may include 65 angular modes (or angular prediction modes), a planar mode, and a DC mode, each corresponding to 65 intra-frame prediction directions. The mapping of intra-frame prediction direction bits representing directions in the encoded video stream may vary depending on the video coding technology. For example, such mapping may range from a simple direct mapping to codewords to complex adaptive schemes involving most probable modes, and similar techniques. However, in most cases, there may be certain directions that are statistically less likely to appear in the video content than certain other directions. Since the goal of video compression is to reduce redundancy, in a well-performing video coding technology, those less likely directions may be represented by a larger number of bits than more likely directions.

[0079] In some examples, the intra prediction mode used to predict the current block may include an angular prediction mode such as Figure 5 65 angle modes and non-angle modes are shown. Examples of non-angle modes include Figure 5 The shown planar mode (e.g., mode 0), DC mode (e.g., mode 1), etc.

[0080] In some examples, a template can be used to derive an intra prediction mode for a current block (also referred to as a current coding block). The current coding block and its neighboring samples can share similar texture features. Therefore, the current coding block can be predicted using neighboring reconstructed samples of the current coding block. The template can include neighboring samples of the current coding block (e.g., neighboring reconstructed samples).

[0081] Method A may include a template-based derivation method that can derive a prediction mode, such as an intra-prediction mode, for a current block. In an example, Method A includes template-based intra-prediction mode derivation (TIMD) that can be applied to the current coding block. When the template has a good correlation with the current coding block, the intra-prediction mode applied to the template can well indicate the current block.

[0082] refer to Figure 6, adjacent reconstructed samples of the current CU (or current block) (602) can be used as a template (604). The template (604) can have any suitable shape and include any suitable number of samples. The template (604) can be adjacent to the current CU (602). In one aspect, the template (604) includes reconstructed samples. Figure 6 In the example shown, the template (604) has an "L" shape and includes a top template above the current CU (602) and a left template to the left of the current CU (602). In the example, the template (604) includes a top template adjacent to the top left corner of the current CU (602). In the example, the template (604) includes a top template above the current CU (602). In the example, the template (604) includes a left template to the left of the current CU (602).

[0083] The reference sample (606) may be adjacent to the template (604). In an example, the reference sample (606) may include at least one row (eg, at least one line and / or at least one column) of samples.

[0084] In some examples, according to method A, intra mode derivation using the template may include one or more of the following steps:

[0085] In step 1, a set of samples can be defined as a reference for the template (604), such as Figure 6 This set of samples can be used as reference samples (606) to generate a prediction signal for the template (604).

[0086] In step 2, an intra prediction mode may be performed (e.g., applied) on the reference sample (606) of the template (604) to generate a prediction signal, e.g., a prediction of the template (604). In an example, the set of predefined intra prediction modes may include predefined intra modes (also referred to as predefined intra prediction modes), and the intra prediction mode applied to the reference sample (606) of the template (604) is one of the predefined intra modes.

[0087] In step 3, a cost between the predicted signal of the template (604) and the template (604) may be determined. In an example, the template (604) is a reconstructed signal of the template (604), the reconstructed signal being represented by reconstructed samples in the template (604).

[0088] In an example, the cost is a sum of absolute transformed differences (SATD) cost between the predicted signal of the template (604) and the reconstructed signal of the template (604), and the cost is calculated. In an example, the cost is a mean removed sum of absolute differences (MRSAD) between the predicted signal of the template (604) and the reconstructed signal of the template (604).

[0089] In step 4, steps 2 and 3 may be repeated for another mode in the set of predefined intra prediction modes.The predefined intra modes may be ordered (eg, ranked) based on corresponding costs (eg, SATD costs or MRSAD costs).

[0090] In step 5, the mode with the minimum cost (eg, minimum SATD cost) is selected as the prediction mode for the current block or current CU (602). For ease of description, the determined prediction mode may be referred to as a template-based intra mode.

[0091] In some examples, in method A, multiple prediction modes (e.g., multiple intra-frame prediction modes) corresponding to one or more minimum costs in the costs can be selected for the current CU (602), and the multiple prediction modes can be referred to as template-based intra-frame modes.

[0092] In an example, method A may include TIMD. In one aspect, the TIMD may use the reference sample of the current CU as a template and select an intra mode from a set of candidate intra prediction modes associated with the TIMD. For example, the selected intra mode may be determined as the best intra mode based on a cost function. Figure 6 As shown, neighboring reconstructed samples of the current CU (602) can be used as a template (604). The reconstructed samples in the template (604) can be compared with the predicted samples of the template (604). The predicted samples can be generated using reference samples (606) of the template (604). The reference samples (606) can include neighboring reconstructed samples around the template (604). A cost function can be used to calculate the cost (or distortion) between the predicted samples and the reconstructed samples in the template (604) based on a corresponding one of a set of candidate intra-prediction modes, such as described in steps 3 to 4 above. The intra-prediction mode with the minimum cost (or distortion) can be selected as the intra-prediction mode (e.g., the best intra-prediction mode) to perform intra-prediction on the current CU (602), such as described in step 5. As described above, in some examples, in TIMD, multiple intra-prediction modes for the current CU (602) corresponding to one or more minimum costs can be selected, and the multiple prediction modes can be referred to as template-based intra-modes.

[0093] refer to Figure 6, the size of the reference sample (606) can depend on the size of the current CU (602) (such as the block width M and the block height N) and the size of the template (604) (such as the template width L1 of the left template and the template height L2 of the top template). In an example, if the reference sample (606) consists of only one row of samples (e.g., a column of samples to the left of the template (604) and a row of samples above the template (604)), the width of the reference sample (606) can be equal to 2(M+L1)+1, and the height of the reference sample (606) can be equal to 2(N+L2)+1.

[0094] In an example, method A may include decoder-side intra mode derivation (DIMD). When DIMD is applied, N intra modes may be derived based on reconstructed neighboring samples around the current block (701), and N prediction values obtained using these N intra modes may be combined with planar mode prediction values having corresponding weights. These weights may be derived based on gradients, such as a histogram of gradients (HoG) calculation. Figure 7 An example of DIMD is shown. HoG calculation can be performed by applying filters (e.g., a horizontal Sobel filter and a vertical filter) to pixels in a template (702) around the current block (701). The template (702) can include reconstructed neighboring samples around the current block (701). In the example, the width of the template is 3. In the example, pixels in the center line of the template (702) (marked in gray) can be involved in the HoG calculation. Reference Figure 7 , a window (703) around the pixel (705) can be used to determine a gradient associated with the pixel (705). The window (703) can have a size of 3×3, where the pixel (705) is located at the center of the window (703). For example, a horizontal Sobel filter and a vertical Sobel filter can be used to obtain a horizontal gradient and a vertical gradient, respectively. A direction or orientation can be obtained based on the horizontal gradient and the vertical gradient. An intra-frame prediction mode associated with the direction can be determined. Subsequently, a histogram (also called HoG) of intra-frame prediction modes can be obtained (710). The intra-frame prediction mode corresponding to the N highest histogram bins can be selected for the current block (701). In the example, the N intra-frame prediction modes can be referred to as template-based intra-frame modes.

[0095] Aspects of the present disclosure describe a fine-grained intra prediction fusion approach.

[0096] Video coding has been widely used in many applications. Various video coding standards such as H264, H265, H266 (or VVC), AV1 and AVS have been widely adopted. In one aspect, a video codec may include multiple modules, including inter-frame prediction, transform coding, quantization, entropy coding, loop filtering, etc. Intra-frame prediction may be one of the main modules and may include signaling processing methods (e.g., signaling processing methods) and neural network-based methods. The present disclosure includes methods for intra-frame prediction modules of video codecs. For example, a fine-grained intra-frame prediction fusion method is disclosed to improve the quality of intra-frame prediction.

[0097] As described above, in intra-frame prediction, samples of a given block, such as the current block, can be predicted using the spatially neighboring blocks of the current block. The current block and the spatially neighboring blocks of the current block may be in the same picture. In this example, for a given block, the samples in the given block are predicted using the reconstructed samples of the top and left neighbors of the given block. For a given block, the samples obtained from the spatial neighbors are called reference samples, and these reference samples are used to generate the intra-frame prediction for the given block (also called the intra-frame prediction signal).

[0098] Given a reference sample, various intra prediction methods can be applied to generate an intra prediction signal. Intra prediction methods can also be referred to as intra prediction modes. Intra prediction methods can be categorized into angular prediction methods (also referred to as angular prediction modes) and non-angular prediction methods (also referred to as non-angular prediction modes).

[0099] An intra-frame prediction method (or intra-frame prediction mode) can generate an intra-frame prediction (also called an intra-frame prediction signal) by emphasizing a specific pattern in a reference sample. Intra-frame prediction fusion may refer to a method in which intra-frame predictions (or intra-frame prediction signals) from multiple intra-frame prediction modes (or multiple intra-frame prediction methods) are combined to generate a fused prediction, which may be referred to as a final intra-frame prediction signal. The intra-frame prediction fusion described in the present disclosure may be referred to as intra-frame prediction fusion or intra-frame prediction signal fusion. In various examples, since the fused prediction signal includes multiple emphasized patterns, the quality of the fused prediction signal may be better than the quality of the corresponding prediction signal from each individual intra-frame prediction mode.

[0100] In one aspect, the fusion of intra predictions can be described as follows. For example, k intra predictions (or intra prediction signals) s1, ..., s k The fused signal (expressed as ) is given by:

[0101] x,y are sample indices. (x,y) can indicate the sample position being predicted in the current block. i(x,y) is a specified weighting function (such as the one with the i-th intra prediction s i The value (such as an integer value) of the sample weighting function associated with the prediction signal is used. In this example, 1≤x≤W and 1≤y≤H, where W and H represent the width and height of the prediction signal, respectively. W and H can represent the width and height of the current block being predicted. When the weighting function varies with sample position, the weighting function is called a sample weighting function.

[0102] In the related intra-frame prediction fusion method, the sample weighting function w i (x,y) is a function with the same analytical form over the entire intra prediction signal. In some examples, this design fails to address the heterogeneous region-dependent characteristics of different prediction signals used in the fusion process.

[0103] According to one aspect of the present disclosure, a fine-grained sample weighting function can be applied in intra-frame prediction fusion. In some examples, the current block to be intra-predicted (also called intra-frame prediction block) is partitioned into l sub-blocks R1, ..., R l , and l is a positive integer greater than 1. In the j-th sub-block R j (e.g., when the sample is within the jth sub-block, as represented by (x,y)∈R j Indicated), the weighting function can be determined separately for the i-th intra prediction mode based on information related to the intra prediction mode, the current block and the sub-block partition information This information may include characteristics of the intra prediction mode, block shape, block size, sub-block shape information, sub-block size information, sub-block position information, etc. In one aspect, the weighting function for the entire block or the entire current block is given by:

[0104] Figures 8A to 8D An example of sub-block partitioning of a current block (or intra-predicted block) (800) according to an aspect of the present disclosure is shown. Figure 8A , the current block (800) is partitioned into two sub-blocks, such as a lower left sub-block (eg, a lower left triangular sub-block) R1 and an upper right sub-block (eg, an upper right triangular sub-block) R2, where l is 2.

[0105] exist Figure 8B , the current block (800) is partitioned into two sub-blocks, such as an upper left sub-block (eg, an L-shaped upper left sub-block) R1 and a lower right sub-block (eg, a rectangular lower right sub-block) R2, where l is 2.

[0106] exist Figure 8C , the current block (800) is partitioned into three sub-blocks, such as a left sub-block R1, a top sub-block R2, and a bottom-right sub-block R3. R1 is on the left side of R3, R2 is above R3, and l is 3.

[0107] exist Figure 8D , the current block (800) is partitioned into four sub-blocks, such as an upper left sub-block R1, a lower left sub-block R2, an upper right sub-block R3, and a lower right sub-block R4, and l is 4. In the example, Figure 8D R1 to R4 in FIG. 5 are rectangular, however, the shapes of R1 to R4 may be different from the rectangular shape.

[0108] The proposed fine-grained intra prediction fusion method can improve the quality of the fused intra prediction signal.

[0109] According to one aspect of the present disclosure, an intra-frame prediction fusion method can be used to predict a current block based on a combination of multiple intra-frame prediction modes. Multiple intra-frame predictions for the current block can be determined based on the corresponding intra-frame prediction modes. The fused prediction for the current block can be determined based on a weighted sum of the multiple intra-frame predictions for the current block. The weighted sum can be based on corresponding weights associated with the multiple intra-frame predictions. The current block can be encoded (e.g., reconstructed or encoded) based on the fused prediction. Each of these weights can be based on one of a plurality of weighting functions that depends on the sample position (x, y) and the intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0110] In one aspect, the current block is partitioned into sub-blocks. The sub-block partition information of the current block indicates the sub-block position, sub-block shape, etc. of each sub-block. For example, in addition to the intra prediction mode, a corresponding one of the plurality of weighting functions (e.g., Indication) is further based on the sub-block partition information of the current block. Weighting function can be predicted with the i-th intra-frame s indicated by index i i The jth sub-block R indicated by index j in the sum sub-block j The i-th intra prediction mode can be used to determine the i-th intra prediction mode. i The index i is from 1 to k, k is the number of multiple intra predictions, and the index j is from 1 to 1, 1 is the number of subblocks.

[0111] In one aspect, the intra prediction signal (eg, i Each subblock (e.g., indicated by R j Instructions) determine the sample weighting function separately (for example, by For example, the sample weighting function associated with the same intra prediction mode (e.g., the i-th intra prediction mode) is determined separately in each sub-block. Thus, the sample weighting function associated with the i-th intra prediction mode in the sub-block R1 is determined separately. The sample weighting function associated with the i-th intra prediction mode in sub-block R2 The associated sample weighting function is thus solved for the heterogeneous region-dependent features of intra prediction used in intra prediction fusion.

[0112] In each sub-block, the prediction mode can be based on the intra prediction mode (which is related to the intra prediction s i associated) related information, intra-frame prediction block (or current block) and information such as Figures 8A to 8D The sample weighting function is determined by using the sub-block partitioning information shown. The sub-block partitioning information may indicate how to partition a block into sub-blocks or a partitioning scheme. The sub-block partitioning information may include a sub-block shape, a sub-block position, and / or a sub-block size of a corresponding sub-block in the current block.

[0113] In one aspect, each sub-block (eg, j The sample weighting function in the instruction (e.g., Indicated) is based on the intra prediction mode (eg, indicated by the i-th intra prediction mode) and the subblock position of the corresponding subblock. In an example, the sample weighting function may vary with the sample position (x, y).

[0114] In an example, the plurality of intra prediction modes include a horizontal angle prediction mode (eg, Figure 5 18) and vertical angular prediction modes (e.g., Figure 5 The multiple intra predictions include s1 associated with the horizontal angle prediction mode and s2 associated with the vertical angle prediction mode, and k is 2. The subblock partition information of the current block indicates that the current block is partitioned into a lower left triangle subblock R1 and an upper right triangle subblock R2, and l is 2, such as Figure 8A As shown. Multiple weighting functions include those associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 As described herein, a fusion method is used to fuse prediction signals s1 and s2, where s1 is from the horizontal angular prediction mode (or horizontal angular prediction method) and s2 is from the vertical angular prediction mode (or vertical angular prediction method). The sub-block partitions selected for this scene are Figure 8A A total of 4 separate weighting functions are selected. So that more weight is assigned to s1 in sub-block R1 and more weight is assigned to s2 in sub-block R2. and

[0115] In an example, the multiple intra prediction modes include an angular prediction mode and a non-angular prediction mode. The multiple intra prediction modes include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode. The subblock partition information of the current block indicates that the current block is partitioned into an L-shaped upper left subblock R1 and a rectangular lower right subblock R2, such as Figure 8B As shown. Multiple weighting functions include those associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 As described in this paper, the fusion method is used to fuse the prediction signals s1, s2, where s1 comes from the angular prediction method and s2 comes from the non-angular prediction method. The sub-block partitions selected for this scene are Figure 8B A total of 4 separate weighting functions are selected. This causes more weight to be assigned to s1 in sub-block R1, and thus And more weight is assigned to s2 in sub-block R2, and thus

[0116] In an example, the multiple intra prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode. The multiple intra prediction modes include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode. The sub-block partition information of the current block indicates that the current block is partitioned into a left sub-block R1, a top sub-block R2, and a bottom-right sub-block R3, where R1 and R2 are located to the left of R3 and above R3, respectively, such as Figure 8C As shown. Multiple weighting functions include those associated with s1 in R1 Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 and associated with s3 in R3 For example, the fusion method is used to fuse the prediction signals s1, s2, and s3, where s1 comes from the horizontal angle prediction method, s2 comes from the vertical angle prediction method, and s3 comes from the non-angle prediction method. The sub-block partitions selected for this scene are Figure 8C A total of 9 individual weighting functions are selected. This causes more weight to be assigned to s1 in sub-block R1, and thus Greater than and More weight is assigned to s2 in sub-block R2, and thus Greater than and And more weight is assigned to s3 in sub-block R3, and thus Greater than and

[0117] In an example, the multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode. The multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode. The subblock partition information of the current block indicates that the current block is partitioned into an upper left subblock R1, a lower left subblock R2, an upper right subblock R3, and a lower right subblock R4. The multiple weighting functions include s1 associated with R1. Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 Associated with s3 in R3 Associated with s1 in R4 Associated with s2 in R4 and associated with s3 in R4 As described herein, the fusion method is used to fuse the prediction signals s1, s2, and s3, where s1 comes from the horizontal angular prediction method, s2 comes from the vertical angular prediction method, and s3 comes from the non-angular prediction method. The sub-block partitions selected for this scene are Figure 8D A total of 12 individual weighting functions are selected. Such that equal weights are assigned to s1 and s2 in sub-block R1, and thus More weight is assigned to s1 in sub-block R2, and thus Greater than and More weight is assigned to s2 in sub-block R3, and thus Greater than and And more weight is assigned to s3 in sub-block R4, and thus Greater than and

[0118] In one aspect, the sample weighting function in each sub-block may be based on the intra prediction mode, the sub-block position, and the sample position (also referred to as pixel position) indicated by (x, y).

[0119] In an example, the multiple intra prediction modes include an angular prediction mode and a non-angular prediction mode. The multiple intra prediction modes include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode. The subblock partition information of the current block indicates that the current block is partitioned into an L-shaped upper left subblock R1 and a rectangular lower right subblock R2. The multiple weighting functions include: Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 As described herein, the fusion method is used to fuse the prediction signals s1, s2, where s1 comes from the angular prediction method and s2 comes from the non-angular prediction method. The sub-block partitions selected for this scene are Figure 8B The weighting function in each sub-block is defined as follows:

[0120] Here, C1, C2, C3 are constants (such as weight values that are independent of position), which can be derived based on a cost function such as SAD or SATD. For a region such as sub-block R1, the weighting of the two prediction values does not depend on the sample position, such as shown in equations (3) to (4). For a region such as sub-block R2, the weight of the angular prediction value s1 decreases toward the right direction and / or the bottom direction, as indicated by equation (5), and the weight of the non-angular prediction value s2 increases toward the right direction and / or the bottom direction, as indicated by equation (6). The symbol ∝ indicates this decreasing or increasing trend based on the sample position, which contributes to the final weighting of s1, s2. For example, and or When and When proportional and is proportional to, and when and When proportional and Directly proportional.

[0121] In an example, the plurality of intra-frame prediction modes include a first angular prediction mode, a second angular prediction mode, and a non-angular prediction mode. The plurality of intra-frame prediction modes include s1 associated with the first angular prediction mode, s2 associated with the second angular prediction mode, and s3 associated with the non-angular prediction mode. The sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2. The plurality of weighting functions includes associated with s1 in R1. Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 As described herein, the fusion method is used to fuse prediction signals s1, s2, and s3, where s1 and s2 are from angular prediction methods (e.g., the first angular prediction mode and the second angular prediction mode), and s3 is from a non-angular prediction method. The sub-block partitions selected for this scene are Figure 8B The weighting function in each sub-block is defined as follows: Here, C1, C2, C3, C4, C5 are constants (such as weight values that are independent of position), which can be derived based on a cost function such as SAD or SATD. For a region such as sub-block R1, the weights of the three prediction values s1, s2, s3 do not depend on the sample position, and and is a constant. For example, C6 is a constant. For a region such as sub-block R2, the weighting function of the angle prediction values s1 and s2 may decrease towards the right and / or bottom direction, e.g. and As one of x and y decreases, as shown in equations (9) and (10). The weighting of the non-angular prediction value s3 increases towards the right and / or bottom direction, for example, As one of x and y increases, such as shown in equation (11). The symbol ∝ indicates this decreasing or increasing trend based on the sample position that contributes to the final weighting of the prediction values s1, s2 and s3.

[0122] In this example, the sum of the weights of all prediction values (eg, all intra predictions at a sample position) is equal to 2 N , where N is a non-negative value, such as a non-negative integer. For example, for sub-block R j , (e.g., 32 or 64). In the example, for each sub-block,

[0123] In one aspect, the sample weighting function can be based on the intra prediction mode and the sample position. In some examples, the selection of the weighting function takes into account certain a priori knowledge of the intra prediction. For example, if the intra prediction is generated using an angular prediction mode, a weighting function that gives more weight to the upper left corner can be used. If the intra prediction is generated using a non-angular prediction mode such as a planar mode, a weighting function that gives more weight to the lower right corner can be used. This feature differs from the weighting functions used in some related arts, in which the same type of weighting function is used regardless of the intra prediction mode used to generate the intra prediction.

[0124] In an example, the plurality of intra prediction modes include at least one angular prediction mode and at least one non-angular prediction mode. At least one weighting function associated with the at least one angular prediction mode among the plurality of weighting functions decreases toward one of a right direction and a bottom direction. At least one weighting function associated with the at least one non-angular prediction mode among the plurality of weighting functions increases toward one of a right direction and a bottom direction.

[0125] At least one angular prediction mode is an angular prediction mode, and at least one non-angular prediction mode is a non-angular prediction mode. As described herein, a fusion method is used to fuse prediction signals s1, s2, where s1 is from an angular prediction method and s2 is from a non-angular prediction method. At least one weighting function associated with an angular prediction mode is or Proportional to w1(x,y), such as shown in equation (12), and W and H are the width and height of the current block respectively. When the at least one weighting function associated with the non-angular prediction mode is proportional to is proportional to w2(x,y), and when w1(x,y) is proportional to When the at least one weighting function associated with the non-angular prediction mode is proportional to The weighting function for the entire prediction block (the entire current block) is defined as follows:

[0126] Here, C1 is a constant (such as a weight value that is independent of position) that can be derived based on a cost function such as SAD or SATD. The weight w1(x,y) of the angular prediction value s1 decreases toward the right / bottom direction, such as shown in equation (12), and the weight w2(x,y) of the non-angular prediction value s2 increases toward the right / bottom direction, such as shown in equation (13). The symbol ∝ indicates this decreasing or increasing trend based on the sample position that contributes to the final weighting of the prediction values s1 and s2.

[0127] In an example, at least one angular prediction mode includes a first angular prediction mode and a second angular prediction mode, and at least one non-angular prediction mode is a non-angular prediction mode. A fusion method is used to fuse prediction signals s1, s2, and s3, where s1 and s2 are from angular prediction methods (e.g., the first angular prediction mode and the second angular prediction mode), and s3 is from a non-angular prediction method. Weighting functions w1(x, y), w2(x, y), and w3(x, y) for the entire prediction block are defined as follows:

[0128] The first weighting function w1(x,y) associated with the first angular prediction mode in the at least one weighting function is or is proportional to, as shown in equation (14). When the second weighting function w2(x,y) in the at least one weighting function associated with the second angular prediction mode is proportional to and the third weighting function w3(x,y) of the at least one weighting function associated with the non-angular prediction mode is proportional to When w1(x,y) is proportional to When w2(x,y) is proportional to is proportional to, and w3(x,y) is proportional to Here, C3, C4, and C5 are constants (such as position-independent weight values) that can be derived based on a cost function such as SAD or SATD. The weighting of the angular prediction values s1 and s2 decreases toward the right / bottom. The weighting of the non-angular prediction value s3 increases toward the right / bottom.

[0129] Figure 9 A flowchart outlining a method (900) according to an aspect of the present disclosure is shown. The method (900) can be used in an apparatus such as a video decoder. In various aspects, the method (900) is performed by a processing circuit (such as a processing circuit that performs the functions of the video decoder (110), a processing circuit that performs the functions of the video decoder (210), etc.). In some aspects, the method (900) is implemented as software instructions, so that when the processing circuit executes the software instructions, the processing circuit performs the method (900). The process starts at (S901) and proceeds to (S910).

[0130] At (S910), the method (900) includes receiving encoded information in a code stream, wherein the encoded information indicates that a current block is predicted based on a combination of multiple intra prediction modes.

[0131] At (S920), the method (900) includes determining a plurality of intra predictions for the current block based on corresponding intra prediction modes.

[0132] At (S930), the method (900) includes determining a fused prediction for the current block based on a weighted sum of a plurality of intra predictions for the current block. The weighted sum is based on respective weights associated with the plurality of intra predictions. Each of the weights may be based on one of a plurality of weighting functions that depends on a sample position (x, y) and an intra prediction mode of the intra prediction associated with the respective weight.

[0133] At (S940), the method (900) includes reconstructing the current block based on the fused prediction.

[0134] Then, the process proceeds to (S999) and terminates.

[0135] The method (900) may be adjusted as appropriate. One or more steps in the method (900) may be modified and / or omitted. One or more additional steps may be added. Any suitable order of implementation may be used.

[0136] Figure 10 A flowchart outlining a method (1000) according to an aspect of the present disclosure is shown. The method (1000) may be used in a video encoder. In various aspects, the method (1000) is performed by a processing circuit (such as a processing circuit that performs the functions of the video encoder (103), a processing circuit that performs the functions of the video encoder (303), etc. In some aspects, the method (1000) is implemented as software instructions, so that when the processing circuit executes the software instructions, the processing circuit performs the method (1000). The process starts at (S1001) and proceeds to (S1010).

[0137] At (S1010), the method (1000) includes determining a plurality of intra predictions for a current block based on a plurality of corresponding intra prediction modes.

[0138] At (S1020), the method (1000) includes determining a fused prediction for the current block based on a weighted sum of a plurality of intra predictions for the current block. The weighted sum is based on respective weights associated with the plurality of intra predictions. Each of the weights is based on one of a plurality of weighting functions that depends on a sample position (x, y) and an intra prediction mode of the intra prediction associated with the respective weight.

[0139] At (S1030), the method (1000) includes encoding the current block based on the fused prediction.

[0140] Then, the process proceeds to (S1099) and terminates.

[0141] The method (1000) may be adjusted as appropriate. One or more steps in the method (1000) may be modified and / or omitted. One or more additional steps may be added. Any suitable order of implementation may be used.

[0142] Although the decoding process and the encoding process are provided in separate flow charts for descriptive purposes, it should be noted that aspects of the decoding process and the encoding process can be used in combination. For example, a decoding process such as that described in method (900) can incorporate all or part of method (1000). In another example, an encoding process such as that described in method (1000) can be combined with method (900).

[0143] In one aspect, a method for processing visual media data is disclosed. The method includes processing a codestream of the visual media data according to a format rule. The codestream includes a syntax element indicating that a current block is predicted based on a combination of multiple intra-frame prediction modes. The format rule specifies: determining multiple intra-frame predictions for the current block based on corresponding intra-frame prediction modes; determining a fused prediction for the current block based on a weighted sum of the multiple intra-frame predictions for the current block, the weighted sum being based on corresponding weights associated with the multiple intra-frame predictions; and reconstructing the current block based on the fused prediction. Each of the weights is based on one of a plurality of weighting functions that depends on a sample position (x, y) and the intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0144] The aspects and / or examples of the present disclosure may be used individually or in any combination. For example, some aspects and / or examples performed by a decoder may be performed by an encoder, and vice versa. Each of these methods, aspects, examples, encoders, and decoders may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored in a non-volatile computer-readable medium.

[0145] The above techniques can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 11 A computer system (1100) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0146] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked, or the like to create code comprising instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or the like, or through interpretation, microcode execution, or the like.

[0147] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, and the like.

[0148] Figure 11 The components shown for the computer system (1100) are examples and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing aspects of the present disclosure. The configuration of components should not be interpreted as having any dependency or requirement on any one or combination of components shown in the example aspects of the computer system (1100).

[0149] The computer system (1100) may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0150] The input human-computer interface device may include one or more of the following (only one of each is shown): keyboard (1101), mouse (1102), touchpad (1103), touch screen (1110), data gloves (not shown), joystick (1105), microphone (1106), scanner (1107), camera (1108).

[0151] The computer system (1100) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more human user senses through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (1110), a data glove (not shown), or a joystick (1105), but there may also be tactile feedback devices that are not input devices), audio output devices (e.g., speakers (1109), headphones (not shown)), visual output devices (e.g., screens (1110), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities, some of which may output two-dimensional visual output or output of more than three dimensions through stereo output, virtual reality glasses (not shown), holographic displays, and smoke canisters (not shown)), and printers (not shown).

[0152] The computer system (1100) may also include human-accessible storage devices and their associated media, such as optical media, including CD / DVD ROM / RW (1120) (1121) with CD / DVD and other media, thumb drives (1122), removable hard drives or solid-state drives (1123), traditional magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown), etc.

[0153] Those skilled in the art will also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0154] The computer system (1100) may also include an interface (1154) to one or more communication networks (1155). For example, the network can be wireless, wired, or fiber optic. The network can also be a local area network, a wide area network, a metropolitan area network, an in-vehicle and industrial network, a real-time network, a delay-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless local area networks, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable, satellite, and terrestrial broadcast television, and vehicle and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter connected to some common data port or peripheral bus (1149) (such as a USB port of the computer system (1100)); other interfaces are typically integrated into the core of the computer system (1100) by connecting to the system bus, as described below (for example, an Ethernet interface connected to a PC computer system or a cellular network interface connected to a smartphone computer system). Using any of these networks, the computer system (1100) can communicate with other entities. This communication can be one-way, receive-only (e.g., broadcast television), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, such as to other computer systems using a local area network or wide area digital network. As mentioned above, certain protocols and protocol stacks are available for each of these networks and network interfaces.

[0155] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces may be connected to the core (1140) of the computer system (1100).

[0156] The core (1140) may include one or more central processing units (CPUs) (1141), graphics processing units (GPUs) (1142), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1143), hardware accelerators for certain tasks (1144), a graphics adapter (1150), and the like. These devices, as well as read-only memory (ROM) (1145), random access memory (1146), and internal mass storage (such as an internal non-user accessible hard drive, SSD, etc.) (1147), may be connected via a system bus (1148). In some computer systems, the system bus (1148) may be accessible in the form of one or more physical plugs to enable expansion with additional CPUs, GPUs, and the like. Peripheral devices may be connected directly to the core's system bus (1148) or via a peripheral bus (1149). In one example, a screen (1110) may be connected to a graphics adapter (1150). Peripheral bus architectures include PCI, USB, and the like.

[0157] The CPU (1141), GPU (1142), FPGA (1143), and accelerator (1144) can execute certain instructions, which, when combined, can constitute the aforementioned computer code. The computer code can be stored in ROM (1145) or RAM (1146). Transient data can also be stored in RAM (1146), while permanent data can be stored, for example, in internal mass storage (1147). By using a cache, fast storage and retrieval of any storage device can be achieved, and the cache can be closely associated with one or more CPUs (1141), GPUs (1142), mass storage (1147), ROM (1145), RAM (1146), etc.

[0158] The computer readable medium may have computer codes for executing various computer-implemented operations. The media and computer codes may be those specially designed and constructed for the purposes of the present disclosure, or they may be those well known and available to those skilled in the art of computer software.

[0159] By way of example and not limitation, a computer system having the architecture (1100), and in particular the core (1140), may provide functionality by executing software contained in one or more tangible computer-readable media by a processor (including a CPU, GPU, FPGA, accelerator, etc.). Such computer-readable media may be media associated with a user-accessible mass storage as described above, as well as certain memories of the core (1140) having non-transient properties, such as a core internal mass storage (1147) or ROM (1145). Software implementing various aspects of the present disclosure may be stored in such a device and executed by the core (1140). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software may enable the core (1140), and in particular the processor (including a CPU, GPU, FPGA, etc.) therein, to perform specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (1146) and modifying these data structures according to software-defined processes. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1144)) that may operate in place of or in conjunction with software to perform specific processes or specific portions of specific processes described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0160] As used in this disclosure, "at least one" or "an" is intended to include any one or combination of the recited elements. For example, references to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C are intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include A or B or (A and B). Where applicable, the use of "one of" does not exclude any combination of the recited elements, for example, when the elements are not mutually exclusive.

[0161] Although this disclosure describes examples of several aspects, there are modifications, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will therefore be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of this disclosure and are therefore within its spirit and scope.

[0162] The above disclosure also includes the following features. These features can be combined in various ways and are not limited to the combinations mentioned below.

[0163] (1) A method for video decoding, the method comprising: receiving encoded information in a bitstream, the encoded information indicating that a current block is predicted based on a combination of multiple intra-frame prediction modes; determining multiple intra-frame predictions of the current block based on corresponding intra-frame prediction modes; determining a fused prediction of the current block based on a weighted sum of the multiple intra-frame predictions of the current block, the weighted sum being based on corresponding weights associated with the multiple intra-frame predictions, and reconstructing the current block based on the fused prediction. Each of the weights is based on one of a plurality of weighting functions, the weighting function being dependent on a sample position (x, y) and the intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0164] (2) According to the method of feature (1), the current block is partitioned into sub-blocks; the sub-block partition information of the current block indicates the sub-block position and sub-block shape of each sub-block; a corresponding one of the plurality of weighting functions is further based on the sub-block partition information of the current block; the weighting function With the i-th intra prediction indicated by index i and the j-th subblock R indicated by index j among the subblocks j In association, the index i ranges from 1 to k, where k is the number of the plurality of intra predictions, and the index j ranges from 1 to 1, where 1 is the number of the sub-blocks.

[0165] (3) According to the method described in feature (2), the multiple intra-frame prediction modes include a horizontal angle prediction mode and a vertical angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode and s2 associated with the vertical angle prediction mode, and k is 2; the sub-block partition information of the current block indicates that the current block is partitioned into a lower left triangular sub-block R1 and an upper right triangular sub-block R2, and l is 2; the multiple weighting functions include associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and

[0166] (4) According to the method described in feature (2), the multiple intra-frame prediction modes include an angular prediction mode and a non-angular prediction mode; the multiple intra-frame predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; the multiple weighting functions include associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and

[0167] (5) According to the method described in feature (2), the multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode; the sub-block partitioning information of the current block indicates that the current block is partitioned into a left sub-block R1, a top sub-block R2, and a bottom-right sub-block R3, R1 and R2 are located to the left of R3 and above R3, respectively; the multiple weighting functions include Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 and associated with s3 in R3 Greater than and Greater than and and Greater than and

[0168] (6) According to the method described in feature (2), the multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an upper left sub-block R1, a lower left sub-block R2, an upper right sub-block R3, and a lower right sub-block R4; the multiple weighting functions include: Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 Associated with s3 in R3 Associated with s1 in R4 Associated with s2 in R4 Associated with s3 in R4 Greater than and Greater than and and Greater than and

[0169] (7) According to the method described in feature (2), the multiple intra-frame prediction modes include an angular prediction mode and a non-angular prediction mode; the multiple intra-frame predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; the multiple weighting functions include Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and or proportional to; when and When proportional, and is proportional to, and when and When proportional, and are directly proportional; and C1, C2 and C3 are constants.

[0170] (8) According to the method described in feature (2), the multiple intra-frame prediction modes include a first angular prediction mode, a second angular prediction mode, and a non-angular prediction mode; the multiple intra-frame predictions include s1 associated with the first angular prediction mode, s2 associated with the second angular prediction mode, and s3 associated with the non-angular prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; the multiple weighting functions include Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 and is a constant; and and As one of x and y decreases, and Increments with the one of x and y.

[0171] (9) According to the method described in feature (1), the multiple intra-frame prediction modes include at least one angular prediction mode and at least one non-angular prediction mode; at least one weighting function associated with the at least one angular prediction mode among the multiple weighting functions decreases toward one of the right direction and the bottom direction; and at least one weighting function associated with the at least one non-angular prediction mode among the multiple weighting functions increases toward the one of the right direction and the bottom direction.

[0172] (10) According to the method of feature (9), the at least one angular prediction mode is an angular prediction mode, and the at least one non-angular prediction mode is a non-angular prediction mode; at least one weighting function associated with the angular prediction mode is or Proportional to w1(x,y), W and H are the width and height of the current block respectively; when w1(x,y) is proportional to When the at least one weighting function associated with the non-angular prediction mode is proportional to proportional to w2(x,y); and when w1(x,y) is proportional to When the at least one weighting function associated with the non-angular prediction mode is proportional to is proportional to w2(x,y).

[0173] (11) According to the method of feature (9), the at least one angular prediction mode includes a first angular prediction mode and a second angular prediction mode, and the at least one non-angular prediction mode is a non-angular prediction mode; the first weighting function w1(x, y) associated with the first angular prediction mode in the at least one weighting function is or Proportional to, W and H are the width and height of the current block respectively; when w1(x,y) is proportional to When the second weighting function w2(x, y) associated with the second angular prediction mode in the at least one weighting function is proportional to and the third weighting function w3(x,y) associated with the non-angular prediction mode in the at least one weighting function is proportional to is proportional to; and when w1(x,y) is When w2(x,y) is proportional to is proportional to, and w3(x,y) is proportional to Directly proportional.

[0174] (12) A method of video encoding, the method comprising: determining a plurality of intra predictions for a current block based on a plurality of corresponding intra prediction modes; determining a fused prediction for the current block based on a weighted sum of the plurality of intra predictions for the current block, the weighted sum being based on corresponding weights associated with the plurality of intra predictions; and encoding the current block based on the fused prediction. Each of the weights is based on one of a plurality of weighting functions, the weighting function being dependent on a sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight.

[0175] (13) According to the method of feature (12), the current block is partitioned into sub-blocks; the sub-block partition information of the current block indicates the sub-block position and sub-block shape of each sub-block; a corresponding one of the plurality of weighting functions is further based on the sub-block partition information of the current block; and the weighting function With the i-th intra prediction indicated by index i and the j-th subblock R indicated by index j among the subblocks j In association, the index i ranges from 1 to k, where k is the number of the plurality of intra predictions, and the index j ranges from 1 to 1, where 1 is the number of the sub-blocks.

[0176] (14) According to the method described in feature (13), the multiple intra-frame prediction modes include a horizontal angle prediction mode and a vertical angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode and s2 associated with the vertical angle prediction mode, and k is 2; the sub-block partition information of the current block indicates that the current block is partitioned into a lower left triangle sub-block R1 and an upper right triangle sub-block R2, and l is 2; the multiple weighting functions include associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and

[0177] (15) According to the method described in feature (13), the multiple intra-frame prediction modes include an angular prediction mode and a non-angular prediction mode; the multiple intra-frame predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; the multiple weighting functions include associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and

[0178] (16) According to the method described in feature (13), the multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode; the sub-block partitioning information of the current block indicates that the current block is partitioned into a left sub-block R1, a top sub-block R2, and a bottom-right sub-block R3, R1 and R2 are located to the left of R3 and above R3, respectively; the multiple weighting functions include Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 and associated with s3 in R3 Greater than and Greater than and and Greater than and

[0179] (17) According to the method described in feature (13), the multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; the multiple intra-frame predictions include s1 associated with the horizontal angle prediction mode, s2 associated with the vertical angle prediction mode, and s3 associated with the non-angle prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an upper left sub-block R1, a lower left sub-block R2, an upper right sub-block R3, and a lower right sub-block R4; the multiple weighting functions include Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 Associated with s3 in R3 Associated with s1 in R4 Associated with s2 in R4 Associated with s3 in R4 Greater than and Greater than and and Greater than and

[0180] (18) According to the method described in feature (13), the multiple intra-frame prediction modes include an angular prediction mode and a non-angular prediction mode; the multiple intra-frame predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; the sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; the multiple weighting functions include associated with s1 in R1 Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and or proportional to; when and When proportional, and is proportional to, and when and When proportional, and are directly proportional; and C1, C2 and C3 are constants.

[0181] (19) According to the method described in feature (12), the multiple intra-frame prediction modes include at least one angular prediction mode and at least one non-angular prediction mode; at least one weighting function associated with the at least one angular prediction mode among the multiple weighting functions decreases toward one of the right direction and the bottom direction; and at least one weighting function associated with the at least one non-angular prediction mode among the multiple weighting functions increases toward the one of the right direction and the bottom direction.

[0182] (20) A method for processing visual media data, the method comprising: processing a code stream of the visual media data according to a format rule, wherein the code stream includes a syntax element indicating that a current block is predicted based on a combination of multiple intra-frame prediction modes; and the format rule specifies: determining multiple intra-frame predictions of the current block based on corresponding intra-frame prediction modes; determining a fused prediction of the current block based on a weighted sum of the multiple intra-frame predictions of the current block, the weighted sum being a weighting function based on corresponding weights associated with the multiple intra-frame predictions to reconstruct the current block based on the fused prediction; and each of the weights is a weighting function based on a plurality of weighting functions, the weighting function depending on a sample position (x, y) and the intra-frame prediction mode of the intra-frame prediction associated with the corresponding weight.

[0183] (21) A video decoding apparatus comprising a processing circuit configured to perform the method of any one of features (1) to (11).

[0184] (22) A video encoding apparatus comprising a processing circuit configured to perform the method of any one of features (12) to (19).

[0185] (23) A non-volatile computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of features (1) to (19).

Claims

1. A video decoding method, characterized in that: The method comprises: Receiving encoded information in a bitstream, the encoded information indicating that a current block is predicted based on a combination of multiple intra-frame prediction modes; determining a plurality of intra predictions for the current block based on corresponding intra prediction modes; determining a fused prediction for the current block based on a weighted sum of the multiple intra predictions for the current block, the weighted sum being according to respective weights associated with the multiple intra predictions, and The current block is reconstructed based on the fused prediction, wherein, Each of the weights is based on one of a plurality of weighting functions that depends on the sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight.

2. The method according to claim 1, characterized in that The current block is partitioned into sub-blocks; The sub-block partition information of the current block indicates the sub-block position and sub-block shape of each sub-block; A corresponding one of the plurality of weighting functions is further based on the sub-block partition information of the current block; and Weighting function With the i-th intra prediction indicated by index i and the j-th subblock R indicated by index j among the subblocks j In association, the index i ranges from 1 to k, where k is the number of the plurality of intra predictions, and the index j ranges from 1 to 1, where 1 is the number of the sub-blocks.

3. The method according to claim 2, characterized in that The multiple intra-frame prediction modes include a horizontal angle prediction mode and a vertical angle prediction mode; The plurality of intra predictions include s1 associated with the horizontal angular prediction mode and s2 associated with the vertical angular prediction mode, and k is 2; The sub-block partition information of the current block indicates that the current block is partitioned into a lower left triangular sub-block R1 and an upper right triangular sub-block R2, and l is 2; The plurality of weighting functions include: Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and 4. The method according to claim 2, characterized in that The multiple intra prediction modes include an angular prediction mode and a non-angular prediction mode; The plurality of intra predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; The sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; The plurality of weighting functions include: Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and 5. The method according to claim 2, characterized in that The multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; The plurality of intra predictions include s1 associated with the horizontal angular prediction mode, s2 associated with the vertical angular prediction mode, and s3 associated with the non-angular prediction mode; The sub-block partition information of the current block indicates that the current block is partitioned into a left sub-block R1, a top sub-block R2, and a bottom-right sub-block R3, where R1 and R2 are located to the left of R3 and above R3, respectively; The plurality of weighting functions include: Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 and associated with s3 in R3 Greater than and Greater than and and Greater than and 6. The method according to claim 2, characterized in that The multiple intra-frame prediction modes include a horizontal angle prediction mode, a vertical angle prediction mode, and a non-angle prediction mode; The plurality of intra predictions include s1 associated with the horizontal angular prediction mode, s2 associated with the vertical angular prediction mode, and s3 associated with the non-angular prediction mode; The sub-block partition information of the current block indicates that the current block is partitioned into an upper left sub-block R1, a lower left sub-block R2, an upper right sub-block R3, and a lower right sub-block R4; The plurality of weighting functions include: Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 Associated with s1 in R3 Associated with s2 in R3 Associated with s3 in R3 Associated with s1 in R4 Associated with s2 in R4 Associated with s3 in R4 Greater than and Greater than and and Greater than and 7. The method according to claim 2, characterized in that The multiple intra prediction modes include an angular prediction mode and a non-angular prediction mode; The plurality of intra predictions include s1 associated with the angular prediction mode and s2 associated with the non-angular prediction mode; The sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; The plurality of weighting functions include: Associated with s2 in R1 Associated with s1 in R2 and associated with s2 in R2 and or proportional to; when and When proportional, and is proportional to, and when and When proportional, and is proportional to; and C1, C2, and C3 are constants.

8. The method according to claim 2, characterized in that The multiple intra-frame prediction modes include a first angular prediction mode, a second angular prediction mode, and a non-angular prediction mode; The plurality of intra predictions include s1 associated with the first angular prediction mode, s2 associated with the second angular prediction mode, and s3 associated with the non-angular prediction mode; The sub-block partition information of the current block indicates that the current block is partitioned into an L-shaped upper left sub-block R1 and a rectangular lower right sub-block R2; The plurality of weighting functions include: Associated with s2 in R1 Associated with s3 in R1 Associated with s1 in R2 Associated with s2 in R2 Associated with s3 in R2 and is a constant; and and As one of x and y decreases, and Increments with the one of x and y.

9. The method according to claim 1, characterized in that The plurality of intra prediction modes include at least one angular prediction mode and at least one non-angular prediction mode; At least one weighting function of the plurality of weighting functions associated with the at least one angular prediction mode decreases toward one of a right direction and a bottom direction; and At least one weighting function of the plurality of weighting functions associated with the at least one non-angular prediction mode increases toward the one of the right direction and the bottom direction.

10. The method according to claim 9, characterized in that The at least one angular prediction mode is an angular prediction mode, and the at least one non-angular prediction mode is a non-angular prediction mode; At least one weighting function associated with the angular prediction mode is or Proportional to w1(x,y), W and H are the width and height of the current block respectively; When w1(x,y) and When the at least one weighting function associated with the non-angular prediction mode is proportional to is proportional to w2(x,y); and When w1(x,y) and When the at least one weighting function associated with the non-angular prediction mode is proportional to is proportional to w2(x,y).

11. The method according to claim 9, characterized in that The at least one angular prediction mode includes a first angular prediction mode and a second angular prediction mode, and the at least one non-angular prediction mode is a non-angular prediction mode; The first weighting function w1(x, y) associated with the first angular prediction mode in the at least one weighting function is or Proportional to, W and H are the width and height of the current block respectively; When w1(x,y) and When the second weighting function w2(x, y) associated with the second angular prediction mode in the at least one weighting function is proportional to and the third weighting function w3(x,y) associated with the non-angular prediction mode in the at least one weighting function is proportional to is proportional to; and When w1(x,y) and When w2(x,y) is proportional to is proportional to, and w3(x,y) is proportional to Directly proportional.

12. A video encoding method, characterized in that: The method comprises: determining a plurality of intra predictions for the current block based on a plurality of corresponding intra prediction modes; determining a fused prediction for the current block based on a weighted sum of the multiple intra predictions for the current block, the weighted sum being according to respective weights associated with the multiple intra predictions, and The current block is encoded based on the fused prediction, wherein Each of the weights is based on one of a plurality of weighting functions that depends on the sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight.

13. The method according to claim 12, characterized in that The current block is partitioned into sub-blocks; The sub-block partition information of the current block indicates the sub-block position and sub-block shape of each sub-block; A corresponding one of the plurality of weighting functions is further based on the sub-block partition information of the current block; and Weighting function With the i-th intra prediction indicated by index i and the j-th subblock R indicated by index j among the subblocks j In association, the index i ranges from 1 to k, where k is the number of the plurality of intra predictions, and the index j ranges from 1 to 1, where 1 is the number of the sub-blocks.

14. The method according to claim 12, characterized in that The plurality of intra prediction modes include at least one angular prediction mode and at least one non-angular prediction mode; At least one weighting function associated with the at least one angular prediction mode among the plurality of weighting functions decreases toward one of a right direction and a bottom direction; and At least one weighting function of the plurality of weighting functions associated with the at least one non-angular prediction mode increases toward the one of the right direction and the bottom direction.

15. A method for processing visual media data, characterized in that The method comprises: Processing the code stream of the visual media data according to the format rules, wherein The code stream includes a syntax element indicating that a current block is predicted based on a combination of multiple intra prediction modes; and The formatting rules specify: determining a plurality of intra predictions for the current block based on corresponding intra prediction modes; determining a fused prediction for the current block based on a weighted sum of the multiple intra predictions for the current block, the weighted sum being according to respective weights associated with the multiple intra predictions, reconstructing the current block based on the fused prediction; and Each of the weights is based on one of a plurality of weighting functions that depends on the sample position (x, y) and the intra prediction mode of the intra prediction associated with the corresponding weight.

Images