Derivation of intra-frame modes based on gradient-based chroma component non-adjacent reference lines at the decoding end

By using a gradient-based intra-frame mode derivation method with multiple reference lines and edge filters, the problem of low efficiency in intra-frame prediction mode derivation in existing technologies is solved, achieving more efficient video compression and decoding.

CN123125865APending Publication Date: 2026-11-17TENCENT AMERICA LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580026236.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2025-07-10
Filing Date
2025-07-11
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video coding techniques are inefficient in intra-frame prediction mode derivation, making it difficult to achieve efficient video compression and decoding.

Method used

A gradient-based intra-frame mode derivation method is adopted, which utilizes multiple reference lines and edge filters to perform intra-frame mode derivation on the reference region, thereby identifying the intra-frame prediction mode set and improving encoding and decoding efficiency.

Benefits of technology

Diversity and adaptability improve encoding and decoding efficiency, enable more accurate video block prediction, and enhance video compression performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125865A_ABST
    Figure CN123125865A_ABST
Patent Text Reader

Abstract

An exemplary video decoding method includes receiving a video bitstream comprising multiple blocks, including a current block. For each of multiple reference lines in the current block: the method includes: identifying a corresponding reference region corresponding to the reference line; and identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region. The method includes: populating a list of intra-mode and reference line combinations using a corresponding intra-prediction mode from each corresponding set of intra-prediction modes and the corresponding reference region based on the corresponding reference line; selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and decoding the current block using the intra-prediction mode.
Need to check novelty before this filing date? Find Prior Art

Description

Related applications

[0001] This application is a continuation-to-file of U.S. Patent Application No. 19 / 266,054, filed July 10, 2025, which claims priority to U.S. Provisional Patent Application No. 63 / 671,179, filed July 13, 2024, entitled “Decoder Side gradient-based Intra Mode Derivation on Non-adjacent Reference Line for Chroma Components,” and U.S. Provisional Patent Application No. 63 / 673,127, filed July 18, 2024, entitled “Intra Mode Derivation using a Simplified Edge Filter when Non-Conventional Intra Predictor is Used,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The disclosed embodiments generally relate to video encoding and decoding, including but not limited to systems and methods for edge filtering and intra-frame pattern derivation. Background Technology

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. These electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited storage resources of storage devices, video encoding can be employed to compress video data according to at least one video encoding standard before transmission or storage. Video encoding can be performed by hardware and / or software on electronic devices / client devices or on servers providing cloud services.

[0004] Video coding typically employs predictive methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that leverage the inherent redundancy in video data. Video coding aims to compress video data into a form using a lower bitrate while avoiding or minimizing video quality degradation. Several video codec standards have been developed. For example, High Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4). Multifunctional Video Coding (VVC / H.266) is a video compression standard designed to succeed HEVC. The ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2). AOMedia Video 1 (AV1) is an open video coding format designed as an alternative to HEVC. On January 8, 2019, specification version 1.0.0, along with errata 1, was released. Enhanced Compression Model (ECM) is a video coding standard currently under development. ECM aims to significantly improve compression efficiency, surpassing existing standards such as HEVC / H.265 and VVC, thereby substantially enabling higher quality video at lower bitrates. Summary of the Invention

[0005] This disclosure describes a set of methods for video (image) compression, including gradient-based intra-frame mode derivation methods. For example, multiple reference lines can be used for decoder-side gradient-based mode derivation and template cost-based intra-frame prediction. One example method involves identifying a corresponding set of intra-frame prediction modes by performing gradient-based intra-frame mode derivation on the corresponding reference region for each of the multiple reference lines in the current block. Using different intra-frame prediction modes for each reference line enables adaptability and increases diversity, thereby improving encoding and decoding efficiency.

[0006] According to some embodiments, a video decoding method includes: (i) receiving a video bitstream (e.g., an encoded video sequence) comprising multiple blocks, including a current block; (ii) for each of multiple reference lines of the current block: (a) identifying a corresponding reference region corresponding to the reference line; and (b) identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region; (iii) populating a list of intra-mode and reference line combinations using a corresponding intra-prediction mode from each corresponding set of intra-prediction modes and the corresponding reference region according to the corresponding reference line; (iv) selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and (v) decoding the current block using the intra-prediction mode.

[0007] According to some embodiments, a video coding method includes: (i) receiving video data (e.g., a source video sequence) comprising a plurality of blocks, the blocks including a current block; (ii) for each of a plurality of reference lines of the current block: (a) identifying a corresponding reference region corresponding to the reference line; and (b) identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region; (iii) populating a list of intra-mode and reference line combinations using a corresponding intra-prediction mode from each corresponding set of intra-prediction modes and the corresponding reference region of the corresponding reference line; (iv) selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and (v) encoding the current block using the intra-prediction mode.

[0008] According to some embodiments, a video decoding method includes: (i) receiving a video bitstream comprising multiple blocks (e.g., an encoded video sequence), the blocks including a current block; (ii) identifying a reference region of the current block; (iii) identifying a set of intra-prediction modes by performing gradient-based intra-prediction on the reference region, including applying an edge filter to reference samples of the reference region; (iv) selecting an intra-prediction mode from the set of intra-prediction modes; and (v) decoding the current block using the intra-prediction mode.

[0009] According to some embodiments, a computing system, such as a streaming media system, a server system, a personal computer system, or other electronic devices, is provided. The computing system includes control circuitry and a memory storing at least one set of instructions. The at least one set of instructions includes instructions for performing any of the methods described in this application. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder).

[0010] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores at least one set of instructions executable by a computing system. The at least one set of instructions includes instructions for performing any of the methods described in this application.

[0011] Therefore, apparatus and systems utilizing video encoding and decoding methods are disclosed. Such methods, apparatus, and systems can complement or replace conventional methods, apparatus, and systems used for video encoding / decoding. The features and advantages described in this specification are not necessarily comprehensive; in particular, some additional features and advantages will become more apparent to those skilled in the art from the accompanying drawings, description, and claims provided in this disclosure. Furthermore, it should be noted that the wording used in this specification has been chosen primarily for readability and guidance purposes and is not necessarily intended to define or limit the subject matter described in this application. Attached Figure Description

[0012] To gain a more detailed understanding of this disclosure, reference can be made to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the drawings only illustrate relevant features of this disclosure and are therefore not necessarily intended to be limiting, as other effective features may also be covered in this specification, as will be understood by those skilled in the art upon reading this disclosure.

[0013] Figure 1 This is a block diagram illustrating an exemplary communication system according to some embodiments.

[0014] Figure 2A This is a block diagram illustrating exemplary elements of an encoder assembly according to some embodiments.

[0015] Figure 2B This is a block diagram illustrating exemplary elements of a decoder component according to some embodiments.

[0016] Figure 3 This is a block diagram illustrating an exemplary server system according to some embodiments.

[0017] Figure 4A The diagram illustrates the use of templates to derive intra-frame prediction modes according to some embodiments.

[0018] Figure 4B An example of gradient-based intra-frame mode derivation at the decoding end according to some embodiments is shown.

[0019] Figure 4C The use of multiple reference lines is illustrated according to some embodiments.

[0020] Figure 5A This illustrates an exemplary gradient-based intra-frame mode derivation applied to both adjacent and non-adjacent templates according to some embodiments.

[0021] Figure 5B Exemplary extended templates according to some embodiments are shown.

[0022] Figure 6A Various directional intra-frame prediction modes according to some embodiments are shown.

[0023] Figure 6B-6G The diagram illustrates the construction of histograms using different window sizes according to some embodiments.

[0024] Figure 7A An exemplary video decoding process according to some embodiments is shown.

[0025] Figure 7B An exemplary video encoding process according to some embodiments is shown.

[0026] By convention, the various features shown in the accompanying drawings are not necessarily drawn to scale, and the same reference numerals may be used to denote the same features throughout the specification and the accompanying drawings. Detailed Implementation

[0027] This disclosure describes a set of methods for video (image) compression, including gradient-based intra-prediction mode derivation methods. For example, when predicting the current block using an unconventional intra-prediction mode, at least one edge filter can be used in the gradient-based intra-prediction mode derivation method to derive a regular intra-prediction mode. Improving the derivation method can increase coding efficiency because it enables more accurate predictions of video blocks. As another example, multiple reference lines can be used to construct a unified, predefined list of intra-prediction modes. For example, gradient-based intra-prediction mode derivation can be performed on each of the multiple reference lines. Constructing the mode list in this way increases list diversity, thereby improving coding efficiency, for example, by including more accurate modes to be selected from the list.

[0028] Exemplary systems and devices

[0029] Figure 1 This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) that are communicatively coupled to each other via at least one network. In some embodiments, the communication system 100 is a streaming media system, for example, for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0030] Source device 102 includes a video source 104 (e.g., a camera assembly or media storage device) and an encoder assembly 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to produce an uncompressed video sample stream). The encoder assembly 106 generates at least one encoded video stream from the video stream. The video stream from the video source 104 may have a larger data volume compared to the encoded video stream 108 generated by the encoder assembly 106. Because the encoded video stream 108 has a smaller data volume (less data) compared to the video stream from the video source, less bandwidth is required to transmit the encoded video stream 108, and less storage space is required to store the encoded video stream 108 compared to the video stream from the video source 104. In some embodiments, source device 102 does not include the encoder assembly 106 (e.g., configured to transmit uncompressed video to one or more networks 110).

[0031] At least one network 110 represents any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (connected) and / or wireless communication networks. At least one network 110 can exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.

[0032] At least one network 110 includes a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming media server (e.g., configured to store and / or distribute video content, such as an encoded video stream from source device 102). The server system 112 includes a codec component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the codec component 114 includes an encoder component and / or a decoder component. In various embodiments, the codec component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the codec component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video stream 108. In some embodiments, the server system 112 serves as a Media-Aware Network Element (MANE). For example, server system 112 can be configured to prune the encoded video stream 108 to customize a potentially different stream for at least one electronic device 120. In some embodiments, MANE is set up separately from server system 112.

[0033] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an output video stream that can be displayed on a display or other type of display device. In some embodiments, at least one electronic device 120 does not include a display component (e.g., it is communicatively coupled to an external display device and / or includes a media storage device). In some embodiments, electronic device 120 is a streaming media client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116. In some embodiments, source device 102 and / or at least one electronic device 120 are examples of: server system, personal computer, portable device (e.g., smartphone, tablet, or laptop), wearable device, video conferencing equipment, and / or other types of electronic devices.

[0034] In an exemplary operation of the communication system 100, a source device 102 transmits an encoded video stream 108 to a server system 112. For example, the source device 102 may encode an image stream acquired by the source device. The server system 112 receives the encoded video stream 108 and may decode and / or encode the encoded video stream 108 using a codec component 114. For example, the server system 112 may apply encoding more suitable for network transmission and / or storage to the video data. The server system 112 may transmit encoded video data 116 (e.g., at least one encoded video stream) to at least one electronic device 120. Each electronic device 120 may decode the encoded video data 116 and optionally display the video image.

[0035] Figure 2AThis is a block diagram illustrating exemplary elements of an encoder assembly 106 according to some embodiments. The encoder assembly 106 receives video data (e.g., a source video sequence) from a video source 104. In some embodiments, the encoder assembly includes a receiver (e.g., transceiver) assembly configured to receive the source video sequence. In some embodiments, the encoder assembly 106 receives the video sequence from a remote video source (e.g., a component of another device different from the encoder assembly 106). The video source 104 can provide a source video sequence in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 YCrCB or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously acquired / prepared video. In some embodiments, the video source 104 is a camera that acquires local image information as a video sequence. The video data can be provided as multiple independent images that, when viewed sequentially, present motion effects. An image itself can be organized as a spatial array of pixels, where each pixel may include at least one sample, depending on the sampling structure, color space, etc., used. Those skilled in the art can readily understand the relationship between pixels and samples.

[0036] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under other time constraints, as required by the application. In some embodiments, encoder component 106 is configured to perform a conversion between the source video sequence and a visual media data stream (e.g., a video stream). Enforcing an appropriate encoding rate is one of the functions of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units as described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer and / or lambda value of rate-distortion optimization techniques), image size, GOP layout, maximum motion vector search range, etc. Other functions of controller 204 can be readily identified by those skilled in the art, as these other functions may involve encoder component 106 being optimized for a specific system design.

[0037] In some embodiments, encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for generating symbols, such as a symbol stream, based on an input image to be encoded and one or more reference images) and a (local) decoder 210. Decoder 210 reconstructs the symbols in a manner similar to that of the (remote) decoder to produce sample data (when the compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to reference image memory 208. Since decoding the symbol stream produces a bit-precise result independent of the decoder's location (local or remote), the contents of reference image memory 208 are also bit-precise between the local encoder and the remote encoder. In this way, the sample values ​​interpreted by the encoder's prediction portion as reference image samples are the same as the sample values ​​interpreted by the decoder when using the prediction during decoding.

[0038] The operation of decoder 210 can be the same as that of a remote decoder, such as decoder component 122, as described below. Figure 2B It will be described in detail. However, a brief reference is provided. Figure 2B When symbols are available and the entropy encoder 214 and the parser 254 are able to encode / decode the symbols into an encoded / decoded video sequence without loss, the entropy decoding portion of the decoder component 122, including the buffer memory 252 and the parser 254, may not be fully implemented in the local decoder 210.

[0039] Aside from parsing / entropy decoding, the decoder techniques described in this application can exist in the corresponding encoders with substantially the same functional form. Therefore, the disclosed subject matter focuses on decoder operation. Furthermore, the description of the encoder techniques can be brief, as these encoder techniques may be the inverse process of the decoder techniques.

[0040] As part of the operation of the source encoder 202, the source encoder 202 can perform motion-compensated predictive coding, referencing at least one previously encoded frame in the video sequence designated as a reference frame, to predictively encode the input frame. In this manner, the encoding engine 212 encodes the differences between pixel blocks of an input frame and pixel blocks of one or more reference frames, wherein the one or more reference frames can be selected as one or more predictive references for the input frame. The controller 204 can manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding video data.

[0041] Decoder 210 decodes encoded video data of frames that can be designated as reference frames based on symbols generated by source encoder 202. The operation of encoding engine 212 can advantageously be a lossy process. When in video decoder ( Figure 2A When decoding encoded video data at a location (not shown), the reconstructed video sequence can be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process, which can be performed on the reference frame by a remote video decoder, and the reconstructed reference frame can be stored in reference image memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that shares common content with the reconstructed reference frame obtained by the remote video decoder (in the absence of transmission errors).

[0042] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., which can be used as an appropriate prediction reference for the new image. Predictor 206 can operate pixel-by-pixel based on the sample blocks to find an appropriate prediction reference. Based on the search results obtained by predictor 206, the input image can have prediction references taken from multiple reference images stored in reference image memory 208.

[0043] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 214. The entropy encoder 214 converts these symbols into an encoded video sequence by lossless compression of the symbols generated by the various functional units according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0044] In some embodiments, the output of entropy encoder 214 is coupled to a transmitter. The transmitter can be configured to buffer one or more encoded video sequences generated by entropy encoder 214 in preparation for transmission via communication channel 218, which may be a hardware / software link to a storage device for storing the encoded video data. The transmitter can be configured to combine encoded video data from source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (data sources not shown). In some embodiments, the transmitter may transmit additional data along with the encoded video. Source encoder 202 may include such data as part of the encoded video sequence. Additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant images and slices, Supplementary Enhancement Information (SEI) messages, fragments of Visual Usability Information (VUI) parameter sets, etc.

[0045] Controller 204 can manage the operation of encoder component 106. During encoding, controller 204 can assign a specific type of encoded image to each encoded image, which may affect the encoding technique applied to the corresponding image. For example, an image can be assigned as an intra-frame image (I-image), a predictive image (P-image), or a bidirectional predictive image (B-image). Intra-frame images can be encoded and decoded without using any other frames in the sequence as prediction sources. Some video codecs support different types of intra-frame images, including, for example, Independent Decoder Refresh (IDR) images. These variations of I-images and their corresponding applications and characteristics are well known to those skilled in the art and will not be repeated here. Predictive images can be encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using at most one motion vector and reference index. Bidirectional predictive images can be encoded and decoded by predicting sample values ​​for each block via intra-frame prediction or via inter-frame prediction using at most two motion vectors and reference indices. Similarly, multiple predictive images can reconstruct a single block using more than two reference images and associated metadata.

[0046] The source image can typically be spatially divided into multiple sample blocks (e.g., each block contains 4×4, 8×8, 4×8, or 16×16 samples) and encoded block by block. Blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the block. For example, blocks of an I-image can be unpredictably coded, or these blocks can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be unpredictably coded, encoded via spatial prediction, or encoded via temporal prediction with reference to a previously coded reference image. Blocks of a B-image can be unpredictably coded, encoded via spatial prediction, or encoded via temporal prediction with reference to one or two previously coded reference images.

[0047] Video can be captured as multiple source images (video images) in a time-series manner. Intra-frame image prediction (often simply called intra-prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, a specific image in encoding / decoding (called the current image) is divided into multiple blocks. When a block in the current image is similar to a reference block in a previously encoded reference image in the video that is still being buffered, that block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image and, in the case of using multiple reference images, can have a third dimension that identifies that reference image.

[0048] Encoder component 106 can perform encoding operations according to a predetermined video coding technique or standard, which is any technique or standard such as those described in this application. In operation, encoder component 106 can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0049] Figure 2B This is a block diagram illustrating exemplary elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to the display 124 (e.g., via a wired or wireless connection).

[0050] In some embodiments, decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive at least one encoded video sequence to be decoded by decoder component 122. In some embodiments, decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence may be received from channel 218, which may be a hardware / software link to a storage device storing the encoded video data. The receiver may receive the encoded video data along with other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective user entities (not depicted). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by decoder component 122 to decode the data and / or more accurately reconstruct the original video data. Additional data can take the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0051] According to some embodiments, decoder component 122 includes buffer memory 252, parser 254 (sometimes also called entropy decoder), scaler / inverse transform unit 258, intra-frame image prediction unit 262, motion compensation prediction unit 260, aggregator 268, loop filter unit 256, reference image memory 266, and current image memory 264. In some embodiments, decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. Decoder component 122 may be implemented at least partially in software.

[0052] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., for combating network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 inside decoder component 122 (e.g., buffer memory configured to handle playback timing), a separate buffer memory is also provided outside decoder component 122 (e.g., for combating network jitter). Buffer memory 252 may not be necessary or may be small when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isosynchronous network. Buffer memory 252 may be required for use on best-effort packet networks such as the Internet. Buffer memory 252 may be relatively large and / or have an adaptive size and may be at least partially implemented in the operating system or in similar components outside decoder component 122.

[0053] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. These symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling display devices such as display 124. Control information for one or more display devices may be in the form of, for example, SEI messages or VUI parameter set fragments (not depicted). Parser 254 parses (entropy decodes) the encoded video sequence. Encoding of the encoded video sequence may be performed according to video coding techniques or standards and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subgroup parameter set from the encoded video sequence for at least one pixel subgroup in the video decoder based on at least one parameter corresponding to a group. Subgroups may include GOPs, images, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. Parser 254 may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0054] Depending on the type of the encoded video image or its portions (such as inter-frame and intra-frame images, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how these units are involved can be controlled by subgroup control information parsed from the encoded video sequence by parser 254. For clarity, the flow of such subgroup control information between parser 254 and the multiple units described below is not depicted.

[0055] The decoder component 122 can conceptually be divided into multiple functional units, and in some embodiments, these units interact closely with each other and can be at least partially integrated with each other. However, for clarity, the conceptual division of functional units is maintained in this application.

[0056] The scaler / inverse transform unit 258 receives from the parser 254 quantized transform coefficients as one or more symbols 270, along with control information (such as which transform to use, block size, quantization factor, and / or quantization scaling matrix). The scaler / inverse transform unit 258 can output blocks comprising sample values ​​that can be input to the aggregator 268. In some cases, the output samples of the scaler / inverse transform unit 258 relate to intra-coded blocks; that is, the blocks do not use predictive information from previously reconstructed images, but instead can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-image prediction unit 262. The intra-image prediction unit 262 can use surrounding reconstructed information obtained from the current (partially reconstructed) image in the current image memory 264 to generate blocks of the same size and shape as the blocks in the reconstruction. The aggregator 268 can add the predictive information generated by the intra-image prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258 on a per-sample basis.

[0057] In other cases, the output samples of the scaler / inverse transform unit 258 may relate to inter-frame coded and potentially motion-compensated blocks. In such cases, the motion compensation prediction unit 260 can access the reference image memory 266 to obtain samples for prediction. After motion compensation of the obtained samples according to the block-related symbols 270, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (referred to in this case as residual samples or residual signals) to generate output sample information. The address within the reference image memory 266 for the motion compensation prediction unit 260 to obtain predicted samples can be controlled by motion vectors. Motion vectors can be used by the motion compensation prediction unit 260 in the form of symbols 270, which may have, for example, X, Y, and reference image components. Motion compensation may also include, for example, interpolation of sample values ​​obtained from the reference image memory 266 when using subsampled precise motion vectors, and motion vector prediction mechanisms.

[0058] The output samples of aggregator 268 can be used in loop filter unit 256 for various loop filtering techniques. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream and provided to loop filter unit 256 from parser 254 in the form of symbol 270, but may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values. The output of loop filter unit 256 may be a sample stream that can be output to a display device such as display 124 and stored in reference image memory 266 for future inter-frame image prediction.

[0059] Once a certain encoded image is reconstructed, it can be used as a reference image for future predictions. Once an encoded image is reconstructed and has been identified as a reference image (e.g., by the parser 254), the current reference image can become part of the reference image memory 266, and a new current image memory can be reallocated before reconstructing subsequent encoded images begins.

[0060] Decoder component 122 can perform decoding operations according to predetermined video compression techniques that may be described in standards such as any standards described in this application. If the encoded video sequence follows the syntax of a video compression technique or standard, such as the syntax specified in a video compression technique document or standard, particularly in its configuration file document, then the encoded video sequence can conform to the syntax specified by the video compression technique or standard used. Furthermore, in order to conform to some video compression techniques or standards, the complexity of the encoded video sequence can be within the range defined by the hierarchy of the video compression technique or standard. The hierarchy can define the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, in megasamples per second), maximum reference image size, etc. The limitations set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the HRD buffer management metadata indicated by signaling in the encoded video sequence.

[0061] Figure 3This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes control circuitry 302, at least one network interface 304, memory 314, a user interface 306, and at least one communication bus 312 for interconnecting these components. In some embodiments, the control circuitry 302 includes at least one processor (e.g., CPU, GPU, and / or DPU). In some embodiments, the control circuitry includes one or more field-programmable gate arrays, multiple hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0062] One or more network interfaces 304 can be configured to connect to at least one communication network (e.g., a wireless network, a wired network, and / or an optical network). The communication network can be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicular and industrial network, a real-time network, a latency-tolerant network, etc. Examples of communication networks include: LANs such as Ethernet; wireless LANs; cellular networks including GSM, 3G, 4G, 5G, LTE, etc.; wired or wireless wide area digital television networks, including cable TV, satellite TV, and terrestrial broadcast TV; vehicular and industrial networks, including CANBus, etc. Such communication can be one-way receiving (e.g., broadcast TV), one-way transmitting (e.g., CANBus to certain CANBus devices), or bidirectional (e.g., connecting to other computer systems using a LAN or WAN digital network). Such communication can include communication with at least one cloud computing network.

[0063] User interface 306 includes at least one output device 308 and / or at least one input device 310. One or more input devices 310 may include at least one of the following: keyboard, mouse, touchpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. One or more output devices 308 may include at least one of the following: audio output device (e.g., speaker), visual output device (e.g., display or monitor), etc.

[0064] Memory 314 may include high-speed random access memory (such as DRAM, SRAM, DDR RAM, and / or other random access solid-state storage devices) and / or non-volatile memory (such as at least one disk storage device, optical disk storage device, flash memory device, and / or other non-volatile solid-state storage devices). Memory 314 may optionally include at least one storage device located remotely from control circuitry 302. Memory 314, or alternatively, one or more non-volatile solid-state storage devices within memory 314, include non-transitory computer-readable storage media. In some embodiments, memory 314 or the non-transitory computer-readable storage media of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof:

[0065] ● Operating system 316, which includes programs for handling various basic system services and for performing hardware-dependent tasks;

[0066] ● Network communication module 318, which is used to connect server system 112 to other computing devices via at least one network interface 304 (e.g., via wired and / or wireless connection);

[0067] ● Encoding / decoding module 320, which performs various functions related to encoding and / or decoding data such as video data. In some embodiments, encoding / decoding module 320 is an instance of codec component 114. Encoding / decoding module 320 includes, but is not limited to, at least one of the following:

[0068] ○ Decoding module 322, which performs various functions related to decoding encoded data, such as those previously described for decoder component 122; and

[0069] ○ Encoding module 340, which performs various functions related to encoding data, such as those previously described for encoder component 106; and

[0070] ● Image memory 352 is used to store images and image data, for example, for use by the encoding / decoding module 320. In some embodiments, image memory 352 includes at least one of the following: reference image memory 208, buffer memory 252, current image memory 264, and reference image memory 266.

[0071] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described for the parser 254), a transformation module 326 (e.g., configured to perform various functions previously described for the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform various functions previously described for the motion compensation prediction unit 260 and / or the intra-frame image prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described for the loop filter 256).

[0072] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described for the source encoder 202 and / or encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described for the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes... Figure 3 A subset of the modules shown. For example, the shared prediction module is shared by both the decoding module 322 and the encoding module 340.

[0073] The modules identified above and stored in memory 314 correspond to instruction sets for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules; therefore, in various embodiments, subsets of these modules may be combined or otherwise rearranged. For example, the codec module 320 may optionally not include separate decoding and encoding modules, but instead use the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the modules and data structures identified above. In some embodiments, memory 314 stores additional modules and data structures not described above.

[0074] although Figure 3 A server system 112 according to some embodiments is shown, but... Figure 3 This is intended more as a functional description of various features that may exist in at least one server system, rather than a structural schematic diagram of the embodiments described in this application. In practice, items shown individually may be combined, and some items may be separated. For example, Figure 3 Some of the items shown separately can be implemented on a single server, while a single item can be implemented by at least one server. The actual number of servers used to implement server system 112 and how features are distributed among these servers will vary depending on the specific implementation and, optionally, in part, on the amount of data traffic processed by the server system during peak and average usage periods.

[0075] Exemplary codec techniques

[0076] The encoding / decoding processes and techniques described below can be performed by the aforementioned devices and systems (e.g., source device 102, server system 112, and / or electronic device 120). This disclosure covers decoding-side intra-prediction mode derivation, for example, using template cost-based intra-prediction employing multiple reference lines. The decoding-side gradient-based mode derivation process may include populating a combined intra-prediction mode list by selecting a corresponding intra-prediction mode from each respective intra-prediction mode set. Using different intra-prediction modes for each reference line enables adaptability and increases diversity, thereby improving encoding / decoding efficiency.

[0077] Several methods for deriving intra-prediction modes at the decoding end are described. For example, to reduce signaling overhead, template cost-based intra-prediction mode derivation and gradient-based intra-prediction mode derivation at the decoding end are employed. Template cost-based intra-prediction leverages the fact that if the neighboring samples of the current block have good correlation, the prediction mode that is effective for the neighboring samples is likely to be effective for the current block as well.

[0078] Exemplary intra-mode derivation in template cost-based intra-prediction includes: such as Figure 4A As shown, sample group 406 is defined as the reference for template 404, and sample group 406 is used as reference samples to generate the prediction signal of template 404. Predictions for template 404 are generated using intra-frame prediction modes. The Sum of Absolute Transformed Differences (SATD) cost between the predicted signal and the reconstructed signal of template 404 is calculated. For each mode in the predefined intra-frame prediction mode set, the prediction of template 404 and the cost calculation between the predicted signal and the reconstructed signal are repeated, and the predefined intra-frame modes are ranked based on the SATD costs of these modes. This template cost-based intra-frame prediction is sometimes referred to as "Method A" below. For the current block 402, the mode with the lowest SATD cost is selected as the prediction mode, or the primary template-based intra-frame mode. For the current block 402, the mode with the second lowest SATD cost is selected as the secondary prediction mode, or the secondary template-based intra-frame mode. Calculate the SATD cost for non-angular modes (e.g., planar mode and DC mode) to select the non-angular mode with the lowest SATD cost as the third prediction mode (e.g., non-angular prediction mode). Based on the SATD difference between the primary, secondary, and third non-angular intra-modes, obtain the final prediction value by: (i) fusing the prediction values ​​from the primary, secondary, and third non-angular intra-modes; (ii) fusing the prediction values ​​from the primary intra-mode and the secondary intra-mode; or (iii) using only the primary intra-mode without fusing with other intra-modes.

[0079] Gradient-based intra-frame pattern derivation is a statistical method that uses the neighboring samples of the current block to generate a gradient histogram, sometimes referred to as "Method B" below. Based on the histogram, the top N gradients are mapped to intra-frame patterns, and these predictions are combined to generate the final prediction. Furthermore, the derived intra-frame predicted patterns can be included in a list of most likely pattern candidates. Figure 4B An example of gradient-based intra-frame pattern derivation at the decoder is shown, considering three rows of neighboring samples (sometimes referred to as template 432 below). The gradient histogram contains only samples within the middle reference line 434, as shown... Figure 4B The shaded sample is indicated by this. Two additional neighboring lines in template 432 (e.g., the upper and lower reference lines) are used to calculate the gradient of the sample in the intermediate reference line 434.

[0080] In some methods, multi-reference line prediction is combined with template cost-based intra-frame prediction, such that the selected reference lines are not limited to samples adjacent to the current block 452 (e.g., from reference line 0, or such as...). Figure 4CInstead of line 460 shown, one can choose a non-adjacent reference line that has a non-zero vertical offset and / or distance from the current block 452 (e.g., reference line 0, reference line 1, reference line 2, reference line 3, corresponding to lines 458, 456, and 454, respectively). This method of combining multi-reference line prediction with template cost-based intra-frame prediction is sometimes referred to as "Method C" below. Figure 4C Samples from four different reference lines 454, 456, 458, and 460 corresponding to CU are shown. Optionally, more reference lines (e.g., seven or other numbers) or fewer reference lines can be used. In some methods, for each reference line in a predefined list of reference lines, for intra modes in a predefined list of intra modes, template cost-based intra prediction is performed to compute SATD costs for each entry (e.g., reference 2, mode 2) of a specific reference line and a specific intra mode, and the list of entries is sorted based on the SATD costs. Subsequently, one of the top S entries in the sorted list is selected as the intra mode of the current block 452. Table 1 provides an example where the list initially contains 50 entries. After sorting based on the costs generated by template cost-based intra prediction, a selected number of entries are retained (e.g., setting S to 20 or a different value).

[0081]

[0082] Table 1 - List of reference lines and intra-frame modes based on cost sorting

[0083] We now turn to an exemplary encoding and decoding approach using multiple reference lines for gradient-based pattern derivation and template cost-based intra-frame prediction at the decoding end. Figure 5A The calculation of a prediction block according to some embodiments is illustrated. Figure 5A In the examples, in example 502 where the reference line index is zero, gradient-based pattern derivation is applied at the decoder end. In examples 504, 506, and 508, the reference index is not zero. In example 502, gradient-based pattern derivation at the decoder end is applied to the template 510 adjacent to the current block 518, and the corresponding histogram 526 (or histogram 0) is derived. Similarly, the reference line indices of examples 504, 506, and 508 are respectively equal to 1, 2, and 3 to generate the corresponding histograms 528, 530, and 532 (e.g., histograms 528, 530, and 532). Figure 1 Histogram 2, Histogram Figure 3 ).

[0084] As described in this application, in some embodiments, for the chroma components of the current block, the decoding end derives a gradient-based mode (e.g., as...). Figure 4BAs shown, or method B) is applied to template cost-based intra-frame prediction using multiple reference lines (e.g., as shown). Figure 4A and 4C As shown, or method C). In some methods, the same set of intra-modes from a predefined intra-mode list is used across different reference lines (e.g., the predefined intra-mode list is consistent across different reference lines). In contrast, in the methods and systems described in this application, the decoder derives additional intra-modes for each reference line based on gradient-based mode derivation. These additional intra-modes may differ across different reference lines, thereby diversifying the intra-modes in the intra-mode list and improving coding efficiency.

[0085] In some embodiments, the length of the unsorted list of reference lines and intra-mode combinations (e.g., sometimes referred to as "list C" below) is L = M × R, where M corresponds to the length of the predefined list of intra-mode combinations (e.g., sometimes referred to as "list A" below), and R corresponds to the length of the predefined list of allowed reference lines (e.g., sometimes referred to as "list B" below). After sorting the reference lines and intra-mode combination lists based on template costs, the first S entries in the sorted list are selected for use.

[0086] In some methods, intra-frame prediction utilizes the spatial redundancy between the current block and its neighboring samples. Traditionally, intra-frame prediction modes can be categorized into directional and non-directional modes, indicating the directional or non-directional correlation between the current block and neighboring reference blocks. Examples of non-directional intra-frame prediction modes include planar mode and DC mode.

[0087] In some systems, the intra-frame prediction mode is explicitly indicated by signaling, and the corresponding prediction signal is generated by applying an interpolation filter to a reference sample. For example... Figure 6A As shown, intra-frame modes include directional prediction modes and non-directional prediction modes. Directional prediction modes include, for example, D45_PRED, V_PRED, H_PRED, and D203_PRED. Non-directional prediction modes include planar mode and DC mode. Based on the directional or non-directional texture features of the current block, mode-specific intra-frame prediction values ​​for the entire coded block are generated.

[0088] In some embodiments, when an unconventional intra-prediction mode is applied to predict the current block, at least one simplified edge filter is used in a gradient-based intra-prediction method to derive a conventional intra-prediction mode. In some embodiments, the derived intra-prediction mode is used to determine the intra-prediction mode used in other coding stages. Unconventional intra-prediction modes include, for example, predictions generated by matrix multiplication, extrapolation models, intra-block copies pointed to by block vectors, or fused predictions formed using at least two predictions, each corresponding to a conventional intra-prediction mode. Other coding stages include, for example, predicting chrominance components based on co-located luma components using a direct mode, constructing a list of the most likely modes for mode propagation, and / or selecting a transform kernel.

[0089] In some methods, unconventional intra-prediction patterns can be generated via matrix-based multiplication. This multiplication is performed using reference samples of the current coding block and some trained coefficients stored in a matrix. Intra-predictions can also be generated via intra-block copying, where the block vector points to a reference region within a predefined search area. Intra-predictions can also be generated via an extrapolation model, where predicted samples are extrapolated from neighboring reference samples. In this mode, intra-predictions are generated sample-by-sample according to a predefined generation order. Newly generated samples can be used as new inputs to the extrapolation model to generate subsequent samples, with the predefined generation order traversing the entire coding block. Such unconventional intra-prediction patterns can complement prediction methods in encoding and decoding when the current coding block does not exhibit the obvious directional or planar texture defined by conventional intra-prediction methods.

[0090] Intra-prediction modes are used not only in the prediction phase to define intra-predicted values ​​but also in other coding phases to determine relevant encoding / decoding information. For example, multiple primary (or secondary) transform kernel sets are selected based on regular intra-prediction modes. In another example, when predicting chroma components from luma components, a direct mode can be used to generate chroma intra-predicted values, which reuses regular intra-prediction mode information for the co-located luma components. As another example, for intra-prediction mode propagation, a Most Probable Mode (MPM) list is constructed using regular intra-prediction mode information from neighboring coded blocks. The intra-prediction mode of the current block uses one of the modes in the MPM to generate the prediction signal.

[0091] When using unconventional intra-prediction modes to predict coded blocks, it may be difficult to define conventional intra-prediction modes for coding stages such as transform kernel selection, direct mode application (for chroma), or intra-mode propagation.

[0092] In some embodiments, gradient-based intra-mode derivation at the decoder is used to determine regular intra-mode information for some unconventional intra-prediction modes. The gradient-based intra-mode derivation method generates a gradient histogram using prediction samples from the current block. Based on this histogram, the highest gradient is mapped to a regular intra-mode.

[0093] like Figure 6B As shown, the gradient is calculated based on a 3×3 sample window. A horizontal Sobel filter (Fi) is applied to this 3×3 window for one or more considered prediction samples. x ) and vertical Sobel filter (F y ), to derive its horizontal gradient (G) respectively. hor ) and vertical gradient (G ver Sobel filter F for edge detection x and F y The definition is as follows:

[0094]

[0095] Subsequently, the angular gradient descriptor D of the sample is calculated based on the following formula. s :

[0096]

[0097] Calculated D s The descriptor maps the direction θ to the closest corresponding regular intra-frame mode. To construct the histogram, such as... Figure 6C As shown, window 602 traverses the circled positions of sample 608 in the current block 606 in raster scan order. Since the window filter shape of window 602 is 3×3, the width and height of the sample 608 used to construct the histogram are considered to be (W-2) and (H-2), respectively. W and H are the original width and height of the current block 606. For example, in Figure 6B-6G In the given information, W=8 and H=4.

[0098] In some embodiments, a simplified horizontal edge filter and simplified vertical edge filter When applied to a 2x2 window, its definition is as follows:

[0099]

[0100] In some embodiments, the position of one or more selected samples 620 in the 2×2 window 622 corresponds to the top-left sample, such as Figure 6D As shown. Accordingly, consider a sample 626 used for the histogram with a width of (W-1) and a height of (H-1), where W and H are the original width and original height of the current block 624, as follows. Figure 6EAs shown.

[0101] In some embodiments, the positions of one or more samples 630 considered in the 2×2 window 622 correspond to other positions, such as the lower right corner position, as shown below. Figure 6F As shown. Accordingly, consider a sample 632 used for the histogram with a width of (W-1) and a height of (H-1), where W and H are the original width and original height of the current block 624, as follows. Figure 6G As shown.

[0102] In some embodiments, at least one simplified edge filter is adaptively applied based on other coding information. For example, a simplified edge filter is applied when the size of the coded block is less than a threshold (e.g., 64 samples). Otherwise, the original Sobel filter is applied. In some embodiments, the edge filter is adaptively applied based on the prediction mode of the current block. For example, this is to increase the coverage of samples considered for determining the histogram in the current block. In some embodiments, different edge filters are used for inter-frame modes and intra-frame modes. In some embodiments, intra-frame mode information is generated for the current block that is inter-coded, and the generated intra-frame mode information is used for other coding purposes or other coding stages.

[0103] In some embodiments, the application of at least one simplified edge filter in the bitstream is explicitly indicated by signaling at the image, sub-image, slice, CTU, or CU level.

[0104] In some embodiments, the samples considered for constructing the histogram are adaptively determined based on other encoding information. In some embodiments, a threshold is determined based on the size of the current coding block. When the number of samples considered is less than the threshold, reference is made to... Figure 6C-6F The described 2x2 window traverses the current block. Otherwise, a histogram is constructed using the number of samples considered when a threshold is reached. For example, histogram construction stops when the threshold is reached.

[0105] Figure 7A This is a flowchart illustrating a method 700 for decoding video according to some embodiments. Method 700 may be executed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and a memory storing instructions for execution by the control circuitry. In some embodiments, method 700 is executed by executing instructions stored in the computing system's memory (e.g., memory 314).

[0106] The system receives (702) a video bitstream (e.g., an encoded video sequence) comprising multiple blocks (e.g., corresponding to at least one frame), including the current block. For each of the multiple reference lines of the current block: the system identifies (704) a corresponding reference region corresponding to the reference line; and the system identifies (706) a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region. The system populates (708) a list of intra-mode and reference line combinations using the corresponding intra-prediction modes from each corresponding set of intra-prediction modes and the corresponding reference region according to the corresponding reference line. The system selects (710) an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations. The system decodes (712) the current block using the intra-prediction mode. Thus, when predicting the current block using an unconventional intra-prediction mode, at least one edge filter is used in the gradient-based intra-mode derivation method to derive a conventional intra-prediction mode.

[0107] In some embodiments, in template cost-based intra-prediction for gradient-based mode derivation at the decoding end (e.g., method C), a unified predefined intra-mode list used across different reference lines is extended with an additional N intra-modes. These N intra-modes correspond to the first N intra-modes derived for each reference line from the gradient-based mode derivation method at the decoding end. The length of the extended intra-mode list is M+N entries, and the length L' of the reference line and intra-mode combination list is equal to L'=(M+N)×R. The extended N intra-modes are derived for each reference line and may differ for each reference line. In some embodiments, list A includes M=8 intra-modes. In some embodiments, there are N=5 extended intra-modes.

[0108] In some embodiments, the quantity S remains constant; for example, the number of allowed combinations remains constant after the sorting process. In some embodiments, the methods and systems described in this application can increase the diversity of the intra-frame mode list even when using the same number of combinations. In some embodiments, the number of allowed combinations S is increased.

[0109] In some embodiments, the permissible reference lines derived using gradient-based pattern derivation at the decoder are {1,3,5}, where the numbers indicate the reference line indices, and therefore R=3.

[0110] In some embodiments, the method of uniformly constructing a predefined intra-frame mode list across different reference lines in method C is modified to include N intra-frame modes. These N intra-frame modes correspond to the first N intra-frame modes derived from the gradient-based mode derivation method at the decoding end for each reference line. In some embodiments, the length of the predefined intra-frame mode list remains unchanged, for example, having M (M>N) entries.

[0111] In some embodiments, the size of the template for gradient-based intra mode derivation (e.g., Figure 4A L1 and L2 in) is different from the size shown in Figure 4A (e.g., the aspect ratio between L1 and L2 is different, or they are different in other aspects). In some embodiments, the template of the current block (e.g., the left template 554 and / or the top template 556) extends to twice the width W and twice the height H of the current block 552, as shown in Figure 5B . Furthermore, in some embodiments, the sizes of the left template 554 and the top template 556 are different (e.g., T1 is not equal to T2). This approach allows the top template 556 and the left template 554 to be different, thereby providing flexibility and enabling better capture of the texture of samples.

[0112] In some embodiments, a flag is signaled in the bitstream to indicate the usage of the disclosed methods. In some embodiments, an index i (i<S) is also signaled to indicate which entry in the sorted list is being used.

[0113] In some embodiments, the SATD cost used for sorting can be changed to other metrics, such as SAD, Mean Removal (MR)-SAD or other metrics.

[0114] Figure 7B is a flowchart illustrating a video encoding method 750 according to some embodiments. The method 750 may be executed at a computing system (e.g., the server system 112, the source device 102 or the electronic device 120) that has a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method 750 is executed by executing instructions stored in the memory (e.g., the memory 314) of the computing system. In some embodiments, the method 750 is executed by the same system as the above-described method 700.

[0115] The system receives (752) video data (e.g., a source video sequence) comprising multiple blocks, including the current block. For each of the multiple reference lines of the current block: the system identifies (754) a corresponding reference region corresponding to the reference line, and the system identifies (756) a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region. The system populates (758) a list of intra-mode and reference line combinations using the corresponding intra-prediction modes from each corresponding set of intra-prediction modes and the corresponding reference region according to the corresponding reference line. The system selects (760) an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations. The system encodes (762) the current block using the intra-prediction mode. In some embodiments, the system encodes the current block using information from at least one syntax element. In some embodiments, the system transmits at least one encoded syntax element and the encoded current block via a video stream. As previously described, the encoding process may correspond to the decoding process described in this application (e.g., gradient-based mode derivation and template cost-based intra-prediction at the decoding end using multiple reference lines). For the sake of brevity, these details will not be repeated here.

[0116] although Figure 7A and 7B Multiple logical stages are shown in a specific order, but these stages are independent of the order and can be reordered, and other stages can be combined or split. Some reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the orderings and groupings presented in this application are not exhaustive. Furthermore, it should be understood that the stages can be implemented in hardware, firmware, software, or any combination thereof.

[0117] (A1) In one aspect, some embodiments include a video decoding method (e.g., method 700). In some embodiments, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). In some embodiments, the method is performed at a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video stream comprising multiple blocks, including the current block; (ii) for each of the multiple reference lines in the current block: identifying a corresponding reference region corresponding to the reference line; and identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region; (iii) populating a list of intra-mode and reference line combinations using the corresponding intra-prediction modes from each corresponding set of intra-prediction modes and the corresponding reference region of the corresponding reference line; (iv) selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and (v) decoding the current block using the intra-prediction mode. Thus, a unified predefined list of intra-modes across different reference lines expands by N intra-modes. These N intra-modes can correspond to the first N intra-modes derived from the gradient-based mode derivation method at the decoding end for each reference line. The length of the expanded list of intra-modes can be M+N entries, where M indicates the length of the predefined list of intra-modes. This makes the length of the list of reference line and intra-mode combinations equal to L' = (M+N)×R. Since the extended N is derived for each reference line, the extended N may be different for each reference line.

[0118] (A2) In some embodiments of A1, the current block is a chroma block. In some embodiments, the combination of multiple reference lines and gradient-based intra-frame mode derivation is applied only to the chroma block. In some embodiments, the combination is applied to one or more luma blocks (e.g., in addition to at least one chroma block).

[0119] (A3) In some embodiments of A1 or A2, the intra-mode and reference line combination list also includes at least one intra-predictive mode identified without performing any gradient-based intra-mode derivation. For example, the combination list may include M=8 intra-modes.

[0120] (A4) In some embodiments of any of A1 to A3, selecting a corresponding intra-prediction mode from each corresponding intra-prediction mode set includes selecting five intra-prediction modes. For example, N=5 intra-prediction modes are added to a combination list.

[0121] (A5) In some embodiments of any of A1 to A4, the method further includes: sorting each set of corresponding intra-prediction modes before selecting the corresponding intra-prediction mode. For example, the number S of entries in the combination list remains unchanged, for example, the allowed combinations remain unchanged after sorting. As another example, the number S of allowed combinations is increased.

[0122] (A6) In some embodiments of A5, ranking is performed based on association cost. For example, the cost may be SATD cost. However, the SATD cost used for ranking can be changed to other metrics, such as SAD, MR-SAD, etc.

[0123] (A7) In some embodiments of any of A1 to A6, the plurality of reference lines consist of three reference lines.

[0124] (A8) In some embodiments of A7, the three reference lines are not adjacent. For example, the permissible reference lines derived using gradient-based pattern derivation at the decoder end are {1,3,5}, where the numbers indicate the reference line indices, so R=3.

[0125] (A9) In some embodiments of any of A1 to A8, the list of intra-mode and reference line combinations has a predefined size. For example, the predefined intra-mode list can be constructed uniformly across different reference lines to include N intra-modes. These N intra-modes correspond to the first N intra-modes derived from a gradient-based mode derivation method at the decoding end for each reference line. The length of the predefined intra-mode list remains constant, i.e., it has M (M>N) entries.

[0126] (A10) In some embodiments of any of A1 to A9, the two or more corresponding reference regions corresponding to the two or more reference lines have different shapes. For example, the template size may be different from... Figure 4B The dimensions shown are different.

[0127] (A11) In some embodiments of any of A1 to A10, the size of the corresponding reference region is twice the size of the corresponding size of the current block. For example, as Figure 5B As shown, the template of the current block can be expanded to twice the width and twice the height of the current block. In some embodiments, the upper portion of the template has different dimensions than the left portion of the template. For example, the dimensions of the left template and the upper template may be different (e.g., T1 != T2).

[0128] (A12) In some embodiments of any of A1 to A10, the method further includes parsing a flag from the video bitstream indicating whether gradient-based intra-frame mode derivation is performed for each of the plurality of reference lines. For example, whether to use the above method is determined based on a flag indicated by signaling in the bitstream.

[0129] (A13) In some embodiments of A12, the method further comprises parsing an index from the video bitstream, wherein the index indicates the size of the combination list of intra modes and reference lines. For example, signaling is used to indicate an index i (i<S) to indicate which entries in the sorted list are to be used.

[0130] (A14) In some embodiments of any one of A1 to A13, performing gradient-based intra mode derivation comprises applying an edge filter to samples of a reference region. For example, when an unconventional intra prediction mode is applied to predict the current block, at least one simplified edge filter is used in the gradient-based intra mode derivation method to derive a conventional intra prediction mode. In some embodiments, whether to apply the edge filter is indicated by signaling in the video bitstream. For example, in the bitstream, the application of the at least one simplified edge filter is explicitly signaled at the image, sub-image, slice, CTU and / or CU level. In some embodiments, the edge filter is a horizontal edge filter or a vertical edge filter.

[0131] (A15) In some embodiments of A14, the window of the edge filter is determined based on coding information. For example, the samples considered for constructing the histogram are adaptively determined based on other coding information. As an example, the threshold is determined based on the size of the current coding block. When the number of considered samples is less than the threshold, a 2×2 window as described above traverses the current block; otherwise, the considered samples up to the threshold are used to construct the histogram.

[0132] (A16) In some embodiments of A14 or A15, the edge filter is applied when an unconventional intra prediction mode is used for the current block. The unconventional intra prediction mode includes but is not limited to a prediction value generated by matrix multiplication, an extrapolation model, intra block copy pointed to by a block vector, or a fused prediction value using at least two prediction values, each corresponding to a conventional intra mode.

[0133] (A17) In some embodiments of any one of A14 to A16, the edge filter is selectively applied based on coding information. For example, at least one simplified edge filter is adaptively applied based on other coding information. In some embodiments, the type of the edge filter is selected based on the coding information. For example, for a larger block, a Sobel filter may be applied, while for a smaller block, a simplified edge filter may be applied.

[0134] (A18) In some embodiments of A17, the encoding information includes at least one of block size and prediction mode. For example, when the encoded block size is less than a threshold (e.g., 64 samples), a simplified edge filter is applied. Otherwise, the original Sobel filter is applied. As an example, the proposed edge filter is adaptively applied based on the prediction mode.

[0135] (B1) In another aspect, some embodiments include a video coding method (e.g., method 750). In some embodiments, the method is performed at a computing system having memory and at least one processor. The method includes: (i) receiving video data comprising a plurality of blocks, the blocks including a current block; (ii) for each of a plurality of reference lines of the current block: identifying a corresponding reference region corresponding to the reference line; and identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region; (iii) populating a list of intra-mode and reference line combinations using a corresponding intra-prediction mode from each corresponding set of intra-prediction modes and according to the corresponding reference region of the corresponding reference line; (iv) selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and (v) encoding the current block using the intra-prediction mode.

[0136] (B2) In some embodiments of B1, the method further includes transmitting a video stream that includes the encoded current block.

[0137] (B3) In some embodiments of B1 and B2, the method also includes coded end features similar to any of the features described above for A1 to A18.

[0138] (C1) In one aspect, some embodiments include a video decoding method. In some embodiments, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). In some embodiments, the method is performed at a source coding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video bitstream comprising a plurality of blocks, the blocks including a current block; (ii) identifying a reference region of the current block; (iii) identifying a set of intra-prediction modes by performing gradient-based intra-mode derivation on the reference region, including applying an edge filter to reference samples of the reference region; (iv) selecting an intra-prediction mode from the set of intra-prediction modes; and (v) decoding the current block using the intra-prediction mode. Thus, when an unconventional intra-prediction mode is applied to predict the current block, at least one simplified edge filter is used in the gradient-based intra-mode derivation method to derive a conventional intra-prediction mode. The derived intra-prediction modes can be used to determine the intra-modes required for other coding stages. These other coding stages include, but are not limited to: predicting chrominance components based on co-located luma components using direct modes, constructing a list of the most likely modes for mode propagation, or selecting a transform kernel.

[0139] (D1) In one aspect, some embodiments include a method for generating a video media stream. In some embodiments, the method is performed at a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed at an encoding / decoding module (e.g., encoding / decoding module 320). In some embodiments, the method is performed at a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) generating a video stream, including: for each of a plurality of reference lines in the current block: (a) identifying a corresponding reference region corresponding to the reference line; and (b) identifying a corresponding set of intra-prediction modes by performing gradient-based intra-mode derivation on the corresponding reference region; (c) populating a list of intra-mode and reference line combinations using the corresponding intra-prediction modes from each corresponding set of intra-prediction modes and the corresponding reference region according to the corresponding reference line; (d) selecting an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and (e) encoding the current block using the intra-prediction modes; and (ii) transmitting a video stream including the encoded current block.

[0140] On the other hand, some embodiments include a computing system (e.g., server system 112) that includes control circuitry (e.g., control circuitry 302) and a memory (e.g., memory 314) coupled to the control circuitry, the memory storing at least one set of instructions configured to be executed by the control circuitry, the at least one set of instructions including instructions for performing any of the methods described in this application (e.g., methods 700 and 750, and A1 to A18, B1 to B3, C1 and D1 above).

[0141] In another aspect, some embodiments include a non-transitory computer-readable storage medium that stores at least one set of instructions executable by control circuitry of a computing system, the at least one set of instructions including instructions for performing any of the methods described in this application (e.g., methods 700 and 750, and A1 to A18, B1 to B3, C1, and D1 above). In some embodiments, the memory or non-transitory computer-readable storage medium stores a video stream that includes any features disclosed in this application (e.g., syntax and encoded information).

[0142] Unless otherwise stated, any syntax element described in this application can be an HLS. As used in this application, HLS is indicated by signaling at a level higher than the block level. For example, HLS can correspond to the sequence level, frame level, slice level, or tile level. As another example, HLS elements can be indicated by signaling in a video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptation parameter set (APS), slice header, picture header, tile header, and / or CTU header.

[0143] It should be understood that although the terms “first,” “second,” etc., may be used to describe various elements in this application, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and covers any and all possible combinations of at least one associated listed item. It should also be further understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of at least one other feature, integer, step, operation, element, component, and / or combination thereof.

[0144] As used in this application, the term "when" may be interpreted, depending on the context, as meaning "if" the stated prerequisite is true, or "when" the stated prerequisite is true, or "in response to determination" the stated prerequisite is true, or "according to determination" the stated prerequisite is true, or "in response to detection" the stated prerequisite is true. Similarly, the phrases "if it is determined [the stated prerequisite is true]" or "if [the stated prerequisite is true]" or "when [the stated prerequisite is true]" may be interpreted, depending on the context, as meaning "when" the stated prerequisite is true, or "in response to determination" the stated prerequisite is true, or "according to determination" the stated prerequisite is true, or "when" the stated prerequisite is true, or "in response to detection" the stated prerequisite is true.

[0145] For illustrative purposes, the foregoing description has been illustrated with reference to specific embodiments. However, the exemplary arguments presented above are not intended to be exhaustive, nor are they intended to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the foregoing teachings. These embodiments were chosen and described to best explain the principles of operation and practical application, thereby enabling those skilled in the art to implement them.

Claims

1. A video decoding method executed at a computing system, said computing system having a memory and at least one processor, characterized in that, The method includes: Receive a video stream comprising multiple blocks, including the current block; For each of the multiple reference lines in the current block: Identify the corresponding reference region corresponding to the reference line; and By performing gradient-based intra-frame mode derivation on the corresponding reference regions, the corresponding intra-frame prediction mode set is identified. The list of intra-mode and reference line combinations is populated using the corresponding intra-mode from each corresponding intra-mode set and the corresponding reference region based on the corresponding reference line. Select an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and The current block is decoded using the intra-frame prediction mode.

2. The method according to claim 1, characterized in that, The current block is a chroma block.

3. The method according to claim 1, characterized in that, The list of intra-mode and reference line combinations also includes at least one intra-prediction mode identified without performing any gradient-based intra-mode derivation.

4. The method according to claim 1, characterized in that, Selecting the corresponding intra-prediction mode from each corresponding intra-prediction mode set includes selecting five intra-prediction modes.

5. The method according to claim 1, characterized in that, Also includes: Before selecting the corresponding intra-prediction mode, each corresponding intra-prediction mode set is sorted.

6. The method according to claim 5, characterized in that, The sorting is performed based on association costs.

7. The method according to claim 1, characterized in that, The multiple reference lines consist of three reference lines.

8. The method according to claim 7, characterized in that, The three reference lines are not adjacent.

9. The method according to claim 1, characterized in that, The list of intra-frame modes and reference lines has a predefined size.

10. The method according to claim 1, characterized in that, Two or more reference lines correspond to two or more corresponding reference regions with different shapes.

11. The method according to claim 1, characterized in that, The size of the corresponding reference area is twice the size of the corresponding size of the current block.

12. The method according to claim 1, characterized in that, Also includes: Parse a flag from the video stream indicating whether the gradient-based intra-frame mode derivation is performed for each of the plurality of reference lines.

13. The method according to claim 12, characterized in that, Also includes: An index is parsed from the video stream, the index indicating the size of the list of intra-frame mode and reference line combinations.

14. The method according to claim 1, characterized in that, Performing the gradient-based intra-frame mode derivation involves applying an edge filter to samples of the reference region.

15. The method according to claim 14, characterized in that, The window of the edge filter is determined based on the encoded information.

16. The method according to claim 14, characterized in that, The edge filter is applied when using an unconventional intra-frame prediction mode on the current block.

17. The method according to claim 14, characterized in that, The edge filter is applied selectively based on encoded information.

18. The method according to claim 17, characterized in that, The encoding information includes at least one of the block size and prediction mode.

19. A video encoding method executed at a computing system having a memory and at least one processor, characterized in that, The method includes: Receive video data comprising multiple blocks, including the current block; For each of the multiple reference lines in the current block: Identify the corresponding reference region corresponding to the reference line; and By performing gradient-based intra-frame mode derivation on the corresponding reference regions, the corresponding intra-frame prediction mode set is identified. The list of intra-mode and reference line combinations is populated using the corresponding intra-mode from each corresponding intra-mode set and the corresponding reference region based on the corresponding reference line. Select an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and The current block is encoded using the intra-frame prediction mode.

20. A non-transitory computer-readable storage medium storing a video stream generated by a video coding method, characterized in that, The video encoding method includes: For each of the multiple reference lines in the current block: Identify the corresponding reference region corresponding to the reference line; and By performing gradient-based intra-frame mode derivation on the corresponding reference regions, the corresponding intra-frame prediction mode set is identified. The list of intra-mode and reference line combinations is populated by using the corresponding intra-prediction modes from each corresponding intra-prediction mode set and the corresponding reference regions according to the corresponding reference lines. Select an intra-prediction mode and a reference line from the list of intra-mode and reference line combinations; and The current block is encoded using the intra-frame prediction mode; and The video stream includes the encoded current block.