Selective transform set in video coding

Abstract

A video bitstream is received, the video bitstream including coded information for a current block in a current picture. The coded information includes a plurality of prediction samples for the current block. The prediction samples are obtained by matrix multiplication of a coefficient matrix and a plurality of reference samples for the current block. A transform set for the current block is determined based on which of a subset of intra prediction modes is applied to the current block when the one of the subset of intra prediction modes is applied to the current block. The subset of intra prediction modes is derived based on the prediction samples for the current block. The current block is reconstructed based on the prediction samples and the transform set.

Description

By incorporating via reference

[0001] This application claims the benefit of U.S. Patent Application No. 19 / 175,818, filed April 10, 2025, entitled “SELECTIVE TRANSFORM SET IN VIDEO CODING”, which claims the benefit of U.S. Provisional Application No. 63 / 633,563, filed April 12, 2024, entitled “METHOD AND APPARATUS FOR SELECTIVE TRANSFORMSET BASED ON DECODER-SIDE INTRA MODE DERIVATION”, and U.S. Provisional Application No. 63 / 679,458, filed August 5, 2024, entitled “MULTIPLE TRANSFORM SET WITH HISTORY-BASED ADAPTIVEOFFSET”. The entire disclosure of the earlier applications is incorporated herein by reference. Technical Field

[0002] This disclosure describes aspects generally related to video encoding and decoding. Background Technology

[0003] The background description provided herein is for the purpose of presenting the overall context of this disclosure. To the extent that the work described in this background section is intended, neither the work of the currently identified inventors nor any aspect of the description which at the time of filing would otherwise be considered prior art is expressly or implicitly acknowledged as prior art to this disclosure.

[0004] Image / video compression can help transmit image / video data across different devices, storage devices, and networks with minimal quality degradation. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. For instance, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current image being reconstructed for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can utilize motion compensation to predict samples in the current image based on previously reconstructed images. Motion compensation can be indicated by a motion vector (MV). Summary of the Invention

[0005] Various aspects of this disclosure include bitstreams, methods, and apparatus for video encoding / decoding. In some examples, the apparatus for video encoding / decoding includes a processing circuitry system.

[0006] According to one aspect of this disclosure, a video decoding method is provided. In this method, a video bitstream is received, the video bitstream including encoded information of a current block in a current image. The encoded information includes multiple prediction samples of the current block. The prediction samples are obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block. When an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, a transform set for the current block is determined based on which intra-prediction mode from the subset is applied to the current block. The intra-prediction mode subset is derived based on the prediction samples of the current block. The current block is reconstructed based on the prediction samples and the transform set.

[0007] According to another aspect of this disclosure, a video coding method is provided. In this method, an intra-prediction mode subset is derived based on multiple prediction samples of a current block in a current image. The prediction samples are obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block. When an intra-prediction mode from the intra-prediction mode subset is applied to the current block, a transform set for the current block is determined based on which intra-prediction mode subset is applied to the current block. The intra-prediction mode subset is derived based on the prediction samples of the current block. The prediction samples of the current block are encoded into a bitstream based on the transform set.

[0008] According to another aspect of this disclosure, a method for processing visual media data is provided. In this method, a bitstream of visual media data is processed according to format rules. The bitstream includes encoded information of a current block in a current image. The encoded information includes multiple prediction samples of the current block. The prediction samples are obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block. The format rules specify that when an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode subset is applied to the current block. The intra-prediction mode subset is derived based on the prediction samples of the current block. The format rules specify that the current block is processed based on the prediction samples and the transform set.

[0009] Aspects of the present invention also provide an apparatus for video decoding. This apparatus for video decoding includes a processing circuitry system configured to implement any of the described methods for video decoding.

[0010] This disclosure also provides an apparatus for video encoding. The apparatus for video encoding includes a processing circuitry system configured to implement any of the described methods for video encoding.

[0011] This disclosure also provides a non-transitory computer-readable medium for storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for video decoding / encoding.

[0012] The technical solutions disclosed herein include methods and apparatus for determining a transform set based on: (i) a decoder-side intra-mode derivation method using matrix multiplication-based modes and / or (ii) a history-based offset search list for updating an offset search list based on previously encoded offsets. In an example, a video bitstream is received, comprising encoded information of a current block in a current image. The encoded information comprises multiple prediction samples of the current block. The prediction samples are obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block. When an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which of the intra-prediction mode subsets is applied to the current block. The intra-prediction mode subset is derived based on the prediction samples of the current block. The current block is reconstructed based on the prediction samples and the transform set.

[0013] In another example, an offset relative to the initial index is determined from the candidate list to obtain an updated index. The initial index indicates the initial transform set. The candidate list includes multiple candidate offsets obtained from the offset transformation table. The transform set is determined based on the updated index.

[0014] In one aspect of this disclosure, a decoder-side intra-frame mode derivation method based on matrix multiplication-based modes improves encoding / decoding quality. In another aspect of this disclosure, a transform set is determined based on a history-based offset search list used to update the offset search list based on previously encoded offsets, thereby saving signaling overhead. Attached Figure Description

[0015] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0016] Figure 1 This is a schematic diagram of an example block diagram of a communication system (100).

[0017] Figure 2 This is a schematic diagram of an example block diagram of a decoder.

[0018] Figure 3 This is a schematic diagram of an example block diagram of an encoder.

[0019] Figure 4 This is a schematic diagram illustrating an example of prediction using reference samples based on matrix multiplication.

[0020] Figure 5This is a schematic diagram illustrating an example of offset (state) updates during encoding.

[0021] Figure 6 This is a schematic diagram of an example of an offset (state) transition table during encoding based on a histogram.

[0022] Figure 7 A flowchart outlining some aspects of the decoding process according to this disclosure is shown.

[0023] Figure 8 A flowchart is shown that provides an overview of the coding process according to some aspects of this disclosure.

[0024] Figure 9 It is a schematic diagram of a computer system based on one aspect. Detailed Implementation

[0025] Figure 1 Block diagrams of some example video processing systems (100) are shown. The video processing system (100) is an example of the application of the disclosed subject matter—video encoders and video decoders—in a streaming environment. The disclosed subject matter can be equivalently applied to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs (Compact Discs), DVDs (Digital Versatile Discs), memory sticks, etc.

[0026] The video processing system (100) includes a capture subsystem (113) that may include a video source (101), such as a digital camera device, that creates, for example, an uncompressed video picture stream (102). In the example, the video picture stream (102) includes samples captured by the digital camera device. The video picture stream (102) is depicted as a thick line to emphasize the high data volume compared to encoded video data (104) (or encoded video bitstream), which may be processed by an electronic device (120) including a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination thereof to implement or realize aspects of the disclosed subject matter as described in more detail below. The encoded video data (104) (or encoded video bitstream) is depicted as a thin line to emphasize its lower data volume compared to the video picture stream (102). The encoded video data (104) (or encoded video bitstream) can be stored on a streaming server (105) for future use. One or more streaming client subsystems, for example... Figure 1Client subsystems (106) and (108) can access a streaming server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of encoded video data and creates an outgoing video picture stream (111) capable of being rendered on a display (112) (e.g., a display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (104), (107), and (109) (e.g., a video bitstream) may be encoded according to certain video codec / compression standards. Examples of such standards include ITU-T Recommendation H.265. In this example, the video codec standard under development is informally referred to as Versatile Video Coding (VVC). The disclosed topics can be used in the context of VVC.

[0027] Note that electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may include a video decoder (not shown), and electronic device (130) may also include a video encoder (not shown).

[0028] Figure 2 An example block diagram of a video decoder (210) is shown. The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuitry system). The video decoder (210) may be used instead of... Figure 1 The video decoder (110) in the example.

[0029] The receiver (231) can receive one or more encoded video sequences, such as those included in a bitstream, to be decoded by the video decoder (210). In one aspect, one encoded video sequence is received at a time, where the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences can be received from a channel (201), which can be a hardware / software link to a storage device storing the encoded video data. The receiver (231) can receive the encoded video data along with other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). The receiver (231) can separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (215) can be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as the "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory can be external to the video decoder (210) (not depicted). In other applications, a buffer memory (not depicted) may exist outside the video decoder (210) to prevent network jitter, for example, and another buffer memory (215) may exist inside the video decoder (210) to handle broadcast timing, for example. The buffer memory (215) may not be necessary, or it may be small, when the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability, or from an isochronous synchronization network. For use in best-effort packet networks such as the Internet, the buffer memory (215) may be required. The buffer memory (215) may be relatively large and advantageously have an adaptive size, and may be implemented at least partially in the operating system or in a similar element (not depicted) outside the video decoder (210).

[0030] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (210), and possibly information for controlling rendering devices such as rendering devices (212) (e.g., a display screen), which are not part of the electronic device (230) but may be coupled to it. Figure 2As shown. Control information for the rendering apparatus can be in the form of Supplemental Enhancement Information (SEI) messages or Video Usability Information (VUI) parameter set fragments (not depicted). The parser (220) can parse / entropy decode the received encoded video sequence. The encoding and decoding of the encoded video sequence can be performed according to video encoding and decoding technologies or video encoding and decoding standards, and can follow various principles, including variable-length encoding and decoding, Huffman coding, arithmetic encoding and decoding with or without context sensitivity, etc. The parser (220) can extract a subgroup parameter set of at least one subgroup of subgroups of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. Subgroups can include Groups of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The parser (220) can also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded and decoded video sequence.

[0031] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0032] Depending on the type of the encoded video picture or its portions (e.g., inter-frame and intra-frame pictures, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) can involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the encoded video sequence by the parser (220). For clarity, such subgroup control information flow between the parser (220) and the following multiple units is not depicted.

[0033] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into multiple functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated into one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the following functional units.

[0034] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives quantization transform coefficients as symbols (221) and control information from the parser (220), including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output a block containing sample values, which can be input to the aggregator (255).

[0035] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to intra-frame encoded blocks. Intra-frame encoded blocks do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-frame image prediction unit (252). In some cases, the intra-frame image prediction unit (252) uses surrounding reconstructed information obtained from the current image buffer (258) to generate blocks of the same size and shape as the blocks being reconstructed. For example, the current image buffer (258) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (255) adds the predictive information already generated by the intra-frame prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) based on each sample.

[0036] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to inter-frame encoding / decoding and may be motion-compensated blocks. In such cases, the motion-compensated prediction unit (253) can access the reference image memory (257) to obtain samples for prediction. After motion compensation of the obtained samples according to the symbols (221) belonging to the block, these samples can be added to the output of the scaler / inverse transform unit (251) by the aggregator (255) (in this case, referred to as residual samples or residual signals) to generate output sample information. The address at which the motion-compensated prediction unit (253) in the reference image memory (257) obtains the predicted samples can be controlled by motion vectors, which are available to the motion-compensated prediction unit (253) in the form of symbols (221) that may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​obtained from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0037] The output samples of the aggregator (255) can be subjected to various loop filtering techniques in the loop filter unit (256). The video compression technique may include in-loop filtering techniques controlled by parameters included as symbol (221) from the parser (220) in the encoded video sequence (also referred to as the encoded video bitstream) and available to the loop filter unit (256). The video compression may also respond to metadata obtained during decoding of a previous portion of the encoded image or encoded video sequence (in decoding order), as well as to previously reconstructed and loop-filtered sample values.

[0038] The output of the loop filter unit (256) can be a sample stream, which can be output to the rendering device (212) and stored in the reference image memory (257) for use in future inter-frame image prediction.

[0039] Once a certain encoded image is fully reconstructed, it can be used as a reference image for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and that encoded image (by, for example, the parser (220)) has been identified as a reference image, the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0040] The video decoder (210) can perform decoding operations according to standards such as ITU-T H.265 Recommendation or predetermined video compression techniques. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and in the sense that the profile documented in the video compression technique or standard is available. Specifically, the profile may select certain tools from all available tools in the video compression technique or standard as tools that are only available under that profile. For compliance, the complexity of the encoded video sequence may also be required to be within the range limited by the level of the video compression technique or standard. In some cases, the level limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, in millions of samples per second), maximum reference image size, etc. In some cases, the limitations set by the level can be further restricted by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is signaled in the encoded video sequence.

[0041] On one hand, the receiver (231) can receive supplementary (redundant) data along with the encoded video. The supplementary data can be included as part of the encoded video sequence. The supplementary data can be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The supplementary data can be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0042] Figure 3 An example block diagram of a video encoder (303) is shown. The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmission circuit system). The video encoder (303) can be used instead of Figure 1 The video encoder (103) in the example.

[0043] The video encoder (303) can obtain data from the video source (301) (which is not...). Figure 3 In one example, a portion of the electronic device (320) receives video samples, and a video source (301) can capture video images to be encoded by a video encoder (303). In another example, the video source (301) is a portion of the electronic device (320).

[0044] The video source (301) can provide a sequence of source video in the form of a digital video sample stream to be encoded by the video encoder (303). The digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601 YCrCb, RGB…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera device capturing local image information as a video sequence. Video data can be provided as multiple individual pictures that convey motion when viewed sequentially. The pictures themselves can be organized as a spatial array of pixels, where each pixel can include one or more samples, depending on the sampling structure, color space, etc. The following description focuses on samples.

[0045] According to one aspect, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or under any other required time constraints. Implementing an appropriate encoding / decoding speed is a function of the controller (350). In some aspects, the controller (350) controls and is functionally coupled to other functional units as described below. For simplicity, the coupling is not depicted. Parameters set by the controller (350) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of images (GOP) layout, maximum motion vector search range, etc. The controller (350) can be configured to have other suitable functions belonging to the video encoder (303) optimized for a particular system design.

[0046] In some respects, the video encoder (303) is configured to operate within an encoding / decoding loop. As a hypersimplified description, in this example, the encoding / decoding loop may include a source codec (330) (e.g., responsible for creating symbols such as a symbol stream based on the input image to be encoded and a reference image) and a (local) decoder (333) embedded within the video encoder (303). The decoder (333) reconstructs the symbols, creating sample data in a manner similar to that created by the (remote) decoder. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since decoding of the symbol stream results in bit-exact results regardless of the decoder's location (local or remote), the contents of the reference image memory (334) are also bit-exact between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same sample values ​​that the decoder will "see" when using prediction during decoding. This basic principle of reference image synchronicity (and the drift that occurs, for example, due to channel errors) is also used in some related techniques.

[0047] The operation of the "local" decoder (333) can be combined with that of the "remote" decoder, as already mentioned above. Figure 2 The video decoder (210) described in detail is the same. However, a brief reference is also made to... Figure 2 Since symbols are available and the encoding of symbols into a decoded video sequence by the entropy codec (345) and the decoding of symbols by the parser (220) can be lossless, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), can be fully implemented in the local decoder (333).

[0048] On the one hand, decoder techniques other than parsing / entropy decoding present in the decoder exist in the corresponding encoder with the same or substantially the same functional form. Therefore, the subject matter disclosed focuses on decoder operation. Since encoder techniques are the opposite of decoder techniques described in detail, the description of encoder techniques can be simplified. More detailed descriptions are provided in certain places below.

[0049] During operation, in some examples, the source codec (330) may perform motion-compensated predictive coding, which predictively codes the input image with reference to one or more previously codec images designated as "reference images" from the video sequence. In this way, the codec engine (332) codes and decodes the differences between pixel blocks of the input image and pixel blocks of the reference image that can be selected as predictive references for the input image.

[0050] The local video decoder (333) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source codec (330). The operation of the codec engine (332) can be advantageously for lossy processing. When the encoded video data can be decoded by the video decoder (330), Figure 3 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process performed on the reference image by the video decoder and can store the reconstructed reference image in the reference image memory (334). In this way, the video encoder (303) can locally store a copy of the reconstructed reference image that shares the same content (without transmission errors) as the reconstructed reference image that will be obtained by the remote video decoder.

[0051] The predictor (335) can perform a prediction search against the encoding / decoding engine (332). That is, for a new image to be encoded / decoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. The predictor (335) can operate pixel-by-pixel based on the sample blocks to find appropriate prediction references. In some cases, as determined by the search results obtained by the predictor (335), the input image may have prediction references extracted from multiple reference images stored in the reference image memory (334).

[0052] The controller (350) can manage the encoding and decoding operations of the source codec (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0053] The outputs of all the functional units mentioned above can undergo entropy encoding and decoding in the entropy codec (345). The entropy codec (345) converts the symbols generated by the various functional units into encoded video sequences by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding and decoding.

[0054] The transmitter (340) can buffer the encoded video sequence created by the entropy codec (345) in preparation for transmission via a communication channel (360), which can be a hardware / software link to a storage device storing the encoded video data. The transmitter (340) can combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0055] The controller (350) can manage the operation of the video encoder (303). During encoding and decoding, the controller (350) can assign a specific encoded image type to each encoded image, which may affect the encoding and decoding techniques that can be applied to the corresponding image. For example, images can typically be assigned to one of the following image types:

[0056] Intra-frame images (I-images) can be encoded and decoded without using any other images in the sequence as a prediction source. Some video codecs allow different types of intra-frame images, including, for example, Independent Decoder Refresh (IDR) images.

[0057] Predictive images (P-images) can be encoded and decoded using intra-frame or inter-frame prediction that uses motion vectors and reference indices to predict sample values ​​for each block.

[0058] Bidirectional predictive images (B-images) can be encoded and decoded using intra-frame or inter-frame prediction that uses two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can be used for the reconstruction of a single block using more than two reference images and associated metadata.

[0059] Source images can typically be spatially subdivided into multiple sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and encoded / decoded on a block-by-block basis. These blocks can be predictively encoded / decoded with reference to other (already encoded / decoded) blocks, determined by the encoding / decoding assignments of the corresponding images applied to the block. For example, blocks of image I can be nonpredictively encoded / decoded, or blocks of image I can be predictively encoded / decoded (spatial prediction or intra-frame prediction) with reference to already encoded / decoded blocks of the same image. Pixel blocks of image P can be predictively encoded / decoded with reference to a previously encoded / decoded reference image via spatial prediction or temporal prediction. Blocks of image B can be predictively encoded / decoded with reference to one or two previously encoded / decoded reference images via spatial prediction or temporal prediction.

[0060] The video encoder (303) can perform encoding operations according to a predetermined video coding / decoding technology or standard, such as ITU-T H.265 Recommendation. In its operation, the video encoder (303) can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded / decoded video data can conform to the syntax specified by the video coding / decoding technology or standard in use.

[0061] On one hand, the transmitter (340) can transmit additional data along with the encoded video. The source codec (330) can include such data as part of the encoded video sequence. The additional data can include temporal / spatial / SNR enhancement layers, other forms of redundant data, such as redundant images and slices, SEI messages, VUI parameter set fragments, etc.

[0062] Video can be captured as multiple source images (video pictures) in a time-series manner. Intra-frame picture prediction (often simply called intra-prediction) utilizes spatial correlations within a given image, while inter-frame picture prediction utilizes (temporal or other) correlations between images. In the example, a specific image during encoding / decoding—referred to as the current image—is segmented into blocks. Where a block in the current image is similar to a reference block in a previously encoded and decoded reference image still being buffered in the video, the block in the current image can be encoded and decoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference images.

[0063] In some aspects, bidirectional prediction techniques can be used for inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly in the past and future in display order). A block in the current image can be encoded and decoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. The block can be predicted using a combination of the first and second reference blocks.

[0064] In addition, merging mode techniques can be used for inter-frame image prediction to improve encoding and decoding efficiency.

[0065] According to some aspects of this disclosure, prediction is performed on a block-by-block basis, such as inter-frame picture prediction and intra-frame picture prediction. For example, according to the HEVC (High Efficiency Video Coding) standard, pictures in a video picture sequence are segmented into Coding Tree Units (CTUs) for compression. The CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU consists of three Coding Tree Blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively partitioned into one or more Coding Units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be partitioned into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In the example, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Based on temporal and / or spatial predictability, a CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one aspect, prediction operations in encoding / decoding are performed on a unit basis of prediction blocks. Using a luma prediction block as an example, a prediction block consists of a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0066] Note that the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using any suitable technology. On the one hand, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. On the other hand, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors that execute software instructions.

[0067] Various aspects of this disclosure provide techniques for improving selective transform sets based on intra-frame prediction modes and / or determining multiple transform sets using history-based adaptive offsets.

[0068] Video codecs are widely used in many applications. Various video codec standards, such as H.264, H.265, H.266 (VVC), AV1, and AVS, have been widely adopted. A video codec can include several modules, such as intra / inter-frame prediction, transform coding / decoding, quantization, entropy coding / decoding, and in-loop filtering. Aspects of this disclosure include a set of methods for video compression, such as template matching prediction.

[0069] Intra-frame prediction explores the spatial redundancy between the current block and its neighboring samples. Regular intra-frame prediction modes can be classified into directional and non-directional modes, indicating directional or non-directional correlation between the current block and neighboring reference blocks. Examples of non-directional intra-frame prediction modes can include planar mode and DC mode.

[0070] In the first conventional method, the intra-prediction mode can be explicitly notified by a signal, and the corresponding prediction signal can be generated using an interpolation filter applied to the reference sample.

[0071] In the second method, for some of the intra-prediction modes, the intra-prediction signal can be generated by using reference samples multiplied by coefficients (e.g., weighting coefficients). The weighting coefficients can be trained offline and stored as matrices. Figure 4 An example of training-based predictions is shown. For example... Figure 4As shown, the current prediction block (402) includes a reference sample (404) at the top and a reference sample (406) at the left. W and H represent the width and height of the current prediction block (402), respectively. In the example, the reference sample (404) at the top may have a width of twice the size, and the reference sample (406) at the left may have a height of twice the size. In the example, the reference sample (404) may have a total of T1 sample rows, and the reference sample (406) may have a total of T2 sample rows. The prediction signal of the current prediction block (402) can be given by Equation (1) as follows.

[0072] Equation (1)

[0073] Where F(x, y, k) are the trained coefficients (or weight coefficients), and r(k) is considered a reference sample. (x, y) represents the coordinates within the current prediction block (402), and k is an iterator that iterates through all reference samples, such as reference samples (404) and (406). The final predictor P(x, y) is a weighted sum of all reference samples according to equation (1).

[0074] In the second method, signaling overhead may not be introduced because the predefined subset of intra-frame modes in the first method is replaced by a matrix multiplication-based method according to the second method. In other words, some regular intra-frame modes are established using a matrix multiplication-based method.

[0075] In video encoding and decoding, multiple primary (or secondary) transform kernel sets can be trained to adaptively compress different residual information. The transform set can be determined based on the intra-frame prediction mode derived in the first method.

[0076] In the example, the transform kernel is a specific mathematical function or matrix used in the transformation. For instance, the Discrete Cosine Transform (DCT) has a specific mathematical definition of a "kernel" that produces the values ​​used in the transformation process. The kernel includes the mathematical operations that perform the transformation from the spatial domain to the frequency domain.

[0077] In the example, a transform set is a collection of transform kernels. Multiple transforms can be used to improve compression efficiency. This means that a video codec standard can have a set of available transform kernels. The encoder can then select the most suitable kernel from the set for each block of the video based on the characteristics of the content. Therefore, a transform set is a group of different transform kernels that a video codec can use.

[0078] This disclosure includes aspects such as introducing a decoder-side intra-mode derivation method to determine a transform set for a predefined subset of intra-modes. In one aspect, the predefined subset of intra-modes can be derived according to a second method. Therefore, for the predefined subset of intra-modes, an intra-predictive signal is generated using a matrix multiplication-based method according to the second method.

[0079] On one hand, the decoder-side intra-frame mode is derived based on the prediction signal obtained using a matrix multiplication-based method. For example, the prediction signal is obtained based on a matrix multiplication-based method. A predefined subset of intra-frame modes is determined based on the prediction signal. This predefined subset of intra-frame modes may include planar mode, DC mode, and angled mode.

[0080] On the one hand, the transform set is determined conditionally using the decoder-side intra-frame mode based on other encoding and decoding information.

[0081] In the example, when the current block size is less than a predefined value N (e.g., N is 1024 samples), the application uses the decoder-side intra-frame mode to determine the transform set.

[0082] In the example, when the ratio of the width to the height of the current block is less than a predefined value R (e.g., R is 4), the application uses the decoder-side intra-frame mode to determine the transform set.

[0083] In the example, when the width and / or height of the current block are less than predefined values ​​W and H (e.g., W=32, H=32), the application uses the decoder-side intra-frame mode to determine the transform set.

[0084] In the example, in the template size (e.g., Figure 4 When T1 and / or T2 in the frame satisfy predefined size conditions (e.g., size T1=T2=2), the application uses the decoder-side intra-frame mode to determine the transform set.

[0085] On the one hand, a predefined subset of intra-frame modes includes planar modes.

[0086] On the one hand, a predefined subset of intra-frame modes includes DC mode.

[0087] On the one hand, the predefined subset of intra-frame modes includes equal to The mode (or angle mode), where k=[0,16].

[0088] On the one hand, the predefined subset of intra-frame modes includes equal to The mode (or angle mode), where k=[0, 32].

[0089] On the one hand, a predefined subset of intra-mode can include various combinations of intra-mode.

[0090] On the one hand, the determined transformation set is the main transformation set.

[0091] On the one hand, the determined transformation set is a quadratic transformation set.

[0092] On one hand, the determination of the transform set is based on a predefined table (or lookup table) between the derived intra-frame mode (e.g., an intra-frame mode derived from the decoder-side intra-frame mode based on a matrix multiplication method) and the transform set to be determined.

[0093] On one hand, multiple intra-frame modes are derived based on a decoder-side intra-frame mode derivation method. This method analyzes the most probable mode based on the predicted signal obtained using a matrix multiplication method. In this example, multiple intra-frame modes are derived based on a matrix multiplication method, and the most probable mode is determined based on a cost value, such as the cost between the predicted samples of the current block based on the multiple intra-frame modes and the samples of the current block.

[0094] Signaling syntax can be used in the bitstream to indicate which intra-mode within the derived intra-mode is used to determine the transform set.

[0095] In one aspect, in the second method, a separate transform kernel set is trained for a predefined subset of intra-frame modes, and in this second method, a matrix multiplication-based method is used to generate the predicted signal of the intra-frame modes in the predefined subset.

[0096] In one aspect, the predefined subset of intra-frame modes is separated from the regular intra-frame modes (e.g., planar mode, DC mode, and angle mode) in the first method. For example, the predefined subset of intra-frame modes obtained according to the second method and the regular intra-frame modes obtained according to the first method are explicitly signaled in the bitstream.

[0097] Emerging image and video codec standards can follow hybrid frameworks. Hybrid video codec frameworks can include various modules such as intra-frame prediction, inter-frame prediction, transform, quantization, and intra-loop filters. In video codecs such as ECM (Enhanced Compression Model), transforms can be used to decorrelate residuals for efficient encoding and decoding. In addition to (or further than) the Discrete Cosine Transform (DCT), Low-Frequency Non-Separable Transform (LFNST) and Non-Separable Primary Transform (NSPT) have been proposed to perform non-separable transforms that can provide better energy compression. In LFNST / NSPT, the set of transforms that can be applied to the current block can be inferred using the intra-frame mode associated with the current transform. To further improve encoding and decoding efficiency, a transform set can be proposed that allows the codec to search from the transform set associated with the current intra-frame mode plus a fixed offset.

[0098] Various aspects of this disclosure include methods for introducing a history-based offset search list to update the offset search list based on previously encoded offsets. A state transition machine (or process) can be applied to describe the proposed method. In an example, K possible states can be applied to encode the previous and / or last block. Each state can represent a corresponding offset. The transform set index can be appended with an offset to indicate a newly selected transform set. In an example, for state... This state It can have pointers to other states. There are 1 directed edge. The states that these edges point to can indicate candidates (or a list of candidates) to which the current transform set offset search can be performed.

[0099] In the example, if the previous state was 0, the search candidate list for the current block could be {0, -1, 1}. If an offset of 1 is chosen to encode the current block, the state can move to state 1, and the search candidate list can become {0, 1, 2}.

[0100] On the one hand, the state transition table can be fixed. In the example, the candidate states (or offsets) in the state transition table are determined (or fixed). Figure 5 An example of offset (or state) updates based on a state transition table (500) during encoding is shown, where the current state (or the state of the previous block) can be used to determine the offset that the current block can use. Figure 5As shown, in the encoding order, when encoding the previous block at offset 0, the state / offset of the current block can be 0, and the candidate list can be {0, -1, 1}, as... Figure 5 The arrows (502), (504) and (506) point to (or indicate) the points respectively.

[0101] If the current block is encoded with an offset of 1, then the next state / offset can be 1, and the search list for the next block can be {0, 1, 2}, which is in Figure 5 The values ​​can be indicated by arrows (508), (510), and (512), respectively. In the example, the offset search list can be determined based on the state x of the previous block, and the offset search list can be limited to (xd, x, x+d), where d is an integer.

[0102] On one hand, the offset (state) transition table is adaptive. In the example, a state (e.g., offset 0) can be added to the candidate list, regardless of whether that state is a previous state.

[0103] On the one hand, the state transition table can be modified during encoding. On the other hand... Figure 6 In the example shown, starting with the initial state transition table (602), the edges of the initial state transition table (602) can be updated during encoding based on the histogram (604). Figure 6 As shown, the previous M states can be collected and presented in a histogram (604) to provide the occurrence of previous states (or offsets). K edges can be identified as edges to be updated based on the histogram. For example, the K edges can be updated based on the most frequently occurring states. When the K edges are updated, an updated state transition table (604) is generated. In an example of the updated state transition table (604), states 1 and 3 can be swapped according to arrows (608) and (610).

[0104] On one hand, previous offsets can be derived under certain constraints based on available information. In the example, a previous offset can be used only if the corresponding CU of the previous offset is close to the current CU. In the example, an update can occur after encoding a single offset. In the example, an update can occur after encoding of a CTU is complete. In the example, when a valid offset history does not exist, the state and / or state transition table can be reinitialized.

[0105] On the one hand, the total number of edges associated with each state is the same.

[0106] On the one hand, the total number of edges associated with each state can be different.

[0107] On the one hand, intra-frame mode or other available information can be used as a classifier to classify offsets into different categories, and different state transition tables can be applied to handle offsets in different categories.

[0108] Figure 7 A flowchart outlining a process (700) according to one aspect of this disclosure is shown. The process (700) can be used in a video decoder. In various aspects, the process (700) is executed by a processing circuitry system, such as a processing circuitry system that performs the functions of a video decoder (110), a processing circuitry system that performs the functions of a video decoder (210), etc. In some aspects, the process (700) is implemented as software instructions, so that when the processing circuitry system executes the software instructions, the processing circuitry system executes the process (700). The process begins at (S701) and proceeds to (S710).

[0109] At (S710), a video bitstream is received, which includes encoded information of the current block in the current image. The encoded information includes multiple predicted samples of the current block. The predicted samples are obtained by matrix multiplication of the coefficient matrix and multiple reference samples of the current block.

[0110] At (S720), when an intra-prediction mode from the subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode from the subset is applied to the current block. The subset of intra-prediction modes is derived based on the prediction samples of the current block.

[0111] At (S730), the current block is reconstructed based on the predicted samples and the transform set.

[0112] On the one hand, transform sets are derived based on a lookup table that indicates the mapping between subsets of intra-predictive modes and multiple transform sets.

[0113] On one hand, when (i) an intra-prediction mode from a subset of intra-prediction modes is applied to the current block and (ii) the size of the current block or the size of the template including the reference sample satisfies a size condition, the transform set for the current block is derived. The size condition includes one of the following: the size of the current block is less than a predefined block size; the ratio of the width to the height of the current block is less than a predefined ratio; the width and height of the current block are less than a predefined size; and the size of the template is equal to a predefined template size.

[0114] On one hand, the subset of intra-prediction modes includes one of the following: a planar mode; a DC mode; a mode equal to (2+4×K), where K is an integer from 0 to 16; and a mode equal to (2+2×L), where L is an integer from 0 to 32.

[0115] On one hand, the transform set is either the primary transform set or the second-order transform set.

[0116] On one hand, the transform set includes a set of transform kernels trained for a subset of intra-frame prediction modes.

[0117] On one hand, an offset relative to the initial index is determined from the candidate list to obtain an updated index. The initial index indicates the initial transform set. The candidate list includes multiple candidate offsets obtained from the offset transformation table. The transform set is determined based on the updated index.

[0118] On one hand, the candidate list includes a first candidate offset applied to the previous reconstructed block of the current block, a second candidate offset in the offset transformation table linked to the first candidate offset according to the first edge, and a third candidate offset in the offset transformation table linked to the first candidate offset according to the second edge.

[0119] On one hand, multiple offsets from previously reconstructed blocks applied to the current block are collected from the offset transformation table. One or more offsets in the offset transformation table are updated based on one or more of the most frequent offsets.

[0120] On the one hand, the previously rebuilt block is adjacent to the current block.

[0121] On the one hand, updates are performed when encoding or decoding a CTU.

[0122] On the one hand, in the offset transformation table, the total number of edges associated with each offset is the same.

[0123] On the one hand, the total number of edges associated with each offset differs in the offset transformation table.

[0124] On the one hand, an offset transformation table is defined based on the intra-prediction mode associated with the current block.

[0125] Then, the process proceeds to (S799) and terminates.

[0126] The process (700) can be adjusted as appropriate. Steps in the process (700) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0127] Figure 8A flowchart outlining a process (800) according to one aspect of this disclosure is shown. This process (800) can be used for a video encoder. In various aspects, the process (800) is executed by a processing circuitry system, such as a processing circuitry system that performs the functions of a video encoder (103), a processing circuitry system that performs the functions of a video encoder (303), etc. In some aspects, the process (800) is implemented as software instructions, so that the processing circuitry system executes the process (800) when it executes the software instructions. The process begins at (S801) and proceeds to (S810).

[0128] At (S810), an intra-frame prediction mode subset is derived based on multiple predicted samples of the current block in the current image. The predicted samples are obtained by matrix multiplication of the coefficient matrix and multiple reference samples of the current block.

[0129] At (S820), when an intra-prediction mode from the subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode from the subset is applied to the current block. The subset of intra-prediction modes is derived based on the prediction samples of the current block.

[0130] At (S830), the predicted samples of the current block are encoded into the bitstream based on the transform set.

[0131] On one hand, a lookup table is used to determine the transform set, which indicates the mapping between a subset of intra-predictive modes and multiple transform sets.

[0132] On one hand, the transform set for the current block is determined when (i) an intra-prediction mode from a subset of intra-prediction modes is applied to the current block and (ii) the size of the current block or the size of the template including the reference sample satisfies a size condition. The size condition includes one of the following: the size of the current block is less than a predefined block size; the ratio of the width to the height of the current block is less than a predefined ratio; the width and height of the current block are less than a predefined size; and the size of the template is equal to a predefined template size.

[0133] On one hand, the subset of intra-prediction modes includes one of the following: a planar mode; a DC mode; a mode equal to (2+4×K), where K is an integer from 0 to 16; and a mode equal to (2+2×L), where L is an integer from 0 to 32.

[0134] On one hand, the transform set is either the primary transform set or the secondary transform set.

[0135] Then, the process proceeds to (S899) and terminates.

[0136] The process (800) can be adjusted as appropriate. Steps in the process (800) can be modified and / or omitted. Additional steps can be added. Any suitable implementation order can be used.

[0137] In one aspect, methods for processing visual media data include processing bitstreams of visual media data according to format rules. For example, a bitstream can be a bitstream decoded / encoded using any of the decoding and / or encoding methods described herein. Format rules can specify one or more constraints on the bitstream and / or one or more processes to be performed by the decoder and / or encoder.

[0138] In this example, the bitstream of visual media data is processed according to format rules. The bitstream includes encoded information for the current block in the current image. The encoded information includes multiple predicted samples for the current block. The predicted samples are obtained by matrix multiplication of the coefficient matrix and multiple reference samples for the current block. The format rules specify that when an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode subset is applied to the current block. The intra-prediction mode subset is derived based on the predicted samples for the current block. The format rules specify that the current block is processed based on the predicted samples and the transform set.

[0139] The above-described technology can be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, Figure 9 A computer system (900) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0140] Computer software can be encoded and decoded using any suitable machine code or computer language. The machine code or computer language can be subjected to mechanisms such as assembly, compilation, and linking to create code that includes instructions. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpretation, microcode execution, etc.

[0141] The instructions can be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0142] Figure 9The components shown for the computer system (900) are examples and are not intended to impose any limitation on the scope or functionality of computer software implementing the aspects of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any component or combination of components shown in the example aspects of the computer system (900).

[0143] The computer system (900) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input made by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., speech, tapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to conscious input made by humans, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images acquired from still image capturing devices), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0144] The input human-machine interface device may include one or more of the following (only one of each is shown): keyboard (901), mouse (902), touchpad (903), touch screen (910), data glove (not shown), joystick (905), microphone (906), scanner (907), and camera device (908).

[0145] The computer system (900) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via a touchscreen (910), data gloves (not shown), or joystick (905), but tactile feedback devices that are not used as input devices may also exist); audio output devices (e.g., speakers (909), headphones (not depicted)); visual output devices (e.g., screens (910), including CRT (Cathode Ray Tube, CRT) screens, LCD (Liquid Crystal Display, LCD) screens, plasma screens, OLED (Organic Light Emitting Diode, OLED) screens, each screen having or not having touchscreen input capability, each screen having or not having tactile feedback capability—some of which may be able to output two-dimensional visual output or more than three-dimensional output in a manner such as stereoscopic output; virtual reality glasses (not depicted); holographic displays and smoke generators (not depicted)); and printers (not depicted).

[0146] The computer system (900) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM (Read-only memory, ROM) / RW (920) having media such as CD / DVD (921), thumb drives (922), removable hard disk drives or solid-state drives (923), conventional magnetic media such as magnetic tape and floppy disks (not depicted), devices based on dedicated ROM / ASIC (Application Specific Integrated Circuit, ASIC) / PLD (Programmable Logic Device, PLD) such as security dongles (not depicted), etc.

[0147] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0148] The computer system (900) may also include an interface (954) to one or more communication networks (955). The network may be, for example, wireless, wired, or optical. The network may also be local area, wide area, metropolitan area, vehicular and industrial, real-time, latency-tolerant, etc. Examples of networks include: local area networks such as Ethernet; wireless LANs; cellular networks including GSM (Global System for Mobile communication), 3G (Third Generation), 4G (Fourth Generation), 5G (Fifth Generation), LTE (Long Term Evolution), etc.; cable or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; vehicular and industrial networks including CAN bus (Controller Area Network Bus), etc. Some networks typically require external network interface adapters attached to certain general-purpose data ports or peripheral buses (949) (such as, for example, the USB (Universal Serial Bus, USB) port of the computer system (900); other networks are typically integrated into the core of the computer system (900) via attachment to a system bus (e.g., an Ethernet interface to a PC (Personal Computer, PC) computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (900) can communicate with other entities. Such communication can be unidirectional, receive-only (e.g., broadcasting TV), transmit-only (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as to other computer systems using local or wide-area digital networks. As described above, certain protocols and protocol stacks can be used on each of these networks and network interfaces.

[0149] The aforementioned human-machine interface device, human-accessible storage device and network interface can be attached to the core (940) of the computer system (900).

[0150] The core (940) may include one or more central processing units (CPUs) (941), graphics processing units (GPUs) (942), dedicated programmable processing units (943) in the form of field programmable gate areas (FPGAs), task-specific hardware accelerators (944), graphics adapters (950), etc. These devices, along with read-only memory (ROM) (945), random access memory (946), and internal mass storage devices (947) such as internal non-user-accessible hard disk drives, SSDs (SSDs), etc., can be connected via the system bus (948). In some computer systems, the system bus (948) may be accessed in the form of one or more physical plugs to allow for expansion via additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (948) or attached to the core's system bus (948) via a peripheral bus (949). In the example, a screen (910) may be connected to the graphics adapter (950). Peripheral bus architectures include PCI (Peripheral Component Interconnect), USB, etc.

[0151] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions that can be combined to form the aforementioned computer code. This computer code can be stored in ROM (945) or RAM (946). Transient data can also be stored in RAM (946), while permanent data can be stored, for example, in an internal mass storage device (947). Fast storage and retrieval to any memory device in the memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage devices (947), ROMs (945), RAMs (946), etc.

[0152] Computer-readable media may have computer code for performing operations of various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type known and available to those skilled in the art of computer software.

[0153] By way of example and not limitation, a computer system (900) having an architecture, and in particular a core (940), can provide functionality through the execution of software embodied in one or more tangible computer-readable media by a processor (including CPU, GPU, FPGA, accelerator, etc.). Such a computer-readable medium can be a medium associated with a user-accessible mass storage device as described above, and with certain storage devices of the core (940) having non-transitory properties, such as a mass storage device (947) or ROM (945) within the core. Software implementing various aspects of this disclosure can be stored in such a device and executed by the core (940). Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause the core (940)—and in particular the processors therein (including CPU, GPU, FPGA, etc.)—to execute a particular process or a particular portion of a particular process described herein, including defining data structures stored in RAM (946) and modifying such data structures according to the process defined by the software. Alternatively or as an alternative, the computer system may provide functionality, either through hardwired logic or otherwise embodied in circuitry (e.g., an accelerator (944)), which may replace or operate with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0154] The use of “at least one of…” or “one of…” in this disclosure is intended to include any one or a combination of the listed elements. For example, references to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C are intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include either A or B or (A and B). The use of “one of…” does not exclude any combination of the listed elements where applicable, such as when the elements are not mutually exclusive.

[0155] Although several examples of various aspects have been described in this disclosure, there are modifications, substitutions, and various equivalents that fall within the scope of this disclosure. It will therefore be appreciated that those skilled in the art will be able to design a variety of systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and are therefore within its spirit and scope.

[0156] The above disclosure also includes the features mentioned below. These features can be combined in various ways, and are not limited to the combinations mentioned below.

[0157] (1) A video decoding method, the method comprising: receiving a video bitstream, the video bitstream including encoded information of a current block in a current picture, the encoded information including multiple prediction samples of the current block, the prediction samples being obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block; when an intra-prediction mode in a subset of intra-prediction modes is applied to the current block, determining a transform set for the current block based on which intra-prediction mode in the subset of intra-prediction modes is applied to the current block, the intra-prediction mode subset being derived based on the prediction samples of the current block; and reconstructing the current block based on the prediction samples and the transform set.

[0158] (2) According to the method of feature (1), determining the transform set further includes: determining the transform set according to a lookup table that indicates the mapping between a subset of intra-frame prediction modes and multiple transform sets.

[0159] (3) The method according to any one of features (1) and (2), wherein: determining the transform set includes: determining the transform set for the current block when an intra-prediction mode in the subset of intra-prediction modes is applied to the current block and the size of the current block or the size of the template including the reference sample satisfies the size condition; and the size condition includes one of the following: the size of the current block is less than the predefined block size, the ratio of the width of the current block to the height of the current block is less than the predefined ratio, the width of the current block to the height of the current block is less than the predefined size, and the size of the template is equal to the predefined template size.

[0160] (4) The method according to any one of features (1) to (3), wherein the subset of intra-prediction modes includes one of the following: a planar mode; a DC mode; a mode equal to (2+4×K), where K is an integer from 0 to 16; and a mode equal to (2+2×L), where L is an integer from 0 to 32.

[0161] (5) The method according to any one of the features (1) to (4), wherein the transformation set is the main transformation set or the second transformation set.

[0162] (6) The method based on any one of features (1) to (5), wherein the transform set includes a transform kernel set trained for a subset of intra-predictive patterns.

[0163] (7) The method according to any one of features (1) to (6), wherein determining the transformation set for the current block further includes: determining an offset relative to an initial index from a candidate list to obtain an updated index, the initial index indicating an initial transformation set, the candidate list including a plurality of candidate offsets obtained from offsets in an offset transformation table; and determining the transformation set based on the updated index.

[0164] (8) The method according to feature (7), wherein the candidate list includes: a first candidate offset of the previous reconstructed block applied to the current block, a second candidate offset linked to the first candidate offset according to the first edge in the offset transformation table, and a third candidate offset linked to the first candidate offset according to the second edge in the offset transformation table.

[0165] (9) The method according to any one of features (7) and (8) further includes: collecting multiple offsets from the offset transformation table for the previously reconstructed blocks applied to the current block; and updating one or more offsets in the offset transformation table based on one or more of the most frequent offsets.

[0166] (10) According to the method of feature (9), wherein the previously reconstructed block is adjacent to the current block.

[0167] (11) According to the method of feature (9), where an update is performed when a CTU is encoded or decoded.

[0168] (12) According to any one of the features (7) to (11), the total number of edges associated with each offset in the offset transformation table is the same.

[0169] (13) According to any one of the features (7) to (12), the total number of edges associated with each offset is different in the offset transformation table.

[0170] (14) The method according to any one of features (7) to (13), wherein the offset transformation table is defined based on the intra-prediction mode associated with the current block.

[0171] (15) A video coding method, the method comprising: deriving an intra-prediction mode subset based on multiple prediction samples of a current block in a current image, the prediction samples being obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block; when an intra-prediction mode in the intra-prediction mode subset is applied to the current block, determining a transform set for the current block based on which intra-prediction mode subset is applied to the current block, the intra-prediction mode subset being derived based on the prediction samples of the current block; and encoding the prediction samples of the current block into a bitstream based on the transform set.

[0172] (16) According to the method of feature (15), wherein determining the transform set further includes: determining the transform set according to a lookup table that indicates the mapping between a subset of intra-predictive modes and multiple transform sets.

[0173] (17) The method according to any one of features (15) and (16), wherein: determining the transform set includes: determining the transform set for the current block when an intra-prediction mode in a subset of intra-prediction modes is applied to the current block and the size of the current block or the size of the template including the reference sample satisfies a size condition; and the size condition includes one of the following: the size of the current block is less than a predefined block size, the ratio of the width of the current block to the height of the current block is less than a predefined ratio, the width of the current block and the height of the current block are less than a predefined size, and the size of the template is equal to a predefined template size.

[0174] (18) The method according to any one of features (15) to (17), wherein the subset of intra-frame prediction modes includes one of the following: a planar mode; a DC mode; a mode equal to (2+4×K), where K is an integer from 0 to 16; and a mode equal to (2+2×L), where L is an integer from 0 to 32.

[0175] (19) According to any one of the features (15) to (18), the transformation set is either the main transformation set or the second transformation set.

[0176] (20) A method for processing visual media data, the method comprising: processing a bitstream of visual media data according to a format rule, wherein: the bitstream includes encoded information of a current block in a current image, the encoded information including multiple prediction samples of the current block, the prediction samples being obtained by matrix multiplication of a coefficient matrix and multiple reference samples of the current block; and the format rule specifies that: when an intra-prediction mode in a subset of intra-prediction modes is applied to the current block, a transform set for the current block is determined based on which intra-prediction mode in the subset of intra-prediction modes is applied to the current block, the intra-prediction mode subset being derived based on the prediction samples of the current block; and processing the current block based on the prediction samples and the transform set.

[0177] (21) An apparatus for video decoding, comprising a processing circuit system configured to perform the method of any one of features (1) to (14).

[0178] (22) An apparatus for video decoding, comprising a processing circuit system configured to perform the method of any one of features (15) to (19).

[0179] (23) A non-transitory computer-readable storage medium storing instructions, which, when executed by at least one processor, cause the at least one processor to perform a method of any one of features (1) to (19).

Claims

1. A video decoding method, comprising: Receive a video bitstream, the video bitstream including encoded information of the current block in the current image, the encoded information including multiple prediction samples of the current block, the prediction samples being obtained by matrix multiplication of the coefficient matrix and multiple reference samples of the current block; When an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode from the subset of intra-prediction modes is applied to the current block, the subset of intra-prediction modes being derived from the prediction samples of the current block. as well as The current block is reconstructed based on the predicted sample and the transformation set.

2. The method according to claim 1, wherein, Determining the transform set further includes: The transform set is determined according to a lookup table, which indicates the mapping between the subset of intra-prediction modes and multiple transform sets.

3. The method according to claim 1 or 2, wherein: Determining the transform set includes: when an intra-prediction mode from the subset of intra-prediction modes is applied to the current block, and the size of the current block or the size of the template including the reference sample satisfies a size condition, determining the transform set for the current block; and The dimensional conditions include one of the following: The size of the current block is smaller than the predefined block size. The ratio of the width to the height of the current block is less than a predefined ratio. The width and height of the current block are smaller than the predefined dimensions, and The size of the template is equal to the predefined template size.

4. The method according to any one of claims 1 to 3, wherein, The subset of intra-frame prediction modes includes one of the following: a planar mode; a DC mode; a mode equal to (2+4×K), where K is an integer from 0 to 16; and a mode equal to (2+2×L), where L is an integer from 0 to 32.

5. The method according to any one of claims 1 to 4, wherein, The transform set is either the primary transform set or the secondary transform set.

6. The method according to any one of claims 1 to 5, wherein, The transform set includes a transform kernel set, which is trained on the subset of intra-frame prediction modes.

7. The method according to any one of claims 1 to 6, wherein, Determining the transformation set for the current block further includes: An updated index is obtained by determining an offset relative to an initial index from a candidate list, the initial index indicating an initial transformation set, the candidate list including multiple candidate offsets obtained from offsets in an offset transformation table; and The transformation set is determined based on the updated index.

8. The method according to claim 7, wherein, The candidate list includes: The first candidate offset applied to the previous reconstructed block of the current block. The offset transformation table is based on the second candidate offset linked from the first edge to the first candidate offset, and The offset transformation table is a third candidate offset linked to the first candidate offset based on the second side.

9. The method according to claim 7 or 8, further comprising: The offset transformation table collects multiple offsets of previously reconstructed blocks that are adjacent to the current block and are applied to the current block. as well as Update one or more offsets in the offset transformation table based on one or more of the most frequent offsets.

10. The method according to claim 9, wherein, The update is performed when encoding or decoding a CTU.

11. The method according to any one of claims 7 to 10, wherein, In the offset transformation table, the total number of edges associated with each offset is the same.

12. The method according to any one of claims 7 to 11, wherein, The total number of edges associated with each offset is different in the offset transformation table.

13. The method according to any one of claims 7 to 12, wherein, The offset transformation table is defined based on the intra-prediction mode associated with the current block.

14. A video coding method, comprising: A subset of intra-frame prediction modes is derived based on multiple prediction samples of the current block in the current image. The prediction samples are obtained by matrix multiplication of the coefficient matrix and multiple reference samples of the current block. When an intra-prediction mode from a subset of intra-prediction modes is applied to the current block, the transform set for the current block is determined based on which intra-prediction mode from the subset of intra-prediction modes is applied to the current block, the subset of intra-prediction modes being derived from the prediction samples of the current block. as well as The predicted samples of the current block are encoded into a bitstream based on the transform set.

15. A method for processing visual media data, the method comprising: The bitstream of the visual media data is processed according to format rules, wherein: The bitstream includes encoded information of the current block in the current image, and the encoded information includes a predicted sample of the current block, which is obtained by matrix multiplication of the coefficient matrix and a reference sample of the current block; and The formatting rules specify: When a subset of intra-prediction modes is applied to the current block, a transform set for the current block is determined based on which subset of intra-prediction modes is applied to the current block, the subset being derived from the prediction samples of the current block; and The current block is processed based on the predicted samples and the transformation set.

Images