Methods, apparatus and media for video encoding and decoding, and methods for processing visual media data.

Abstract

This application discloses a method, apparatus, and medium for video encoding and decoding, and a method for processing visual media data. A template region associated with a location is an aligned template region used for contextual deduction of a set of latent syntax elements for encoding the quantization transform coefficient level at that location. The set of latent syntax elements includes: a first latent syntax element for encoding a basic range level, a second latent syntax element for encoding a lower range level, and a third latent syntax element for encoding a higher range level. For a first location of a transform block, at least one context model is determined based on the first template region associated with the first location, for at least one syntax element. The at least one syntax element is decoded from the encoding information of the transform block based on the at least one context model. A first quantization transform coefficient level at the first location is determined based on the at least one syntax element.

Description

[0001] This application claims priority to U.S. Provisional Application No. 63 / 724,323, filed November 23, 2024, and U.S. Application No. 19 / 366,381, filed October 22, 2025. The entire disclosure of the earlier applications is incorporated herein by reference. Technical Field

[0002] This application relates to video encoding and decoding technologies, and more particularly to methods and apparatus for video decoding and / or encoding, methods for processing visual media data, and non-volatile computer-readable storage media. Background Technology

[0003] The background description provided herein is intended to present the general context of this disclosure. The extent of the work of the currently attributed inventors described in the background section and in the various embodiments of this specification does not imply that it was prior art at the time of this disclosure filing, nor is it expressly or implied that it was acknowledged as prior art to this disclosure.

[0004] Image / video compression helps transmit image / video data across different devices, storage, and networks while minimizing quality degradation. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which compresses images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current image being reconstructed to predict samples. In another example, a video codec can use a technique called inter-frame prediction, which compresses images based on temporal redundancy. For example, inter-frame prediction can predict samples in the current image from previously reconstructed images using motion compensation. Motion compensation can be indicated by motion vectors (MV).

[0005] Based on existing image / video compression technologies, there is a need to provide a technique for encoding and decoding transform coefficients to reduce the data represented by the transform coefficients and to encode them at different levels, thereby enabling the expansion of the codec. Summary of the Invention

[0006] Embodiments of this disclosure include bitstreams, methods, and apparatus for video encoding / decoding. In some embodiments, the video encoding / decoding apparatus includes processing circuitry.

[0007] Some embodiments of this disclosure provide a method for video decoding. In some examples, an encoded video stream is received. The encoded video stream includes encoding information of a transform block of a current block in a current image, the encoding information of which indicates the quantization transform coefficient level at various locations of the transform block, each location being associated with a respective template region for context inference. The template region associated with a location is an aligned template region for context inference of a set of latent syntax elements used to encode the quantization transform coefficient level at the location. The set of latent syntax elements includes: at least one first latent syntax element for encoding a base range level, at least one second latent syntax element for encoding a low range level, and at least one third latent syntax element for encoding a high range level. For a first location of the transform block, at least one context model is determined based on a first template region associated with the first location, each of the at least one syntax element indicating a first quantization transform coefficient level at the first location. The at least one syntax element is decoded from the encoding information of the transform block based on the at least one context model. The first quantization transform coefficient level at the first location is determined based on the at least one syntax element. The transform block is reconstructed at least based on the first quantization transform coefficient level at the first position of the transform block.

[0008] Some embodiments of this disclosure provide a video encoding method. In an example, a residual block of a current block in a current image is computed. The residual block is transformed into a transform block, the transform block including quantized transform coefficient levels at various locations, each location being associated with a template region for context inference. The template region associated with a location is an aligned template region for context inference of a set of latent syntax elements used to encode the quantized transform coefficient levels at the location, the set of latent syntax elements including: at least one first latent syntax element for encoding a base range level, at least one second latent syntax element for encoding a lower range level, and at least one third latent syntax element for encoding a higher range level. For a first location of the transform block, at least one context model is determined based on a first template region associated with the first location, each of the at least one syntax element indicating a first quantized transform coefficient level at the first location. The at least one syntax element is encoded into encoded information in the bitstream based on the at least one context model.

[0009] Embodiments of this disclosure also provide a method for storing a video stream, comprising performing the video encoding method to generate a video stream, and storing the video stream.

[0010] Embodiments of this disclosure also provide an apparatus for video decoding. The video decoding apparatus includes processing circuitry configured to implement any of the methods for video decoding.

[0011] Embodiments of this disclosure also provide a video encoding apparatus. The video encoding apparatus includes processing circuitry configured to implement any of the methods of video encoding.

[0012] Embodiments of this disclosure also provide a non-volatile computer-readable storage medium storing instructions that, when executed by at least one processor of a computer, cause the at least one processor of the computer to perform any of the video encoding methods described above to generate a bitstream and store and transmit the bitstream.

[0013] In the video encoding and / or decoding method provided in this application embodiment, a context model is determined by using template regions associated with each position of the transform block of the current block. On the encoding side, syntax elements indicating the quantization transform coefficient level at each position are encoded into the bitstream according to the context model. Alternatively, on the decoding side, syntax elements indicating the quantization transform coefficient level at each position are decoded from the bitstream according to the context model, thereby reconstructing the transform block based on the quantization transform coefficient level at each position. This encoding / decoding method can be used to extend the codec to improve encoding / decoding efficiency. Attached Figure Description

[0014] Other features, properties, and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, wherein:

[0015] Figure 1 This is a schematic diagram of an example block diagram of a communication system.

[0016] Figure 2 This is a schematic diagram of an example block diagram of a decoder.

[0017] Figure 3 This is a schematic diagram of an example block diagram of an encoder.

[0018] Figure 4 The diagrams show some examples of the base range, low range, and high range.

[0019] Figure 5 The diagram illustrates some examples of region-based coefficient encoding.

[0020] Figure 6 A transformation unit according to an embodiment of the present disclosure is shown.

[0021] Figure 7A schematic diagram of a transformation unit in an example is shown.

[0022] Figure 8 A schematic diagram of the transformation unit in another example is shown.

[0023] Figure 9 A schematic diagram of the transformation unit in another example is shown.

[0024] Figure 10 Schematic diagrams of transformation units with scan-line-related templates for context derivation are shown in some examples.

[0025] Figure 11 The diagram shows some examples of transformation units with scan-line and position-related templates for context derivation.

[0026] Figure 12 A flowchart outlining the decoding process according to some embodiments of this disclosure is shown.

[0027] Figure 13 A flowchart outlining the coding process according to some embodiments of this disclosure is shown.

[0028] Figure 14 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0029] Figure 1 Block diagrams of some example video processing systems (100) are shown. The video processing system (100) is an example application for the disclosed subject matter, video encoders, and video decoders in a streaming environment. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0030] The video processing system (100) includes an acquisition subsystem (113) that may include a video source (101) such as a digital camera, which creates an uncompressed video image stream (102). In an embodiment, the video image stream (102) includes samples captured by a digital camera. The video image stream (102) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (104) (or encoded video bitstream). The video image stream (102) may be processed by an electronic device (120) that includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or practice the various embodiments of the disclosed subject matter described in more detail below. Compared to the video image stream (102), the encoded video data (104) (or encoded video bitstream) is depicted as a thin line to emphasize the lower data volume of the encoded video data (104) (or encoded video bitstream), which can be stored on a streaming server (105) for future use. At least one streaming client subsystem, such as Figure 1 Client subsystems (106) and (108) can access a streaming server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and produces an output video picture stream (111) that can be displayed on a display (112) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), video data (107), and video data (109) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0031] It should be noted that the electronic devices (120) and (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0032] Figure 2An example block diagram of a video decoder (210) is shown. The video decoder (210) may be disposed in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may be used in place of... Figure 1 The video decoder (110) in the embodiment.

[0033] The receiver (231) may receive, for example, at least one encoded video sequence included in the bitstream that will be decoded by the video decoder (210). In one embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (231) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be located outside the video decoder (210) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (210) to prevent network jitter, for example, and another buffer (215) may be configured internally for, for example, handling broadcast timing. When the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (215) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (215) may also be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (210).

[0034] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (210) and potential information for controlling display devices (212), such as displays that are not part of the electronic device (230) but may be coupled to it. Figure 2As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (220) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0035] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0036] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve at least two distinct units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (220). For brevity, the flow of such subgroup control information between the parser (220) and the at least two units described below is not described.

[0037] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0038] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives quantization transform coefficients as symbols (221) and control information from the parser (220), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output a block containing sample values, which can be input into the aggregator (255).

[0039] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed image, but can use predictive information from a previously reconstructed portion of the current image. Such predictive information may be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) uses reconstructed information extracted from the current picture buffer (258) to generate a surrounding block of the same size and shape as the block being reconstructed. For example, the current picture buffer (258) buffers a partially reconstructed current image and / or a fully reconstructed current image. In some cases, the aggregator (255) adds the predictive information generated by the intra-picture prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) based on each sample.

[0040] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (253) can access the reference image memory (257) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (221), these samples can be added by the aggregator (255) to the output of the scaler / inverse transform unit (251) (referred to as residual samples or residual signals in this case) to generate output sample information. The motion compensation prediction unit (253) can obtain the predicted samples from the address in the reference image memory (257) under motion vector control, and the motion vector is available to the motion compensation prediction unit (253) in the form of the symbols (221), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0041] The output samples of the aggregator (255) can be employed by various loop filtering techniques in the loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (221) from the parser (220) in the loop filter unit (256). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0042] The output of the loop filter unit (256) can be a sample stream, which can be output to a display device (212) and stored in a reference image memory (257) for subsequent inter-frame image prediction.

[0043] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded images corresponding to the current image have been fully reconstructed and the encoded images (by, for example, the parser (220)) are identified as reference images, the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0044] The video decoder (210) performs decoding operations according to a predetermined video compression technology or standard (e.g., ITU-T H.265). The encoded video sequence conforms to the syntax specified by the video compression technology or standard, in the sense that the encoded video sequence follows the syntax of the video compression technology or standard and the configuration file recorded in the video compression technology or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technology or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in, for example, megasamples per second), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0045] In this embodiment, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0046] Figure 3 An example block diagram of a video encoder (303) is shown. The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., transmission circuitry). The video encoder (303) can be used in place of... Figure 1 The video encoder (103) in the embodiment.

[0047] The video encoder (303) can be obtained from the video source (301) (not Figure 3 In one embodiment, a portion of the electronic device (320) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (303). In another embodiment, the video source (301) is a portion of the electronic device (320).

[0048] A video source (301) can provide a sequence of source videos in the form of a digital video sample stream, which will be encoded by a video encoder (303). This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit…), any color space (e.g., BT.601 YCrCb, RGB…), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera that captures local image information as a video sequence. The video data can be provided as at least two separate images, which are given motion when viewed sequentially. The images themselves can be constructed as spatial pixel arrays, where each pixel can include at least one sample, depending on the sampling structure, color space, etc., used. The following focuses on describing the samples.

[0049] According to an embodiment, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or under any other required time constraints. Implementing an appropriate encoding rate is a function of the controller (350). In some embodiments, the controller (350) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (350) may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) may be used with other suitable functions related to the video encoder (303) optimized for a particular system design.

[0050] In some embodiments, the video encoder (303) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (333) embedded within the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (334) also correspond bit-accurately between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs, for example, when synchronization cannot be maintained due to channel errors) is also used in some related techniques.

[0051] The operation of the “local” decoder (333) can be combined with, for example, the above-mentioned... Figure 2 The video decoder (310) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 2 When symbols are available and the entropy encoder (345) and parser (220) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (210), including the buffer (215) and parser (220), may not be fully implemented in the local decoder (333).

[0052] In the embodiments, in addition to the parsing / entropy decoding present in the decoder, the decoder technology also exists in the corresponding encoder in the same or substantially the same functional form. Therefore, this application focuses on decoder operation. The description of the encoder technology can be simplified because the encoder technology is the inverse of the fully described decoder technology. In certain areas, a more detailed description is provided below.

[0053] During operation, in some embodiments, the source encoder (330) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image with reference to at least one previously encoded image from the video sequence designated as a "reference image." In this manner, the encoding engine (332) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0054] The local video decoder (333) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (330). The operation of the encoding engine (332) can be a lossy process. When the encoded video data can be decoded by the video decoder (333), Figure 3 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image memory (334). In this way, the video encoder (303) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0055] The predictor (335) can perform a prediction search against the encoding engine (332). That is, for a new image to be encoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (335) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (335), it can be determined that the input image can have prediction references obtained from at least two reference images stored in the reference image memory (334).

[0056] The controller (350) can manage the encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0057] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (345). The entropy encoder (345) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into an encoded video sequence.

[0058] The transmitter (340) can buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (340) can combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0059] The controller (350) manages the operation of the video encoder (303). During encoding, the controller (350) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0060] Intra-frame pictures (I-pictures) can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures.

[0061] Predictive images (P-images) can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses motion vectors and reference indices to predict sample values ​​for each block.

[0062] Bidirectional predictive images (B-images) can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses two motion vectors and a reference index to predict sample values ​​for each block. Similarly, at least two predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0063] The source image is typically spatially subdivided into at least two sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively encoded with reference to other (already encoded) blocks, which are determined based on the encoding assignment of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively encoded, or the blocks can be predictively encoded (spatial prediction or intra-frame prediction) with reference to already encoded blocks of the same image. Pixel blocks of a P-image can be predictively encoded with reference to a previously encoded reference image via spatial prediction or temporal prediction. Blocks of a B-image can be predictively encoded with reference to one or two previously encoded reference images via spatial prediction or temporal prediction.

[0064] The video encoder (303) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations utilizing temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0065] In this embodiment, the transmitter (340) may transmit additional data while transmitting encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0066] The acquired video can serve as at least two source images (video images) in a time-series format. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when using at least two reference images, the motion vector may have a third dimension that identifies the reference image.

[0067] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0068] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0069] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into at least one coding unit (CU) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into at least one prediction unit (PU). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0070] It should be noted that any suitable technology can be used to implement the video encoders (103) and (303) and the video decoders (110) and (210). In one embodiment, at least one integrated circuit can be used to implement the video encoders (103) and (303) and the video decoders (110) and (210). In another embodiment, at least one processor executing software instructions can be used to implement the video encoders (103) and (303) and the video decoders (110) and (210).

[0071] Some embodiments of this disclosure provide techniques for video and image compression, including techniques for transform coefficient coding.

[0072] Video coding is widely used in many applications. Video codecs can adopt appropriate video coding standards, such as H.264 / AVC, H.265 / HEVC, H.266 / VVC, AV1, and AVS / AVS2 / AVS3. Video codecs can include various components, such as intra-frame prediction, inter-frame prediction, image transformation, quantization, coefficient coding, entropy coding, loop filtering, etc.

[0073] For example, transform techniques can reduce redundancy in video signals by decorrelation, and quantization techniques can reduce the data represented by transform coefficients by reducing precision (e.g., by removing imperceptible details), thereby reducing the inconsistency in the data.

[0074] In some examples, the transform decorrelates the signal by transforming it from the spatial domain to the transform domain (e.g., the frequency domain) using a suitable transform basis. For instance, the transform is applied to the prediction residual (whether it comes from inter-frame or intra-frame prediction), i.e., the difference between the prediction and the original input video signal. In the transform domain, fundamental information is typically concentrated in a few coefficients. At the decoder, an inverse transform can be applied to reconstruct the residual samples.

[0075] Typically, quantization is used to reduce the precision of an input value or a set of input values ​​to reduce the amount of data required to represent those values. In some examples, quantization is often applied to residual samples of a single transform (e.g., transform coefficients), resulting in integer coefficient levels. The quantization process is applied at the encoder. At the decoder, the corresponding process is called inverse quantization (also known as dequantization), which restores the original range of values ​​without restoring precision.

[0076] In some codec examples (e.g., VVC), multiple transform types can be used in the main transform, such as Type 2 DCT (DCT-2), Type 7 DST (DST-7), Type 8 DCT (DCT-8), etc. In some examples, a technique called Multiple Transform Selection (MTS) can be used. In one example, explicit MTS can use signals to explicitly indicate the selection of the transform kernel. In another example, implicit MTS can implicitly derive the selection of the transform kernel. In some embodiments, explicit MTS can be applied to both intra-coded blocks and inter-coded blocks, while implicit MTS can only be applied to intra-coded blocks. In one embodiment, in explicit MTS, the selection of DST-7 / DCT-8 is indicated by explicit signaling of the transform type. In another embodiment, in implicit MTS, the transform type is selected based on coding information known to both the encoder and decoder, and transform type signaling is not required.

[0077] In some examples, in explicit MTS, indices (e.g., represented by `mts_idx`) are signaled as CU-level syntax to indicate the transform types used for horizontal and vertical transforms. In one example, the value of `mts_idx` ranges from 0 to 4. For example, a value of `mts_idx` of 0 indicates that DCT-2 is used for both horizontal and vertical transforms; a value of `mts_idx` of 1 indicates that DST-7 is used for both horizontal and vertical transforms; a value of `mts_idx` of 2 indicates that DCT-8 is used for both horizontal and vertical transforms; a value of `mts_idx` of 3 indicates that DST-7 is used for both horizontal and vertical transforms; and a value of `mts_idx` of 4 indicates that DCT-8 is used for both horizontal and vertical transforms.

[0078] In some examples, a secondary transform can be applied after the primary transform. For instance, the Low-Frequency Inseparable Transform (LFNST) is an inseparable transform that can be applied to the upper-left low-frequency region of the primary transform coefficients. In some examples (e.g., VVC), LFNST can be applied to intra-coded blocks using DCT-2 as the primary transform. The transform kernel defined in the LFNST can include at least two transform sets, such as the four transform sets in VVC. In some examples, the selection of a transform set from the four LFNST sets (e.g., denoted by lfnstSetIdx) depends on the intra-prediction mode (e.g., denoted by intraPredMode). In some examples, a lookup table is used that maps the LFNST sets to intra-prediction modes.

[0079] In some embodiments, different types of DCTs can be used as candidates, and a suitable DCT can be selected to improve coding efficiency. In some examples, the Non-Separable Master Transform (NSPT) and the Low-Frequency Non-Separable Transform (LFNST) provide learned kernels to maximize energy compaction capabilities. On the decoder side, the inverse transform kernel is applied to the dequantized coefficients to reconstruct the spatial block.

[0080] Some embodiments of this disclosure provide various techniques for coordinating context derivation to arithmetically encode transform coefficients generated by inseparable transforms. These techniques can be used individually or in combination in any order. Furthermore, each technique, encoder, and decoder can be implemented by processing circuitry (e.g., at least one processor or at least one integrated circuit).

[0081] According to embodiments of this disclosure, these techniques can be used in future video codecs and extensions of existing codecs or codecs mentioned in this disclosure.

[0082] Note that the term "block" in this disclosure may correspond to a coding tree block, a maximum coding block, a predefined fixed-size block, a coding block, a prediction block, a residual block, or a transform block.

[0083] Note that the term "coefficient encoding" in this disclosure may correspond to encoding (or parsing) at the quantization transform coefficient level. In some embodiments, the quantization transform coefficient level may be encoded within different level ranges. For example, the Alliance for Open Media (AOM) has developed a video coding standard called the AOM Video Model (AVM). In the AVM, quantization transform coefficients are encoded using three ranges, referred to as the base range (BR), low range (LR), and high range (HR). These three ranges are defined based on a threshold and region of the level value in the block. Furthermore, in the AVM, the context derivation of the Rice parameters used to parse the truncated Rice codes for the base range, low range, and high range may also depend on a predefined template region covering the parsed sample. The number of neighbors used in the template region can vary depending on the Rice parameter derivation for the base range, low range, and high range.

[0084] Figure 4 The diagram illustrates ranges in some examples. For instance, the magnitude (level) of a quantization transform coefficient can be encoded across at least two passes based on a range. For example, the base range includes the lowest binary number of the quantization transform coefficient (e.g., ...). Figure 4 The symbols from 0 to N-1), the lower range includes the next set of binary numbers after the quantization transform coefficients (e.g., Figure 4The lower range includes symbols from N to N+2 (e.g., when the quantization transform coefficients are greater than the maximum value represented by the symbols in the lower range), and the higher range includes the remaining binary numbers of the quantization transform coefficients (e.g., when the quantization transform coefficients are greater than the maximum value represented by the symbols in both the lower and lower ranges). A binary number (bin) refers to a single binary symbol that represents a piece of information in the video data. In some examples, symbols in the lower and lower ranges can be encoded using entropy coding, and when symbols are present in the higher range, symbols in the higher range can be encoded using truncated Rice coding, with entropy coding applied to the Rice parameters of the truncated Rice coding.

[0085] According to an embodiment, the number of symbols in the base range may depend on other factors such as region, color component, etc.

[0086] Figure 5 A first schematic diagram (510) and a second schematic diagram (520) of region-based coefficient encoding (e.g., transform coefficient encoding) are shown in some examples.

[0087] The first schematic diagram (510) illustrates an 8×8 transform unit with two-dimensional (2D) transformation. Note that in the example, the transform unit may be a transform block to which mathematical transformations are applied. The 8×8 transform unit includes a low-frequency region (511) (e.g., the shaded portion) and a default region (512) (e.g., the unshaded portion). The low-frequency region (511) is encoded using a base region with 6 symbols, and the default region (512) is encoded using a base region with 4 symbols.

[0088] Similarly, the second schematic diagram (550) shows an 8×8 transform unit with a one-dimensional (1D) vertical transform. This 8×8 transform unit includes a low-frequency region (551) (e.g., the shaded area) and a default region (552) (e.g., the unshaded area). The low-frequency region (551) is encoded using a base region with 6 symbols, and the default region (552) is encoded using a base region with 4 symbols.

[0089] Furthermore, in some examples (e.g., AVM), the context derivation of the Rice parameters used to parse symbols in the base range, symbols in the lower range, and truncated Rice codes in the higher range can also depend on a predefined template region covering the parsed sample.

[0090] Figure 6A transformation unit (601) according to an embodiment of this disclosure is shown. The transformation unit (601) may be a transformation block. In the example, the transformation unit scans in a top-right diagonal scanning order. The current scan position (610) is shown as a black square. When encoding the symbol at the current scan position (610), the local neighbors (shown by shaded squares) of the encoded portion (620) (also referred to as the template region (620) of the current scan position (610)) can be used for context deduction.

[0091] The number of neighbors used by the template region can vary depending on the context derivation used for symbol encoding in the base range, the context derivation used for symbol encoding in the low range, and the Rice parameter derivation used in the high range.

[0092] Figure 7 A schematic diagram of a transformation unit (701) in one example is shown. In some examples, previous encoding positions can be used for context derivation, but different numbers of encoding positions are used for context derivation of the basic range symbols and for context derivation of the lower range symbols. For example, in Figure 7 In this process, the transformation unit (701) is scanned (e.g., encoded) according to the scanning order of the upper right diagonal (e.g., in the order of 7, 6, 5, 4, 3, 2, and 1). Figure 7 In the example, at least two previous scan positions of the current scan position form a template for context inference, and the context inference of the base range and the context inference of the lower range use different numbers of previous scan positions.

[0093] In the example, the current scan position (e.g., used to encode the symbol in the current scan position) is 1, and the decoded symbols (corresponding to 5 quantization transform coefficients) of the previous 5 scan positions (e.g., 6, 5, 4, 3, and 2) in the encoding order are used for context deduction of the base range symbol at scan position 1; the decoded symbols (corresponding to 3 quantization transform coefficients) of the previous 3 scan positions (e.g., 4, 3, and 2) in the encoding order are used for context deduction of the lower range symbol at scan position 1.

[0094] Figure 8 A schematic diagram of the transform unit (801) in another example is shown. In some examples, previous encoded positions can be used for context derivation, but the context derivation of the base range symbol and the context derivation of the lower range symbol use different numbers of encoded positions. For example, in Figure 8 In this process, the transformation unit (801) is scanned (e.g., encoded) according to the raster scanning sequence (e.g., in the order of 7, 6, 5, 4, 3, 2 and 1).

[0095] In the example, the current scan position (e.g., used to encode the symbol in the current scan position) is 1, and the decoded symbols (corresponding to 5 quantization transform coefficients) of the previous 5 scan positions (e.g., 6, 5, 4, 3, and 2) in the encoding order are used for context deduction of the base range symbol at scan position 1; the decoded symbols (corresponding to 3 quantization transform coefficients) of the previous 3 scan positions (e.g., 4, 3, and 2) in the encoding order are used for context deduction of the lower range symbol at scan position 1.

[0096] In some embodiments, the template used to derive the Rice parameters of the truncated Rice encoding for the high range may be different from the template used for context derivation of the basic and low ranges.

[0097] Figure 9 A schematic diagram of the transformation unit (901) in another example is shown. In some examples, the template for the context derivation of the Rice parameters is formed by the adjacent positions of the current scan position. For example, in Figure 9 In the process, the transform unit (901) is scanned (e.g., encoded) according to the scanning order of the upper right diagonal (e.g., in the order of 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, and 1). When the current scan position is 4, the template for the context derivation of the Rice parameter at that current scan position includes positions 7, 8, and 11.

[0098] Some embodiments of this disclosure provide techniques for aligning templates (e.g., using the same template) for context derivation of Rice parameter derivations of base range, low range, and high range level values ​​of inseparable primary / secondary transform coefficients. In some examples, the template region associated with a location is an aligned template region for context derivation of a set of latent syntax elements used to encode the quantized transform coefficient level at that location. This set of latent syntax elements includes at least one first latent syntax element for encoding the base range level, at least one second latent syntax element for encoding the low range level, and at least one third latent syntax element for encoding the high range level. For a first location of a transform block, at least one context model for at least one syntax element can be determined based on the first template region associated with that first location. The at least one syntax element can be encoded / decoded according to the at least one context model.

[0099] In one embodiment, the first N, M, and K coefficients in the encoding order are used for Rice parameter derivation of the base range, low range, and high range. In some examples, {N, M, K} may include, but is not limited to, {5, 3, 3}.

[0100] For example, return to reference Figure 7In the example, the context derivation for decoding the base range symbol at the current encoding position 1 uses five coefficients from positions 2, 3, 4, 5, and 6 (e.g., decoded quantization transform coefficients); the context derivation for decoding the low range symbol at the current encoding position 1 uses three coefficients from positions 2, 3, and 4 (e.g., decoded quantization transform coefficients); and the context derivation for decoding the high range Rice parameter at the current encoding position 1 uses three coefficients from positions 2, 3, and 4 (e.g., decoded quantization transform coefficients).

[0101] In another embodiment, the first N, M, and K coefficients in the encoding order (excluding the nearest neighbors) are used for Rice parameter derivation of the base range, low range, and high range. In some examples, {N, M, K} may include, but is not limited to, {5, 3, 3}.

[0102] For example, return to reference Figure 7 In the example, for the context derivation of decoding the basic range symbol at the current encoding position 1, five coefficients from positions 3, 4, 5, 6, and 7 (excluding position 2) are used (e.g., decoded quantization transform coefficients); for the context derivation of decoding the low range symbol at the current encoding position 1, three coefficients from positions 3, 4, and 5 (excluding position 2) are used (e.g., decoded quantization transform coefficients); and for the context derivation of decoding the high range Rice parameter at the current encoding position 1, three coefficients from positions 3, 4, and 5 (excluding position 2) are used (e.g., decoded quantization transform coefficients).

[0103] In some embodiments, the template used for context deduction is associated with the scan line. For example, the same template can be used at at least two locations within the scan line.

[0104] Figure 10 Schematic diagrams of transformation units with scan-line-related templates for context derivation are shown in some examples.

[0105] exist Figure 10 In the example, the transformation unit is 4×4 and includes 0 to 15 scan positions. The transformation unit is scanned in the upper right diagonal scan order (e.g., in the order of 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 and 0).

[0106] exist Figure 10 In the schematic diagram (1010), the scan line only includes position 0, and the template for position 0 includes 5 positions, as shown in 1, 2, 3, 4 and 5.

[0107] exist Figure 10In the schematic diagram (1020), the scan lines include positions 2 and 1, and the template for either position 2 or 1 includes 5 positions, as shown in 3, 4, 5, 6 and 7.

[0108] exist Figure 10 In the schematic diagram (1030), the scan lines include positions 5, 4 and 3, and the template for any one of positions 5, 4 and 3 includes 5 positions, as shown in 6, 7, 8, 9 and 10.

[0109] exist Figure 10 In the schematic diagram (1040), the scan lines include positions 9, 8, 7 and 6, and the template for any one of positions 9, 8, 7 and 6 includes 5 positions, as shown in 10, 11, 12, 13 and 14.

[0110] exist Figure 10 In the schematic diagram (1050), the scan lines include positions 12, 11 and 10, and the template for any one of positions 12, 11 and 10 includes three positions, as shown in 13, 14 and 15.

[0111] exist Figure 10 In the schematic diagram (1060), the scan lines include positions 14 and 13, and the template for either position 14 or 13 includes one position, as shown in 15.

[0112] In another embodiment, the template used for context deduction is associated with the scan line and position (within the scan line).

[0113] Figure 11 Schematic diagrams of transformation units with scan-line and position-related templates for context derivation are shown in some examples.

[0114] exist Figure 11 In the example, the transformation unit is 4×4 and includes 0 to 15 scan positions. The transformation unit is scanned in the upper right diagonal scan order (e.g., in the order of 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 and 0).

[0115] exist Figure 11 In the schematic diagram (1131), the scan lines include positions 5, 4 and 3, and the current scan position is 3. The template for the current position 3 includes 5 positions, as shown in 6, 7, 8, 10 and 11.

[0116] exist Figure 11 In the schematic diagram (1132), the scan lines include positions 5, 4 and 3, and the current scan position is 4. The template for the current position 4 includes 5 positions, as shown in 6, 7, 8, 9 and 11.

[0117] exist Figure 11 In the schematic diagram (1133), the scan lines include positions 5, 4 and 3, and the current scan position is 5. The template for the current position 5 includes 5 positions, as shown in 7, 8, 9, 11 and 12.

[0118] In some embodiments, the values ​​of N, M, and K are position-dependent. The values ​​of N, M, and K depend on the position (x, y) of the currently resolved transform coefficient. This (x, y) corresponds to the Euclidean coordinates of the coefficient relative to the top-left sample position in the transform unit (transform block).

[0119] In the embodiment, the set This can depend on the coefficient region. For the low-frequency region, Larger numbers can be used; for the mid-frequency range, Smaller values ​​can be used compared to the low-frequency region; for the high-frequency region, smaller parameter values ​​can be used compared to the mid-frequency region.

[0120] In some examples, this low-frequency region can be defined as the first three coefficients in reverse order of the encoding, for example... Figure 9 In the example, positions 0, 1, and 2 are in a 4×4 block; this intermediate frequency region can be defined as the 4th to 10th coefficients in reverse order of the coding sequence, for example... Figure 9 In the example, positions 3 to 9 are in a 4×4 block; and this high-frequency region can be defined as all subsequent coefficients in reverse order of the coding order, for example... Figure 9 Positions 10 to 15 in the 4×4 block in the example.

[0121] In one example, the low-frequency, mid-frequency, and high-frequency regions can depend on the block size.

[0122] In some embodiments, the values ​​of N, M, and K may include, but are not limited to, elements of the set {1, 2, 3, 4, 5, ... 15}.

[0123] Figure 12 A flowchart of an overview process (1200) according to an embodiment of the present disclosure is shown. Process (1200) can be used in a video decoder. In various embodiments, process (1200) is executed by processing circuitry, such as processing circuitry that performs the functions of video decoder (110), processing circuitry that performs the functions of video decoder (210), etc. In some embodiments, process (1200) is implemented as software instructions, so that when processing circuitry executes the software instructions, processing circuitry executes process (1200). The process begins at (S1201) and proceeds to (S1210).

[0124] In (S1210), an encoded video stream is received. The encoded video stream includes encoding information of the transform block of the current block in the current image. The encoding information of the transform block indicates the quantization transform coefficient level at each position of the transform block, each position being associated with a template region for context inference. The template region associated with a position is an aligned template region for context inference of a set of latent syntax elements used to encode the quantization transform coefficient level at that position. The set of latent syntax elements includes: at least one first latent syntax element for encoding the basic range level, at least one second latent syntax element for encoding the lower range level, and at least one third latent syntax element for encoding the higher range level.

[0125] In (S1220), for a first position of the transform block, at least one context model for at least one syntax element is determined based on a first template region associated with the first position, the at least one syntax element indicating a first quantization transform coefficient level at the first position.

[0126] In (S1230), the at least one syntax element is decoded from the encoding information of the transform block according to the at least one context model.

[0127] In (S1240), the first quantization transformation coefficient level at the first position is determined based on the at least one syntax element.

[0128] In (S1250), the transform block is reconstructed at least according to the first quantization transform coefficient level at the first position of the transform block.

[0129] In some examples, the transform block is dequantized, and then an inverse transform is performed on the dequantized transform block to compute the residual block. Furthermore, the residual block is combined with the prediction block to reconstruct the current block.

[0130] In some embodiments, the set of latent syntax elements is used to encode at least one of the following: inseparable principal transform coefficients; and inseparable quadratic transform coefficients.

[0131] In some embodiments, the first template region associated with the first position includes at least two positions having decoded transform coefficient information, the at least two positions being decoded before the first position in the encoding order.

[0132] In one embodiment, a first context model for encoding a first syntax element of the basic range level is determined based on decoded transform coefficient information at a first set of positions in the first template region. In another embodiment, a second context model for encoding a second syntax element of the lower range level is determined based on decoded transform coefficient information at a second set of positions in the first template region. In yet another embodiment, a third context model for encoding a third syntax element of the higher range level is determined based on decoded transform coefficient information at a third set of positions in the first template region.

[0133] In some examples, the first group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order; the second group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order; and the third group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order.

[0134] In some examples, the first set of positions is a series of positions other than the position closest to the first position in the encoding order; the second set of positions is a series of positions other than the position closest to the first position in the encoding order; and the third set of positions is a series of positions other than the position closest to the first position in the encoding order.

[0135] In some examples, the first group of positions includes 5 positions; the second group of positions includes 3 positions; and the third group of positions includes 3 positions.

[0136] In some examples, the template region is associated with a scan line. For instance, when the first position and the second position are located on the same scan line among multiple scan lines, the first template region associated with the first position and the second template region associated with the second position are the same region, the multiple scan lines being used to scan the quantization transform coefficient levels in the transform block in encoding order.

[0137] In some examples, the template region is associated with scan lines and positions. For instance, when the first position and the second position are located at different positions on one of a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are different regions, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in encoding order.

[0138] In some embodiments, a first number of the first group of positions, a second number of the second group of positions, and a third number of the third group of positions are set based on the first position.

[0139] In some examples, when the first position is located in a first frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; when the first position is located in a second frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; and when the frequency of the first frequency region is lower than the frequency of the second frequency region, the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values.

[0140] In some examples, when the first position is located in a low-frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; when the first position is located in a mid-frequency region with a frequency higher than the low-frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; when the first position is located in a high-frequency region with a frequency higher than the mid-frequency region, the first quantity, the second quantity, and the third quantity have a third set of values; the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values; and the second set of values ​​are respectively greater than the corresponding values ​​in the third set of values.

[0141] In some examples, the low-frequency region includes a first consecutive position in reverse order of the encoding order; the mid-frequency region includes a second consecutive position in reverse order of the encoding order following the first consecutive position; and the high-frequency region includes a third consecutive position in reverse order of the encoding order following the second consecutive position.

[0142] In some examples, the low-frequency region, the mid-frequency region, and the high-frequency region are defined based on the block size of the transform block.

[0143] In some examples, the first quantity, the second quantity, and the third quantity are elements from a set of predefined values.

[0144] Then, the process proceeds to (S1299) and ends.

[0145] The process (1200) can be adjusted as appropriate. At least one step in the process (1200) can be modified and / or omitted. At least one additional step can be added. Any suitable implementation order can be used.

[0146] Figure 13A flowchart of an overview process (1300) according to an embodiment of the present disclosure is shown. Process (1300) can be used in a video encoder. In various embodiments, process (1300) is executed by processing circuitry, such as processing circuitry that performs the functions of a video encoder (103), processing circuitry that performs the functions of a video encoder (303), etc. In some embodiments, process (1300) is implemented as software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes process (1300). The process begins at (S1301) and proceeds to (S1310).

[0147] In (S1310), the residual block of the current block in the current image is calculated. For example, the residual block is calculated as the difference between the current block and the predicted block.

[0148] In (S1320), the residual block is converted into a transform block, the transform block including quantized transform coefficient levels at various locations within the transform block. For example, a transform is applied to the residual block to generate transform coefficients, and quantization is applied to the transform coefficients to generate quantized transform coefficient levels. Each location is associated with a template region for context inference, the template region associated with a location being an aligned template region for context inference of a set of latent syntax elements used to encode the quantized transform coefficient levels at that location. The set of latent syntax elements includes: at least one first latent syntax element for encoding a basic range level, at least one second latent syntax element for encoding a lower range level, and at least one third latent syntax element for encoding a higher range level.

[0149] At (S1330), for a first position of the transform block, at least one context model for at least one syntax element is determined based on a first template region associated with the first position. The at least one syntax element indicates a first quantization transform coefficient level at the first position.

[0150] In (S1340), the at least one syntax element is encoded into encoded information in the bitstream according to the at least one context model.

[0151] In some embodiments, the set of latent syntax elements is used to encode at least one of the following: inseparable principal transform coefficients; and inseparable quadratic transform coefficients.

[0152] In some embodiments, the first template region associated with the first position includes at least two positions having transform coefficient information, the at least two positions being encoded in an encoding order preceding the first position.

[0153] In one embodiment, a first context model for encoding a first syntax element of the basic range level is determined based on the transform coefficient information of a first set of positions in the first template region. In another embodiment, a second context model for encoding a second syntax element of the lower range level is determined based on the transform coefficient information of a second set of positions in the first template region. In yet another embodiment, a third context model for encoding a third syntax element of the higher range level is determined based on the transform coefficient information of a third set of positions in the first template region.

[0154] In some examples, the first group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order; the second group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order; and the third group of positions is a series of consecutive positions, including the position closest to the first position in the encoding order.

[0155] In some examples, the first set of positions is a series of positions other than the position closest to the first position in the encoding order; the second set of positions is a series of positions other than the position closest to the first position in the encoding order; and the third set of positions is a series of positions other than the position closest to the first position in the encoding order.

[0156] In some examples, the first group of positions includes 5 positions; the second group of positions includes 3 positions; and the third group of positions includes 3 positions.

[0157] In some embodiments, the template region is associated with a scan line. For example, when the first position and the second position are located on the same scan line among multiple scan lines, the first template region associated with the first position and the second template region associated with the second position are the same region, the multiple scan lines being used to scan the quantization transform coefficient levels in the transform block in encoding order.

[0158] In some embodiments, the template region is associated with scan lines and positions. For example, when the first position and the second position are located at different positions on one of a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are different regions, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in encoding order.

[0159] In some embodiments, a first number of the first group of positions, a second number of the second group of positions, and a third number of the third group of positions are set based on the first position.

[0160] In some embodiments, when the first position is located in a first frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; when the first position is located in a second frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; and when the frequency of the first frequency region is lower than the frequency of the second frequency region, the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values.

[0161] In some embodiments, when the first position is located in a low-frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; when the first position is located in a mid-frequency region with a frequency higher than the low-frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; when the first position is located in a high-frequency region with a frequency higher than the mid-frequency region, the first quantity, the second quantity, and the third quantity have a third set of values; the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values; and the second set of values ​​are respectively greater than the corresponding values ​​in the third set of values.

[0162] In some embodiments, the low-frequency region includes a first consecutive position in reverse order of the encoding order; the mid-frequency region includes a second consecutive position following the first consecutive position in reverse order of the encoding order; and the high-frequency region includes a third consecutive position following the second consecutive position in reverse order of the encoding order.

[0163] In some embodiments, the low-frequency region, the mid-frequency region, and the high-frequency region are defined based on the block size of the transform block.

[0164] In some embodiments, the first quantity, the second quantity, and the third quantity are elements of a set of predefined values.

[0165] Then, the process proceeds to (S1399) and ends.

[0166] The process (1300) can be adjusted as appropriate. At least one step in the process (1300) can be modified and / or omitted. At least one additional step can be added. Any suitable implementation order can be used.

[0167] According to embodiments of this disclosure, a method for processing visual media data is provided. In this method, a conversion between a visual media file and a bitstream of visual media data is performed according to format rules. For example, the bitstream may be a bitstream decoded / encoded using any of the decoding and / or encoding methods described herein. The format rules may specify at least one constraint on the bitstream and / or at least one process to be performed by the decoder and / or encoder.

[0168] In the example, the bitstream includes encoding information for the transform block of the current block in the current image. The encoding information for the transform block indicates the quantization transform coefficient level at various locations within the transform block. Each location is associated with a template region for context inference. The template region associated with a location is an aligned template region for context inference of a set of latent syntax elements used to encode the quantization transform coefficient level at that location. This set of latent syntax elements includes at least one first latent syntax element for encoding a basic range level, at least one second latent syntax element for encoding a lower range level, and at least one third latent syntax element for encoding a higher range level. The formatting rules specify that for a first location of the transform block, at least one context model is determined based on the first template region associated with the first location, each for at least one syntax element indicating a first quantization transform coefficient level at the first location. The format rules also specify that the at least one syntax element is decoded from the encoding information of the transform block according to the at least one context model; a first quantization transform coefficient level at the first position is determined based on the at least one syntax element; and the transform block is reconstructed at least based on the first quantization transform coefficient level at the first position of the transform block. The transform block can be dequantized, and an inverse transform can be applied to the dequantized transform block to compute the residual block of the current block. The residual block can be combined with the prediction block to generate the reconstructed current block.

[0169] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored on at least one computer-readable medium. For example, Figure 14 A computer system (1400) is shown, which is adapted to implement certain embodiments of the disclosed subject matter.

[0170] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by at least one computer central processing unit (CPU), graphics processing unit (GPU), or through decoding, microcode, or other means.

[0171] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0172] Figure 14The components shown for the computer system (1400) are exemplary and are not intended to limit the scope or functionality of the computer software used to implement embodiments of this disclosure. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of the computer system (1400).

[0173] The computer system (1400) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from at least one human user through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media, which need not be directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0174] The human-machine interface input device may include at least one of the following (only one is shown): keyboard (1401), mouse (1402), touchpad (1403), touch screen (1410), data glove (not shown), joystick (1405), microphone (1406), scanner (1407), camera (1408).

[0175] The computer system (1400) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of at least one human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1410), data gloves (not shown), or joystick (1405), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1409), headphones (not shown)), visual output devices (e.g., screens (1410) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each of which may or may not have touchscreen input functionality, each of which may or may not have tactile feedback functionality—some of which may output two-dimensional or more three-dimensional visual outputs by means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0176] The computer system (1400) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (1420) or similar media (1421) with CD / DVD, thumb drives (1422), removable hard disk drives or solid-state drives (1423), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), and so on.

[0177] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0178] The computer system (1400) may also include an interface (1454) to at least one communication network (1455). For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), wide area network (WAN), metropolitan area network (MAN), vehicular and industrial network, real-time network, latency-tolerant network, etc. Networks also include Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicular and industrial networks (including CANBus), etc. Some networks typically require an external network interface adapter for connection to certain general-purpose data ports or peripheral buses (1449) (e.g., a USB port on the computer system (1400); other systems are typically integrated into the core of the computer system (1400) via a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (1400) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0179] The aforementioned human-computer interface device, human-accessible storage device, and network interface can be connected to the core (1440) of the computer system (1400).

[0180] The core (1440) may include at least one central processing unit (CPU) (1441), a graphics processing unit (GPU) (1442), a dedicated programmable processing unit in the form of a field-programmable gate array (FPGA) (1443), a task-specific hardware accelerator (1444), a graphics adapter (1450), etc. These devices, as well as read-only memory (ROM) (1445), random access memory (1446), internal mass storage (e.g., internal non-user-accessible hard disk drive, solid-state drive, etc.) (1447), etc., may be connected via a system bus (1448). In some computer systems, the system bus (1448) may be accessed in the form of at least one physical plug so that it can be expanded with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (1448) or connected via a peripheral bus (1449). In the example, a screen (1410) may be connected to the graphics adapter (1450). The architecture of the peripheral bus includes external peripheral component interconnect (PCI), universal serial bus (USB), etc.

[0181] The CPU (1441), GPU (1442), FPGA (1443), and accelerator (1444) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1445) or RAM (1446). Transient data can also be stored in RAM (1446), while permanent data can be stored, for example, in internal mass storage (1447). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with at least one CPU (1441), GPU (1442), mass storage (1447), ROM (1445), RAM (1446), etc.

[0182] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0183] By way of example and not limitation, a computer system having an architecture (1400), particularly a core (1440), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in at least one tangible computer-readable medium. Such a computer-readable medium can be a medium associated with the aforementioned user-accessible mass storage, as well as specific memory of the non-volatile core (1440), such as internal mass storage (1447) or ROM (1445). Software implementing various embodiments of this disclosure can be stored in such a device and executed by the core (1440). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (1440), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to software-defined processes. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (1444)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This disclosure includes any suitable combination of hardware and software.

[0184] As used in this disclosure, "at least one" or "one of" is intended to include any one or a combination of the listed elements. For example, references to at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A through C are intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include either A or B or (A and B). Where applicable, the use of "one of" does not exclude any combination of the listed elements, for example, when the elements are not mutually exclusive.

[0185] While at least two exemplary embodiments have been described in this disclosure, various modifications, arrangements, and equivalent substitutions of the embodiments are within the scope of this disclosure. Therefore, it should be understood that those skilled in the art can design various systems and methods that, while not explicitly shown or described herein, embody the principles of this disclosure and are thus within its spirit and scope.

[0186] The above disclosure also includes the following features. These features can be combined in various ways, and are not limited to the combinations mentioned below.

[0187] (1) A video decoding method, comprising: receiving an encoded video stream, the encoded video stream including encoding information of a transform block of a current block in a current image, the encoding information of the transform block indicating the quantization transform coefficient level at each position of the transform block, each position being associated with a template region for context inference, the template region associated with a position being an aligned template region for context inference of a set of latent syntax elements, the set of latent syntax elements being used to encode the quantization transform coefficient level at the position, the set of latent syntax elements including: at least one first latent syntax element for encoding a basic range level, and for encoding a lower range level. The transformation block is configured with at least one second latent syntax element and at least one third latent syntax element for encoding high-range levels; for a first position of the transform block, at least one context model is determined based on a first template region associated with the first position, each context model indicating a first quantization transform coefficient level at the first position; the at least one syntax element is decoded from the encoding information of the transform block based on the at least one context model; the first quantization transform coefficient level at the first position is determined based on the at least one syntax element; and the transform block is reconstructed based at least on the first quantization transform coefficient level at the first position of the transform block.

[0188] (2) The method according to feature (1), wherein the set of latent syntax elements is used to encode at least one of the following: inseparable principal transform coefficients; and inseparable secondary transform coefficients.

[0189] (3) The method according to any one of features (1) to (2), wherein the first template region associated with the first position includes at least two positions having decoded transform coefficient information, the at least two positions being decoded before the first position in the encoding order.

[0190] (4) The method according to any one of features (1) to (3), wherein determining at least one context model comprises at least one of the following: determining a first context model for encoding a first syntax element for the base range level based on decoded transform coefficient information of a first set of positions in the first template region; determining a second context model for encoding a second syntax element for the lower range level based on decoded transform coefficient information of a second set of positions in the first template region; and determining a third context model for encoding a third syntax element for the higher range level based on decoded transform coefficient information of a third set of positions in the first template region.

[0191] (5) The method according to any one of features (1) to (4), wherein the first group of positions is a series of positions, including the position closest to the first position in the encoding order; the second group of positions is a series of positions, including the position closest to the first position in the encoding order; and the third group of positions is a series of positions, including the position closest to the first position in the encoding order.

[0192] (6) The method according to any one of features (1) to (5), wherein the first set of positions is a series of positions other than the position closest to the first position in the encoding order; the second set of positions is the series of positions other than the position closest to the first position in the encoding order; and the third set of positions is the series of positions other than the position closest to the first position in the encoding order.

[0193] (7) The method according to any one of features (1) to (6), wherein the first group of positions comprises 5 positions; the second group of positions comprises 3 positions; and the third group of positions comprises 3 positions.

[0194] (8) The method according to any one of features (1) to (7), wherein when the first position and the second position are located on the same scan line among a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are the same region, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in the encoding order.

[0195] (9) The method according to any one of features (1) to (8), wherein when the first position and the second position are at different positions on one of a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are different regions, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in the encoding order.

[0196] (10) The method according to any one of features (1) to (9), wherein a first number of the first group of positions, a second number of the second group of positions and a third number of the third group of positions are set based on the first position.

[0197] (11) The method according to any one of features (1) to (10), wherein when the first position is located in a first frequency region, the first quantity, the second quantity and the third quantity have a first set of values; when the first position is located in a second frequency region, the first quantity, the second quantity and the third quantity have a second set of values; and when the frequency of the first frequency region is lower than the frequency of the second frequency region, the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values.

[0198] (12) The method according to any one of features (1) to (11), wherein when the first position is located in a low-frequency region, the first quantity, the second quantity and the third quantity have a first set of values; when the first position is located in a mid-frequency region with a frequency higher than that of the low-frequency region, the first quantity, the second quantity and the third quantity have a second set of values; when the first position is located in a high-frequency region with a frequency higher than that of the mid-frequency region, the first quantity, the second quantity and the third quantity have a third set of values; the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values; and the second set of values ​​are respectively greater than the corresponding values ​​in the third set of values.

[0199] (13) The method according to any one of features (1) to (12), wherein the low-frequency region includes a first consecutive position in reverse order of the encoding order; the mid-frequency region includes a second consecutive position in reverse order of the encoding order following the first consecutive position; and the high-frequency region includes a third consecutive position in reverse order of the encoding order following the second consecutive position.

[0200] (14) The method according to any one of features (1) to (13), wherein the low-frequency region, the mid-frequency region and the high-frequency region are defined based on the block size of the transform block.

[0201] (15) The method according to any one of features (1) to (14), wherein the first quantity, the second quantity and the third quantity are elements in a set of predefined values.

[0202] (16) A video coding method, comprising: calculating a residual block of a current block in a current image; converting the residual block into a transform block, the transform block including quantization transform coefficient levels at various locations of the transform block, each location being associated with a respective template region for context inference, the template region associated with a location being an aligned template region for context inference of a set of latent syntax elements for encoding the quantization transform coefficient levels at the location, the set of latent syntax elements including: at least one first latent syntax element for encoding a base range level, at least one second latent syntax element for encoding a low range level, and at least one third latent syntax element for encoding a high range level; for a first location of the transform block, determining at least one context model for at least one syntax element based on a first template region associated with the first location, the at least one syntax element indicating a first quantization transform coefficient level at the first location; and encoding the at least one syntax element into encoded information in a bitstream based on the at least one context model.

[0203] (17) The method according to feature (16), wherein the set of latent syntax elements is used to encode at least one of the following: inseparable principal transform coefficients; and inseparable secondary transform coefficients.

[0204] (18) The method according to any one of features (16) to (17), wherein the first template region associated with the first position includes at least two positions having transform coefficient information, the at least two positions being encoded in an encoding order prior to the first position.

[0205] (19) The method according to any one of features (16) to (18), wherein determining at least one context model comprises at least one of the following: determining a first context model for encoding a first syntax element for the base range level based on transform coefficient information of a first set of positions in the first template region; determining a second context model for encoding a second syntax element for the lower range level based on transform coefficient information of a second set of positions in the first template region; and determining a third context model for encoding a third syntax element for the higher range level based on transform coefficient information of a third set of positions in the first template region.

[0206] (20) The method according to any one of features (16) to (19), wherein the first group of positions is a series of positions including the position closest to the first position in the encoding order; the second group of positions is a series of positions including the position closest to the first position in the encoding order; and the third group of positions is a series of positions including the position closest to the first position in the encoding order.

[0207] (21) The method according to any one of features (16) to (20), wherein the first set of positions is a series of positions other than the position closest to the first position in the encoding order; the second set of positions is the series of positions other than the position closest to the first position in the encoding order; and the third set of positions is the series of positions other than the position closest to the first position in the encoding order.

[0208] (22) The method according to any one of features (16) to (21), wherein the first group of positions comprises 5 positions; the second group of positions comprises 3 positions; and the third group of positions comprises 3 positions.

[0209] (23) The method according to any one of features (16) to (22), wherein when the first position and the second position are located on the same scan line among a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are the same region, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in the encoding order.

[0210] (24) The method according to any one of features (16) to (23), wherein when the first position and the second position are at different positions on one of a plurality of scan lines, the first template region associated with the first position and the second template region associated with the second position are different regions, the plurality of scan lines being used to scan the quantization transform coefficient levels in the transform block in an encoding order.

[0211] (25) The method according to any one of features (16) to (24), wherein a first number of the first group of positions, a second number of the second group of positions and a third number of the third group of positions are set based on the first position.

[0212] (26) The method according to any one of features (16) to (25), wherein when the first position is located in a first frequency region, the first quantity, the second quantity and the third quantity have a first set of values; when the first position is located in a second frequency region, the first quantity, the second quantity and the third quantity have a second set of values; and when the frequency of the first frequency region is lower than the frequency of the second frequency region, the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values.

[0213] (27) The method according to any one of features (16) to (26), wherein when the first position is located in a low-frequency region, the first quantity, the second quantity and the third quantity have a first set of values; when the first position is located in a mid-frequency region with a frequency higher than that of the low-frequency region, the first quantity, the second quantity and the third quantity have a second set of values; when the first position is located in a high-frequency region with a frequency higher than that of the mid-frequency region, the first quantity, the second quantity and the third quantity have a third set of values; the first set of values ​​are respectively greater than the corresponding values ​​in the second set of values; and the second set of values ​​are respectively greater than the corresponding values ​​in the third set of values.

[0214] (28) The method according to any one of features (16) to (27), wherein the low-frequency region includes a first consecutive position in reverse order of the encoding order; the mid-frequency region includes a second consecutive position in reverse order of the encoding order following the first consecutive position; and the high-frequency region includes a third consecutive position in reverse order of the encoding order following the second consecutive position.

[0215] (29) The method according to any one of features (16) to (28), wherein the low-frequency region, the mid-frequency region and the high-frequency region are defined based on the block size of the transform block.

[0216] (30) The method according to any one of features (16) to (29), wherein the first quantity, the second quantity and the third quantity are elements in a set of predefined values.

[0217] (31) A method for storing a video stream, comprising performing the video encoding method of any one of features (16) to (19) to generate a video stream; and storing the video stream.

[0218] (32) A non-volatile computer-readable storage medium storing a bitstream, the bitstream being encoded by a video coding method, the video coding method comprising: calculating a residual block of a current block in a current image; converting the residual block into a transform block, the transform block comprising quantized transform coefficient levels at various positions of the transform block, each position being associated with a template region for context inference, the template region associated with a position being an aligned template region for context inference of a set of latent syntax elements, the set of latent syntax elements being used to encode the quantized transform coefficient levels at the position, the set of latent syntax elements comprising: encoding a base range level. The transformation block comprises at least one first latent syntax element, at least one second latent syntax element for encoding a low-range level, and at least one third latent syntax element for encoding a high-range level; for a first position of the transform block, at least one context model is determined based on a first template region associated with the first position, each for at least one syntax element indicating a first quantization transform coefficient level at the first position; the at least one syntax element is encoded into encoding information in a bitstream based on the at least one context model; and the bitstream is transmitted, the bitstream including encoding information of at least one image, the at least one image including the current image.

[0219] (33) A video decoding apparatus, comprising processing circuitry configured to perform the method of any one of features (1) to (15).

[0220] (34) A video encoding apparatus, including processing circuitry, configured to perform the method of any one of features (16) to (30).

[0221] (35) A non-volatile computer-readable storage medium for storing instructions that, when executed by at least one processor, cause the at least one processor to perform a video encoding method of any one of features (16) to (30) to generate a bitstream, and to store and transmit the bitstream.

[0222] (36) A non-volatile computer-readable storage medium for storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of features (1) to (31).

Claims

1. A method for video decoding, characterized in that, include: The system receives an encoded video stream, which includes encoding information of the transform block of the current block in the current image. The encoding information of the transform block indicates the quantization transform coefficient level at each position of the transform block. Each position is associated with a template region for context inference. The template region associated with a position is an aligned template region for context inference of a set of latent syntax elements. The set of latent syntax elements is used to encode the quantization transform coefficient level at the position. The set of latent syntax elements includes: at least one first latent syntax element for encoding the basic range level, at least one second latent syntax element for encoding the low range level, and at least one third latent syntax element for encoding the high range level. For a first position of the transform block, at least one context model is determined for at least one syntax element based on a first template region associated with the first position, the at least one syntax element indicating a first quantization transform coefficient level at the first position; Decode the at least one syntax element from the encoding information of the transform block according to the at least one context model; Determine the first quantization transform coefficient level at the first position based on the at least one syntax element; and The transform block is reconstructed at least based on the first quantization transform coefficient level at the first position of the transform block.

2. The method according to claim 1, characterized in that, The set of potential syntax elements is used to encode at least one of the following: Inseparable principal transform coefficients; and Inseparable quadratic transformation coefficients.

3. The method according to claim 1, characterized in that, The first template region associated with the first position includes at least two positions with decoded transform coefficient information, which are decoded in the encoding order before the first position.

4. The method according to claim 3, characterized in that, Determining at least one context model includes at least one of the following: Based on the decoded transform coefficient information of the first set of positions in the first template region, a first context model for encoding the first syntax element of the basic range level is determined; Based on the decoded transform coefficient information of the second set of positions in the first template region, a second context model for encoding the second syntax element of the lower range level is determined; and Based on the decoded transform coefficient information of the third set of positions in the first template region, a third context model for encoding the third syntax element of the higher range level is determined.

5. The method according to claim 4, characterized in that, The first group of positions is a continuous sequence of positions, including the position closest to the first position according to the encoding order; The second group of positions consists of consecutive positions, including the position closest to the first position according to the encoding order; and The third group of positions is a series of consecutive positions, including the position that is closest to the first position according to the encoding order.

6. The method according to claim 4, characterized in that, The first group of positions consists of consecutive positions excluding the position closest to the first position according to the encoding order; The second group of positions consists of consecutive positions excluding the position closest to the first position according to the encoding order; and The third set of positions refers to the consecutive positions other than the position closest to the first position according to the encoding order.

7. The method according to claim 4, characterized in that, The first group of positions includes 5 positions; The second group of positions includes 3 positions; and The third group of positions includes three positions.

8. The method according to claim 1, characterized in that, When the first position and the second position are located on the same scan line among multiple scan lines, the first template region associated with the first position and the second template region associated with the second position are the same region, and the multiple scan lines are used to scan the quantization transform coefficient levels in the transform block in the encoding order.

9. The method according to claim 1, characterized in that, When the first position and the second position are located at different positions on one of the multiple scan lines, the first template region associated with the first position and the second template region associated with the second position are different regions, and the multiple scan lines are used to scan the quantization transform coefficient levels in the transform block in the encoding order.

10. The method according to claim 4, characterized in that, Based on the first position, set a first quantity of the first group of positions, a second quantity of the second group of positions, and a third quantity of the third group of positions.

11. The method according to any one of claims 4 to 10, characterized in that, When the first position is located in the first frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; When the first position is located in the second frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; and When the frequency of the first frequency region is lower than the frequency of the second frequency region, the first set of values ​​is greater than the corresponding value in the second set of values.

12. The method according to any one of claims 4 to 10, characterized in that, When the first position is located in a low-frequency region, the first quantity, the second quantity, and the third quantity have a first set of values; When the first position is located in a mid-frequency region with a higher frequency than the low-frequency region, the first quantity, the second quantity, and the third quantity have a second set of values; When the first position is located in a high-frequency region with a higher frequency than the intermediate frequency region, the first quantity, the second quantity, and the third quantity have a third set of values; The first set of values ​​is greater than the corresponding values ​​in the second set of values; and The second set of values ​​is greater than the corresponding values ​​in the third set of values.

13. The method according to claim 12, characterized in that, The low-frequency region includes a first consecutive position in reverse order of the encoding order; The intermediate frequency region includes a second consecutive position following the first consecutive position in reverse order of the encoding order; and The high-frequency region includes a third consecutive position following the second consecutive position in reverse order of the encoding order.

14. The method according to claim 12, characterized in that, The low-frequency region, the mid-frequency region, and the high-frequency region are defined based on the block size of the transform block.

15. The method according to claim 12, characterized in that, The first quantity, the second quantity, and the third quantity are elements in a set of predefined values.

16. A video encoding method, characterized in that, include: Calculate the residual block of the current block in the current image; The residual block is transformed into a transform block, the transform block including quantized transform coefficient levels at various locations, each location being associated with a template region for context inference. The template region associated with a location is an aligned template region for context inference of a set of latent syntax elements used to encode the quantized transform coefficient levels at the location. The set of latent syntax elements includes: at least one first latent syntax element for encoding a base range level, at least one second latent syntax element for encoding a lower range level, and at least one third latent syntax element for encoding a higher range level. For a first position of the transform block, at least one context model is determined based on a first template region associated with the first position, each for at least one syntax element, the at least one syntax element indicating a first quantization transform coefficient level at the first position; and The at least one syntax element is encoded into encoded information in the bitstream according to the at least one context model.

17. The method according to claim 16, characterized in that, The set of potential syntax elements is used to encode at least one of the following: Inseparable principal transform coefficients; and Inseparable quadratic transformation coefficients.

18. The method according to claim 16, characterized in that, The first template region associated with the first position includes at least two positions with transform coefficient information, which are encoded in the order of encoding before the first position.

19. The method according to any one of claims 16 to 18, wherein, Determining at least one context model includes at least one of the following: Based on the transformation coefficient information of the first set of positions in the first template region, a first context model for encoding the first syntax element of the basic range level is determined; Based on the transformation coefficient information of the second set of positions in the first template region, a second context model for encoding the second syntax element of the lower range level is determined; and Based on the transformation coefficient information of the third set of positions in the first template region, a third context model for encoding the third syntax element of the high range level is determined.

20. A method for storing video streams, characterized in that, The video encoding method according to any one of claims 16-19 is used to generate a video stream; and the video stream is stored.

21. A video decoding apparatus, characterized in that, Includes processing circuitry configured to perform the video decoding method according to any one of claims 1 to 15.

22. A video encoding apparatus, characterized in that, Includes processing circuitry configured to perform the video encoding method according to any one of claims 16 to 19.

23. A non-volatile computer-readable storage medium, characterized in that, The device stores instructions that, when executed by at least one processor, cause the at least one processor to perform the video encoding method according to any one of claims 16 to 19 to generate a bitstream and to store and transmit the bitstream.

Images