Video decoding method and device, electronic device, and storage medium

Abstract

A method of video decoding can include receiving a bitstream including encoded bits of a plurality of sections of syntax elements of transform coefficients in a transform block of a coded picture, the syntax elements indicating a property of the transform coefficients; for each transform coefficient in a given frequency region of the transform block, performing context modeling to determine a context model for the plurality of sections of the syntax elements; and based on the context models determined for the plurality of sections of the syntax elements, decoding the coded bitstream to determine a section of the syntax elements.

Description

[0001] This application claims priority to U.S. Provisional Application No. 62 / 735,405, filed September 24, 2018; U.S. Provisional Application No. 62 / 741,524, filed October 4, 2018; U.S. Provisional Application No. 62 / 798,405, filed January 29, 2019; and U.S. Application No. 16 / 541,367, filed August 15, 2019, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0002] This application relates to video decoding technology, and more particularly to video decoding methods, apparatus, electronic devices, and storage media. Background Technology

[0003] The background description provided herein is intended to state the scope of this application. For the purposes of this background section, and in respect of aspects that may not conform to the prior art at the time of filing, the work of the currently attributed inventors is neither expressly nor implicitly acknowledged as prior art to this application.

[0004] Inter-frame prediction with motion compensation can be used for video encoding and decoding. Uncompressed digital video can comprise a series of images, each with a spatial dimension of, for example, 1920×1080 luma samples and associated chroma samples. This series of images can have a fixed or variable image rate (also informally referred to as the frame rate), such as 60 images per second or 60Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luma sample resolution at a frame rate of 60Hz) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 gigabytes of storage space.

[0005] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression can help reduce bandwidth or storage space requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations of both can be employed. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion that can be tolerated depends on the application; for example, users of some consumer streaming applications may tolerate higher distortion compared to users of television distribution applications. The achievable compression ratio reflects the higher the allowable / tolerable distortion that can produce a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including motion compensation, transform, quantization, and entropy coding.

[0007] Video codec techniques may include techniques known as intra-frame coding. In intra-frame coding, sample values ​​are not referenced to samples or other data representations from previously reconstructed reference images. In some video codecs, images are spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the image can be an intra-frame coded image. Intra-frame coded images and their derivatives, such as stand-alone decoder refresh images, can be used to reset the decoder state and thus can be used as the first image in an encoded video bitstream and video session, or as a still image. Samples of intra-frame coded blocks can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and AC coefficients after the transform, the fewer bits are needed to represent the block after entropy coding for a given quantization step size.

[0008] Traditional intra-frame coding techniques, such as those derived from MPEG-2 generation coding, do not use intra-frame prediction. However, some newer video compression techniques include those that attempt to use surrounding sample data and / or metadata acquired during encoding / decoding, for example, from spatially adjacent data blocks that are decoded first. Such techniques are hereinafter referred to as "intra-frame prediction" techniques. Note that in at least some cases, intra-frame prediction uses only reference data from the reconstructed current frame and not reference data from a reference frame.

[0009] Intra-frame prediction can take many different forms. When more than one such technique can be used in a given video coding technique, the techniques in use can be encoded in an intra-frame prediction mode. In some cases, a mode may have sub-modes and / or parameters, and those can be encoded individually or included in the mode codeword. Which codeword is used for a given combination of mode / sub-mode / parameter can affect the coding efficiency gain through intra-frame prediction, and therefore entropy coding techniques can be used to convert the codeword into a bitstream.

[0010] Intra-frame prediction was introduced with H.264, refined in H.265, and further improved in newer coding techniques such as the Jointly Developed Model (JEM), Universal Video Coding (VVC), and Baseline Sets (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the predictor block according to the orientation. References to the orientation in use can be encoded in the bitstream or can be predicted themselves.

[0011] Motion compensation can be a lossy compression technique and can involve techniques in which blocks of sample data from a previously reconstructed image or a portion of that image (reference image) are spatially shifted in the direction indicated by a motion vector (hereinafter referred to as MV) to predict the most recently reconstructed image or a portion of that image. In some cases, the reference image can be the same as the image currently being reconstructed. MV can have two dimensions, X and Y, or three dimensions, the third dimension being an indication of the reference image in use (indirectly, the latter can be a temporal dimension).

[0012] In some video compression techniques, the MV applicable to a region of the sample data can be predicted from other MVs (e.g., those MVs related to another region of sample data adjacent to the reconstructed region space and preceding the MV applicable to a region of the sample data in decoding order). This significantly reduces the amount of data required to encode the MV, thereby removing redundancy and improving compression. MV prediction can work effectively, for example, because when encoding an input video signal from a camera (referred to as natural video), there is a statistical probability that a larger region than the region applicable to a single MV moves in a similar direction, and therefore, in some cases, similar motion vectors from MVs from neighboring regions can be used for prediction. This results in the MV found for a given region being similar or identical to the MV predicted from surrounding MVs, and in turn, after entropy coding, it can be represented with a smaller number of bits compared to the number of bits required to directly encode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself may be lossy, for example, due to rounding errors when calculating predictions from several surrounding MVs.

[0013] To reduce the number of context models used for transform coefficient encoding, related techniques use different rules for different frequency regions to share context models. However, this irregular design is not hardware- or software-friendly, reducing the efficiency of video encoding and decoding. Summary of the Invention

[0014] Various embodiments of this application provide a video decoding method. The method may include receiving a bitstream comprising encoded bits of multiple segments of syntax elements. Syntax elements have various types, corresponding to transform coefficients of transform blocks in an encoded image. Context modeling is performed to determine a context model for each segment of syntax elements. Within a given frequency region of a transform block, for a given syntax element type, a set of transform coefficients with different template sizes within a predetermined range share the same context model, or, possible different template sizes for any transform coefficient use the same context model. The possible different template sizes are within a predetermined range. The encoded bits are decoded based on the context model determined for the syntax elements to determine the multiple segments of syntax elements.

[0015] Various embodiments of this application provide a video decoding apparatus. The apparatus may include processing circuitry configured to receive a bitstream comprising encoded bits of syntactic elements. Syntactic elements have various types, corresponding to transform coefficients of transform blocks in an encoded image. Context modeling is performed to determine a context model for each segment of syntactic elements. Within a given frequency region of a transform block, for a given syntactic element type, a set of transform coefficients with different template sizes within a predetermined range share the same context model, or, possible different template sizes for any transform coefficient use the same context model. The possible different template sizes are within a predetermined range. The encoded bits are decoded based on the context model determined for the syntactic elements to determine the syntactic elements.

[0016] Various embodiments of this application also provide a video decoding apparatus, including:

[0017] A receiving module is used to receive a bit stream, the bit stream comprising encoded bits of multiple syntax elements, the syntax elements comprising various types of transform coefficients corresponding to transform blocks in an encoded image;

[0018] An execution module is configured to perform context modeling to determine a context model for each segment of grammatical elements, wherein, in a given frequency region of the transform block, for one type of the grammatical element: a set of transform coefficients with different template sizes within a predetermined range share the same context model, or, any possible different template sizes of any transform coefficient use the same context model, the possible different template sizes being within the predetermined range; and

[0019] A decoding module is used to decode the encoded bits based on the context model determined for the syntax element to determine the multi-segment syntax element.

[0020] Various embodiments of this application also provide a computer device, including a processor and a memory connected to the processor, the memory storing machine-readable instructions, the processor executing the machine-readable instructions to perform the methods described in any embodiment of this application.

[0021] Various embodiments of this application also provide a non-volatile computer-readable medium having instructions stored thereon that can be executed by a computer to perform the methods described in any embodiment of this application.

[0022] According to the above technical solution of this application, within a given frequency region of a transform block, if the size of adjacent transform coefficients covered by the local template of each member in a set of transform coefficients is within a predetermined range, then for a given type of syntax element, the set of transform coefficients can share the same context model. The size of adjacent transform coefficients covered by the template can be characterized by the template size. Alternatively, within a frequency region, when different possible template sizes of transform coefficients are within a predetermined range, the transform coefficients can use the same context model for different possible template sizes of the transform coefficients. Therefore, this application can significantly reduce the number of context models used for transform coefficient encoding, thereby further reducing the size of the memory storing these context models and the cost of maintaining these context models, and further improving video encoding and decoding efficiency. Attached Figure Description

[0023] Other features, properties, and various advantages of the subject matter of this application will become clearer from the following detailed description and accompanying drawings, in which:

[0024] Figure 1 This is a schematic diagram of the current block and its surrounding space as merge candidates in an example;

[0025] Figure 2 This is a simplified block diagram of a communication system according to an embodiment;

[0026] Figure 3 This is a simplified block diagram of a communication system according to another embodiment;

[0027] Figure 4 This is a simplified block diagram of the decoder according to an embodiment;

[0028] Figure 5 This is a simplified block diagram of the encoder according to an embodiment;

[0029] Figure 6 A block diagram of an encoder according to another embodiment is shown;

[0030] Figure 7 A block diagram of a decoder according to another embodiment is shown;

[0031] Figure 8A An exemplary context-based adaptive binary arithmetic code (CABAC) entropy encoder according to one embodiment is shown;

[0032] Figure 8B An exemplary CABAC-based entropy decoder according to another embodiment is shown;

[0033] Figure 9 An example of the sub-block scan order according to one embodiment is shown;

[0034] Figure 10 An example of a sub-block scanning process according to one embodiment is shown, wherein different types of syntax elements are generated from the sub-block scanning process to produce transform coefficients;

[0035] Figure 11 An example of a local template for context selection of the current coefficient is shown;

[0036] Figure 12 This indicates the diagonal position of the coefficients or coefficient levels within the coefficient block;

[0037] Figure 13A A flowchart of an entropy decoding process according to one embodiment is shown;

[0038] Figure 13B A schematic diagram of a video decoding apparatus according to one embodiment is shown;

[0039] Figure 14 This is a schematic diagram of a computer system according to one embodiment. Detailed Implementation

[0040] H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this application describes the technique hereinafter referred to as “spatial combining”.

[0041] Please refer to Figure 1The current block (101) includes samples that have been discovered by the encoder during the motion search process, and these samples can be predicted based on previous blocks of the same size that have generated spatial offsets. Alternatively, the MV can be derived from metadata associated with one or more reference images, rather than being directly encoded. For example, using the MV associated with any of the five surrounding samples A0, A1 and B0, B1, B2 (corresponding to 102 to 106 respectively), the MV is derived from the metadata of the nearest reference image (in decoding order). In H.265, MV prediction can use predictions from the same reference image that is also being used in adjacent blocks.

[0042] Figure 2 This is a simplified block diagram of a communication system (200) according to an embodiment disclosed in this application. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first terminal device (210) and a second terminal device (220) interconnected via a network (250). Figure 2 In this embodiment, the first terminal device (210) and the second terminal device (220) perform unidirectional data transmission. For example, the first terminal device (210) may encode video data (e.g., a video image stream captured by the terminal device (210)) for transmission over a network (250) to the second terminal device (220). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video data, and display video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0043] In another embodiment, the communication system (200) includes a third terminal device (230) and a fourth terminal device (240) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (230) and the fourth terminal device (240) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission over a network (250) to the other terminal device. Each of the third terminal device (230) and the fourth terminal device (240) may also receive encoded video data transmitted by the other terminal device and may decode the encoded video data to recover the video data, and may display the video images on an accessible display device based on the recovered video data.

[0044] exist Figure 2In the embodiments disclosed herein, the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240) may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (250) refers to any number of networks that transmit encoded video data between the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240), including, for example, wired (connected) and / or wireless communication networks. The communication network (250) may exchange data in circuit-switched and / or packet-switched channels. The network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation of this application.

[0045] As an example, Figure 3 The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0046] The streaming system may include an acquisition subsystem (313) that may include a video source (301) such as a digital camera, which creates an uncompressed video image stream (302). In an embodiment, the video image stream (302) includes samples captured by a digital camera. The video image stream (302) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (304) (or encoded video bitstream). The video image stream (302) may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (302), the encoded video data (304) (or the encoded video bitstream (304)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (304) (or the encoded video bitstream (304)), which can be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3Client subsystems (306) and (308) can access a streaming server (305) to retrieve copies (307) and (309) of encoded video data (304). Client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and produces an output video picture stream (311) that can be displayed on a display (312) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (304), video data (307), and video data (309) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0047] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0048] Figure 4 This is a block diagram of a video decoder (410) according to an embodiment disclosed in this application. The video decoder (410) may be disposed in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of... Figure 3 The video decoder (310) in the embodiment.

[0049] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (431) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be located external to the video decoder (410) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (410) to prevent network jitter, for example, and another buffer (415) may be configured internally for, for example, handling broadcast timing. When the receiver (431) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (415) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (415) may be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (410).

[0050] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (410) and potential information for controlling a display device (412) (e.g., a display screen), which is not part of the electronic device (430) but may be coupled to it, such as... Figure 4As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0051] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0052] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed by the parser (420) from the encoded video sequence. For brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not described.

[0053] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0054] The first unit is the scaler / inverse transform unit (451). The scaler / inverse transform unit (451) receives quantization transform coefficients as symbols (421) and control information from the parser (420), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block containing sample values, which can be input into the aggregator (455).

[0055] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) uses reconstructed information extracted from the current picture buffer (458) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current picture buffer (458) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (455) adds the predictive information generated by the intra-picture prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0056] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (453) can access the reference image memory (457) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (421), these samples can be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation prediction unit (453) can obtain the prediction samples from the address in the reference image memory (457) under motion vector control, and the motion vector is available to the motion compensation prediction unit (453) in the form of the symbols (421), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (457) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0057] The output samples of the aggregator (455) can be employed by various loop filtering techniques in the loop filter unit (456). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (421) from the parser (420) in the loop filter unit (456). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0058] The output of the loop filter unit (456) can be a sample stream, which can be output to a display device (412) and stored in a reference image memory (457) for subsequent inter-frame image prediction.

[0059] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and the encoded image (by, for example, the parser (420)) is identified as the reference image, the current image buffer (458) can become part of the reference image memory (457), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0060] The video decoder (410) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0061] In this embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0062] Figure 5 This is a block diagram of a video encoder (503) according to an embodiment disclosed in this application. The video encoder (503) is disposed in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used to replace... Figure 3 The video encoder (303) in the embodiment.

[0063] The video encoder (503) can obtain data from the video source (501) (not) Figure 5 In one embodiment, a portion of the electronic device (520) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (503). In another embodiment, the video source (501) is a portion of the electronic device (520).

[0064] A video source (501) can provide a sequence of source video samples encoded by a video encoder (503) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (501) can be a storage device storing previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed in sequence. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0065] According to an embodiment, the video encoder (503) can encode and compress images of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (550) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be used with other suitable functions related to the video encoder (503) optimized for a particular system design.

[0066] In some embodiments, the video encoder (503) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (533) embedded within the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques considered in this application, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (534). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory (534) are also bit-precisely corresponding between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0067] The operation of the “local” decoder (533) can be combined with, for example, the above-described method. Figure 4 The video decoder (410) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 4 When symbols are available and the entropy encoder (545) and parser (420) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (410), including the buffer (415) and parser (420), may not be fully implemented in the local decoder (533).

[0068] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0069] During operation, in some embodiments, the source encoder (530) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (532) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0070] The local video decoder (533) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (530). The operation of the encoding engine (532) can be a lossy process. When the encoded video data can be decoded by the video decoder (533), Figure 5 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (534). In this way, the video encoder (503) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0071] The predictor (535) can perform a prediction search against the encoding engine (532). That is, for a new image to be encoded, the predictor (535) can search in the reference image memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (535) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (535), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (534).

[0072] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0073] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (545). The entropy encoder (545) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0074] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0075] The controller (550) manages the operation of the video encoder (503). During encoding, the controller (550) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0076] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics.

[0077] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0078] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0079] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0080] The video encoder (503) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (503) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0081] In this embodiment, the transmitter (540) may transmit additional data while transmitting encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0082] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0083] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0084] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0085] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0086] Figure 6 This is a diagram of a video encoder (603) according to another embodiment disclosed in this application. The video encoder (603) is used to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (603) is used instead of Figure 3The video encoder (303) in the embodiment.

[0087] In the HEVC embodiment, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use an intra-frame mode, an inter-frame mode, or a bidirectional prediction mode to encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (603) can use intra-frame prediction techniques to encode the processing block into an already encoded picture; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (603) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into an already encoded picture, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector prediction values ​​without relying on already encoded motion vector components outside the prediction values. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the embodiment, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0088] exist Figure 6 In one embodiment, the video encoder (603) includes, as shown below: Figure 6 The inter-frame encoder (630), intra-frame encoder (622), residual calculator (623), switch (626), residual encoder (624), general controller (621) and entropy encoder (625) are shown coupled together.

[0089] An inter-frame encoder (630) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., redundancy information description, motion vectors, merging mode information based on inter-frame coding techniques), and calculate inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image based on encoded video information.

[0090] The intra encoder (622) is used to receive samples of the current block (e.g., the processing block), in some cases compare the block with previously encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also (e.g., based on intra prediction direction information of one or more intra coding techniques) generate intra prediction information. In an embodiment, the intra encoder (622) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same image.

[0091] A general-purpose controller (621) determines general-purpose control data and controls other components of the video encoder (603) based on the general-purpose control data. In an embodiment, the general-purpose controller (621) determines the mode of a block and provides control signals to a switch (626) based on the mode. For example, when the mode is an intra-frame mode, the general-purpose controller (621) controls the switch (626) to select an intra-frame mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select intra-frame prediction information and add the intra-frame prediction information to the bitstream; and when the mode is an inter-frame mode, the general-purpose controller (621) controls the switch (626) to select an inter-frame prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select inter-frame prediction information and add the inter-frame prediction information to the bitstream.

[0092] A residual calculator (623) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (622) or the inter encoder (630). A residual encoder (624) is used to operate on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (624) is used to transform the residual data from the time domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (622) and the inter encoder (630). For example, the inter encoder (630) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (622) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, and in some embodiments, the decoded image may be buffered in a memory circuit (not shown) and used as a reference image.

[0093] An entropy encoder (625) is used to format the bitstream to produce encoded blocks. The entropy encoder (625) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (625) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0094] Figure 7This is a diagram of a video decoder (710) according to another embodiment disclosed in this application. The video decoder (710) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (710) is used instead of Figure 3 The video decoder (310) in the embodiment.

[0095] exist Figure 7 In this embodiment, the video decoder (710) includes, as follows: Figure 7 The entropy decoder (771), inter-frame decoder (780), residual decoder (773), reconstruction module (774), and intra-frame decoder (772) are shown coupled together.

[0096] An entropy decoder (771) can be used to reconstruct certain symbols from an encoded image, these symbols representing the syntax elements constituting the encoded image. Such symbols may include, for example, a mode for encoding the block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In an embodiment, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (772). Residual information may be provided to the residual decoder (773) via inverse quantization.

[0097] The inter-frame decoder (780) is used to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0098] The intra-frame decoder (772) is used to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0099] The residual decoder (773) performs inverse quantization to extract the dequantized transform coefficients and processes the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to obtain the quantizer parameters QP), and this information may be provided by the entropy decoder (771) (the data path is not indicated because this is only low-level control information).

[0100] The reconstruction module (774) is used to combine the residual output by the residual decoder (773) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0101] It should be noted that any suitable technology can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In one embodiment, one or more integrated circuits can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710).

[0102] Entropy coding can be performed in the final stage of video encoding (or the first stage of video decoding) after the video signal has been reduced to a series of syntax elements. Entropy coding can be a lossless compression scheme that uses statistical properties to compress data so that the number of bits representing the data is logarithmically proportional to the probability of the data. For example, by entropy coding a set of syntax elements, the bits representing the syntax elements (called segments) can be converted into fewer bits (called encoded bits) in the bitstream (also called the bit stream). Context-based Adaptive Binary Arithmetic Coding (CABAC) is a form of entropy coding. In CABAC, a context model providing probability estimates is determined for each segment in a segment sequence based on the context associated with the corresponding segment. Then, a binary arithmetic coding process is performed using the probability estimates to encode the segment sequence into encoded bits in the bitstream. Furthermore, the context model is updated using new probability estimates based on the encoded segments.

[0103] Figure 8A An exemplary CABAC-based entropy encoder (800A) according to one embodiment is shown. For example, the entropy encoder (800A) can be... Figure 5 The entropy encoder (545) in the example or Figure 6This is implemented in the entropy encoder (625) in the example. The entropy encoder (800A) may include a context modeler (810) and a binary arithmetic encoder (820). In one example, various types of syntax elements are provided as input to the entropy encoder (800A). For example, a segment of a binary value syntax element may be directly input to the context modeler (810), while a non-binary value syntax element may first be binary-coded into a sequence of segments, and then the segments of the sequence of segments may be input to the context modeler (810).

[0104] In one example, the context modeler (810) receives segments of syntax elements and performs a context modeling process, selecting a context model for each received segment. For example, it receives a segment of binary syntax elements containing transform coefficients in a transform block. Therefore, the context model for this segment can be determined based on, for example, the type of the syntax element, the color component type of the transform component, the position of the transform coefficients, and previously processed neighboring transform coefficients. The context model can provide a probability estimate for this segment.

[0105] In one example, a set of context models can be configured for each type of syntax element. These context models can be arranged in a list (802) of context models stored in memory (801), such as Figure 8A As shown. Each entry in the context model list (802) can represent a context model. An index, called the context model index or context index, can be assigned to each context model in the list. Furthermore, each context model can include a probability estimate, or a parameter representing the probability estimate. The probability estimate can represent the likelihood that a segment is 0 or 1. For example, during context modeling, the context modeler (810) can calculate the context index of a segment and select the appropriate context model based on the context index in the context model list (802) and assign it to the segment.

[0106] Furthermore, the probability estimates in the context model list can be initialized when the entropy encoder (800A) starts operating. After the context models on the context model list (802) are assigned to segments and used to encode those segments, the context models can then be updated based on the values ​​of the segments with updated probability estimates.

[0107] In one example, the binary arithmetic encoder (820) receives a segment and a context model (e.g., probability estimate) assigned to the segment, and performs a binary arithmetic encoding process accordingly. Thus, encoded bits are generated and transmitted in the bitstream.

[0108] Figure 8BAn exemplary CABAC-based entropy decoder (800B) according to one embodiment is shown. For example, the entropy decoder (800B) can be... Figure 4 The parser (420) in the example or Figure 7 This is implemented in the entropy decoder (771) in the example. The entropy decoder (800B) may include a binary arithmetic decoder (830) and a context modeler (840). The binary arithmetic decoder (830) receives encoded bits from the bitstream and performs a binary arithmetic decoding process to recover segments from the encoded bits. The context modeler (840) may operate similarly to the context modeler (810). For example, the context modeler (840) may select a context model from a list (804) of context models stored in memory (803) and provide the selected context model to the binary arithmetic decoder (830). However, the context modeler (840) determines the context model based on the segments recovered in the binary arithmetic decoder (830). For example, based on the recovered segments, the context modeler (840) may know the syntax element type of the next segment to be decoded, as well as the values ​​of previously decoded syntax elements. This information is used to determine the context model of the next segment to be decoded.

[0109] In one embodiment, the residual signal of the transform block is first transformed from the spatial domain to the frequency domain to generate a transform coefficient block. Then, quantization is performed to quantize the transform coefficient block into a transform coefficient level block. In various embodiments, different techniques can be used to convert the residual signal into a transform coefficient level. The transform coefficient level block is further processed to generate syntax elements, which can be provided to an entropy encoder and encoded as bits of a bitstream. In one embodiment, the process of generating syntax elements from the transform coefficient level can be performed as follows.

[0110] The transform coefficient level block can first be divided into several sub-blocks, for example, a sub-block can be 4×4 positions in size. These sub-blocks can be processed according to a predefined scan order. Figure 9 An example of a sub-block scanning order is shown, called the anti-diagonal scanning order. As shown, block (910) is divided into 16 sub-blocks (901). The lower right sub-block is processed first, and the upper left sub-block is processed last. In one example, a sub-block with all transformation coefficients at zero level can be skipped without processing.

[0111] For each subblock with at least one non-zero transform coefficient level, four scans can be performed in each subblock. During each scan, the 16 positions within each subblock can be scanned in anti-diagonal scan order. Figure 10 An example of a subblock scan procedure (1000) is shown, in which different types of syntax elements of transform coefficients are generated from the subblock scan procedure.

[0112] The positions of the sixteen coefficients within the sub-block (1010) are as follows: Figure 10 The bottom one-dimensional view is shown. Positions (1010) are numbered from 0 to 15, reflecting their respective scanning order. During the first scan, position (1010) is scanned, and three types of syntax elements (1001-1003) can be generated at each position (1010):

[0113] (i) The first type of binary syntax element (1001) (called the validity flag, denoted by sig_coeff_flag) indicates whether the absolute transform coefficient level (denoted by absLevel) of the corresponding transform coefficient is zero or greater than zero.

[0114] (ii) The second type of binary syntax element (1002) (called the parity flag, denoted by par_level_flag) indicates the parity of the absolute transform level of the corresponding transform coefficient. The parity flag is generated only when the absolute transform level of the corresponding transform coefficient is not zero.

[0115] (iii) The third type of binary syntax element (1003) (called the greater than 1 flag, denoted by rem_abs_gt1_flag) indicates whether (absLevel-1) >> 1 is greater than 0 for the corresponding transform coefficient. The greater than 1 flag is generated only when the absolute transform coefficient level of the corresponding transform coefficient is not zero.

[0116] During the second scan, a fourth type of binary syntax element (1004) may be generated. This fourth type of syntax element (1004) is called the greater than 2 flag, denoted by `rem_abs_gt2_flag`. The fourth type of syntax element (1004) indicates whether the absolute transform level of the corresponding transform coefficient is greater than 4. A flag greater than 2 is only generated if (absLevel-1) >> 1 for the corresponding transform coefficient is greater than 0.

[0117] During the third scan, a fifth type of non-binary syntax element (1005) may be generated. The fifth type of syntax element (1005) is denoted by `abs_remainer`, indicating the remainder of the absolute transform coefficient order of the corresponding transform coefficient if it is greater than 4. The fifth type of syntax element (1005) is only generated if the absolute transform coefficient order of the corresponding transform coefficient is greater than 4.

[0118] During the fourth scan, a sixth type of syntax element (1006) can be generated at each scan position (1010), whose non-zero coefficient level indicates the sign of the corresponding transformation coefficient level.

[0119] The various types of syntax elements (1001-1006) described above can be provided to the entropy encoder according to the order of the scan count and the order of each scan. Different entropy encoding schemes can be used to encode different types of syntax elements. For example, in one embodiment, validity flags, parity flags, flags greater than 1, and flags greater than 2 can be encoded using a CABAC-based entropy encoder, such as... Figure 8A As shown in the example. Conversely, the syntax elements generated during the third and fourth scans can be encoded using a CABAC bypass entropy encoder (e.g., a binary arithmetic encoder with a fixed probability estimate of the input segment).

[0120] Context modeling can be used to determine the context model of segments of certain types of transform coefficient syntax elements. In one embodiment, the context model can be determined based on the local template and the diagonal position of each current coefficient (e.g., the coefficient currently being processed), and possibly in combination with other factors.

[0121] Figure 11 An example of a local template (1130) for context selection of the current coefficient is shown. The local template (1130) can cover a set of adjacent positions or coefficients of the current coefficient (1120) in a coefficient block (1110). The coefficient block (1110) can have a size of 8×8 positions and include a coefficient level of 64 positions. The coefficient block (1110) is divided into 4 sub-blocks, each with a size of 4×4 positions. Figure 11 In the example, the local template (1130) is defined as a 5-position template that covers 5 coefficient levels at the lower right of the current coefficient (1120). When multiple scans are performed on the scan positions within the coefficient block (1110) using the anti-angle scan sequence, adjacent positions within the local template (1130) are processed before the current coefficient (1120).

[0122] During context modeling, information about the coefficient levels within the local template (1130) can be used to determine the context model. For this purpose, in some embodiments, a metric is referred to as template size to measure or indicate the magnitude of the transform coefficients or transform coefficient levels within the local template (1130). The template size can then be used as a basis for selecting the context model.

[0123] In one example, the template size is defined as the sum (denoted by sumAbs1) of the absolute transform coefficient levels of the partial reconstruction within the local template (1130). The absolute transform coefficient levels of the partial reconstruction can be determined based on the syntax element segment, sig_coeff_flag, par_level_flag, and rem_abs_gt1_flag of the corresponding transform coefficients. These three types of syntax elements are obtained after the first scan of the sub-block's scan position in the entropy encoder or entropy decoder. In one embodiment, the absolute transform coefficient level of the partial reconstruction at position (x, y) can be determined according to the following formula:

[0124] absLevel1[x][y]=sig_coeff_flag[x][y]+par_level_flag[x][y]+2*rem_abs_gt1_flag[x][y],

[0125] Where x and y are coordinates relative to the top left corner of coefficient block (1110), and absLevel1[x][y] represents the absolute transformation coefficient level of the partial reconstruction at position (x, y).

[0126] In another example, the template size is defined as the difference (denoted by tmplCpSum1) between the sum of the absolute transform coefficient levels of the partially reconstructed template and the number of non-zero coefficients in the local template (1130) (denoted by numSig). Therefore, this difference can be determined according to the following formula:

[0127] tmplCpSum1=sumAbs1-numSig.

[0128] In other examples, the template size can be defined in other ways to represent the size of the transform coefficients or transform coefficient levels.

[0129] Figure 12 The diagonal positions of coefficients or coefficient classes within coefficient block (1210) are shown. In one embodiment, the diagonal position of the scan position (x, y) is defined according to the following formula:

[0130] d = x + y

[0131] Where d represents the diagonal position, and x and y are the coordinates of the corresponding position. The diagonal position d of each coefficient can be used to define different frequency regions within the coefficient block (1210) based on one or two diagonal position thresholds. Two examples are given: defining the low-frequency region (1220) with d <= 3 and defining the high-frequency region (1230) with d >= 11. Figure 12 As shown.

[0132] In one embodiment, during context modeling in the entropy encoder or entropy decoder, the context index can be determined from the binary syntax elements of the context-encoded current coefficient (1120) as described below. This determination operation can be performed based on the diagonal position of the local template (1130) and the current coefficient (1120).

[0133] (1) sig_coeff_flag

[0134] When encoding the sig_coeff_flag of the current coefficient (1120), the context index can be selected based on the sumAbs1 and diagonal position d of the current coefficient (1120). For example, for the luminance component, the context index is determined according to the following formula:

[0135] ctxSig=18*max(0,state-1)+min(sumAbs1,5)+(d<2?12:(d<5?6:0))(1)

[0136] Here, ctxSig represents the context index of the validity flag syntax element, and "state" specifies the state of the scalar quantizer that depends on the quantization scheme, where the value of the state can be 0, 1, 2 or 3.

[0137] Dependent quantization refers to the concept that a set of permissible reconstructed values ​​for transform coefficients depends on the values ​​of transform coefficient levels preceding the current transform coefficient level, according to the reconstruction order (e.g., a predetermined scan order on sub-blocks or coefficient blocks). For example, the quantizer for transform coefficients can be selected via a state machine with four states, where the states are updated in reconstruction order based on parity of previous levels. For encoding transform coefficient levels, context modeling can depend on the quantizer used for the corresponding transform coefficient.

[0138] Expression (1) is equivalent to:

[0139] ctxIdBase = 18 * max(0, state - 1) + (d < 2 ? 12 : (d < 5 ? 6 : 0)), (2)

[0140] ctxSig = ctxIdSigTable[min(sumAbs1, 5)] + ctxIdBase. (3)

[0141] In expressions (2) and (3), ctxIdBase represents the context index library. The context index library can be determined based on the state and the diagonal position d. For example, the value of the state can be 0, 1, 2, or 3, so the value of max(0, state-1) could be one of 0, 1, or 2. For example, (d<2?12:(d<5?6:0)) can take the values ​​12, 6, or 0, corresponding to different ranges of d: d<2, 2<=d<5, or 5<=d.

[0142] In expressions (2) and (3), ctxIdSigTable[] can represent an array data structure and store the context index offset of the validity flag associated with ctxIdBase. For example, for different sumAbs1 values, min(sumAbs1,5) limits the sumAbs1 value to less than or equal to 5. The limit value is then mapped to the context index offset. For example, under the definition of ctxIdSigTable[0~5]={0,1,2,3,4,5}, the limit values ​​0, 1, 2, 3, 4 or 5 are mapped to 0, 1, 2, 3, 4 or 5 respectively.

[0143] For chromaticity components, the context index can be determined according to the following formula:

[0144] ctxSig = 12 * max(0, state - 1) + min(sumAbs1, 5) + (d < 2 ? 6 : 0), (4)

[0145] It is equivalent to:

[0146] ctxIdBase = 12 * max(0, state - 1) + (d < 2? 6: 0), (5)

[0147] ctxSig = ctxIdSigTable[min(sumAbs1, 5)] + ctxIdBase. (6)

[0148] (2) par_level_flag

[0149] When encoding the par_level_flag of the current coefficient (1120), the context index can be selected based on sumAbs1, numSig, and the diagonal position d. For example, for the luma component, if the current coefficient is the first non-zero coefficient in the decoding order, the context index ctxPar is assigned a value of 0; otherwise, the context index can be determined according to the following formula:

[0150] ctxPar=1+min(sumAbs1-numSig,4)+(d==0?15:(d<3?10:(d<10?5:0))), (7)

[0151] It is equivalent to:

[0152] ctxIdBase = (d == 0 ? 15 : (d < 3 ? 10 : (d < 10 ? 5 : 0))), (8)

[0153] ctxPar=1+ctxIdTable[min(sumAbs1-numSig,4)]+ctxIdBase, (9)

[0155] Here, ctxPar represents the context index of the parity flag, and ctxIdTable[] represents another array data structure that stores the context index offset associated with the corresponding ctxIdBase. For example, ctxIdTable[0~4]={0,1,2,3,4}.

[0156] For chroma, if the current coefficient is the first non-zero coefficient in the decoding order, the context index ctxPar is assigned a value of 0; otherwise, the context index can be determined according to the following formula:

[0157] ctxPar = 1 + min(sumAbs1 - numSig, 4) + (d == 0 ? 5 : 0), (10)

[0158] It is equivalent to:

[0159] ctxIdBase = (d == 0 ? 5 : 0), (11)

[0160] ctxPar=1+ctxIdTable[min(sumAbs1-numSig,4)]+ctxIdBase. (12)

[0162] (3) rem_abs_gt1_flag and rem_abs_gt2_flag

[0163] When encoding the rem_abs_gt1_flag and rem_abs_gt2_flag for the current coefficient (1120), the context model index can be determined in the same way as par_level_flag:

[0164] ctxGt1 = ctxPar,

[0165] ctxGt2 = ctxPar,

[0166] Where ctxGt1 and ctxGt2 represent the context indices of flags greater than 1 and greater than 2, respectively.

[0167] It should be noted that different context model sets are used for different types of syntax elements: sig_coeff_flag, par_level_flag, rem_abs_gt1_flag, and rem_abs_gt2_flag. For example, the context model used for rem_abs_gt1_flag is different from the context model used for rem_abs_gt2_flag, even if the value of ctxGt1 is equal to the value of ctxGt2.

[0168] Table 1 shows the context indices of the validity flags (corresponding to a scalar quantizer state) determined using the context modeling method described above. Table 2 shows the context indices of the parity flag, the flag greater than 1, and the flag greater than 2 using the context modeling method described above. In Table 2, in one embodiment, for the last non-zero coefficient level of a coefficient block under a forward diagonal scan order (e.g., the reverse of a reverse diagonal scan order), the context index of the context model for the three syntax elements of the last non-zero coefficient is assigned a value of 0. Based on Tables 1 and 2, 186 context models need to be maintained in memory for context modeling of the four types of coefficient syntax elements: in Table 1, for a state, (18+12) = 30, and for the three possible max(0,state-1) values, the value is 90; in Table 2, for one of the three syntax elements, (21+11) = 32, and for all three syntax elements, the value is 96.

[0169] Table 1

[0170]

[0171] Table 2

[0172]

[0173] In the above embodiment, 186 context models are maintained in memory to perform context modeling for the four types of coefficient syntax elements. Considering that there are many other syntax elements besides the four types of coefficient syntax elements, the entropy encoder or decoder could use hundreds of context models. Reducing the number of these context models would reduce the required memory size and the cost of maintaining them, and is therefore preferable.

[0174] Therefore, embodiments of this application provide some methods for reducing the number of context models used for entropy coding of transform coefficients. For example, within a given frequency region of a transform block, if the size of adjacent transform coefficients covered by the local template of each member in a set of transform coefficients is within a predetermined range, then for a given type of syntax element, the set of transform coefficients can share the same context model. The size of the adjacent transform coefficients covered by the template can be characterized by the template size. Alternatively, within a frequency region, when different possible template sizes of transform coefficients are within a predetermined range, the transform coefficients can use the same context model for different possible template sizes of the transform coefficients. Embodiments and examples of the context model reduction method are described below.

[0175] In one embodiment, the same context model can be shared among a set of transform coefficients from the current block and other blocks. Each coefficient in the transform coefficient set can be located in a low-frequency region of the coefficient block, and each coefficient has adjacent transform coefficients covered by a smaller local template. For example, the size of the adjacent transform coefficients covered by the local template can be represented by the template size, and the template size is less than a threshold. The low-frequency region can be specified using the diagonal position (d) of the corresponding coefficient. The local template can be similar to... Figure 11 The template in the example.

[0176] Alternatively, in this embodiment, the transform coefficients in the low-frequency region can use the same context model for different possible template sizes. For example, transform coefficients may have different template sizes depending on their proximity. For those possible template sizes, if they are less than a threshold (within the same range), the same context model can be determined corresponding to those different template sizes of the transform coefficients.

[0177] In one example, for transform coefficients with d=0 (there is only one transform coefficient in the low-frequency region where d=0), and for the par_level_flag of the transform coefficients, a single context model can be shared between possible template sizes where tmplCpSum1=0 or tmplCpSum1=1. For example, the first possible template size can be computed as templateSum1=0, or the second possible template size can be computed as templateSum1=1. If expression (7) is used for context modeling, two different context indices corresponding to the two context models maintained in memory will be determined for the first and second template sizes. However, using the context model reduction method, the same context model is determined for two different templCpSum1 values ​​(0 or 1) corresponding to the transform coefficients.

[0178] For example, operations can be performed on the table ctxIdTable, which stores the context model index offsets, to achieve sharing of the same context model. Specifically, ctxIdTable can be designed as ctxIdTable[0~4]={0,0,1,2,3}. To calculate the context model index, ctxIdBase can first be calculated based on d using, for example, expressions (8) or (11). Then, the context model index is calculated by 1+ctxIdBase+ctxIdTable[min(tmplCpSum1,4)], where ctxIdTable is designed as ctxIdTable[0~4]={0,0,1,2,3}. Therefore, different values ​​of tmpCpSum1 can be mapped to the same context index.

[0179] Similarly, in another example, for the transform coefficients of d=0, and for rem_abs_gt1_flag or rem_abs_gt2_flag, the same context model can be shared between different template sizes of tmplCpSum1=0 and tmplCpSum1=1.

[0180] In one example, for transformation coefficients where d < 2, and for sig_coeff_flag, a context model can be shared between transformation coefficients where sumAbs1 = 0 and sumAbs1 = 1. For example, the table ctxIdSigTable storing context model index offsets can be designed as ctxIdSigTable[0~5] = {0,0,1,2,3,4}. To calculate the context model index, ctxIdBase can first be calculated based on d and the state using, for example, expression (2). Then, the context model index can be calculated by ctxIdBase + ctxIdSigTable[min(sumAbs1,5)], where ctxIdSigTable is designed as ctxIdSigTable[0~5] = {0,0,1,2,3,4}.

[0181] In one embodiment, the same context model can be shared among a set of transform coefficients. This set of transform coefficients may be located in the high-frequency region of a coefficient block, and each transform coefficient has adjacent transform coefficients covered by a larger local template (e.g., the template size is represented by a template size greater than a threshold). The high-frequency region can be specified using the diagonal position (d) of the correlation coefficients. The local template can be similar to... Figure 11 The template in the example.

[0182] In one example, for transform coefficients of d>=10, and for par_level_flag, the same context model can be shared between transform coefficients of tmplCpSum1=3 and tmplCpSum1=4. For example, the table ctxIdTable storing context model index offsets can be designed as ctxIdTable[0~4]={0,1,2,3,3}. To calculate the context model index, ctxIdBase is first calculated based on d using, for example, expressions (8) or (11). Then, the context model index is calculated by 1+ctxIdBase+ctxIdTable[min(tmplCpSum1,4)], where ctxIdTable is designed as ctxIdTable[0~4]={0,1,2,3,3}.

[0183] Similarly, in another example, for transform coefficients with d>=10, and for either rem_abs_gt1_flag or rem_abs_gt2_flag, the same context model can be shared between transform coefficients with tmplCpSum1=3 and tmplCpSum1=4.

[0184] In one example, for transform coefficients with d>=5, and for sig_coeff_flag, a context model can be shared between transform coefficients with sumAbs1=4 and sumAbs1=5. For example, the table ctxIdSigTable storing context model index offsets can be designed as ctxIdSigTable[0~5]={0,1,2,3,4,4}. To compute the context model index, ctxIdBase can first be computed based on d and state using, for example, expressions (2) or (5). Then, the context model index can be computed by ctxIdBase+ctxIdSigTable[min(sumAbs1,5)], where ctxIdSigTable is designed as ctxIdSigTable[0~5]={0,1,2,3,4,4}.

[0185] In one embodiment, the same context model can be shared among a set of transform coefficients for the chrominance components. This set of transform coefficients may have adjacent transform coefficients covered by a larger local template (e.g., the template size is represented by a template size greater than a threshold). The local template can be similar to... Figure 11 The template in the example.

[0186] In one example, for par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag, a context model can be shared between the transform coefficients of the chromaticity components at tmplCpSum1=3 and tmplCpSum1=4. For example, the table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4]={0,1,2,3,3}. To calculate the context model index, ctxIdBase can first be calculated based on d using, for example, expression (11). Then, the context model index is calculated by ctxIdBase+ctxIdTable[min(tmplCpSum1,4)], where ctxIdTable is designed as ctxIdTable[0~4]={0,1,2,3,3}.

[0187] In one example, for sig_coeff_flag, a context model can be shared among the transform coefficients of chromaticity components with sumAbs1 = 4 and sumAbs1 = 5. For example, the table ctxIdSigTable storing the context model index offset can be designed as ctxIdSigTable[0~5] = {0,1,2,3,4,4}. To calculate the context model index, ctxIdBase can first be calculated based on d and the state, for example, using expression (5). Then, the context model index can be calculated by ctxIdBase + ctxIdSigTable[min(sumAbs1,5)], where ctxIdSigTable is designed as ctxIdSigTable[0~5] = {0,1,2,3,4,4}.

[0188] In one embodiment, a context model index mapping table ctxIdMap[] is trained using a machine learning algorithm to share the context model among transform coefficients. In one example, the context model index ctxId of the syntax element is first computed using the method described above. Then, the final context model index is determined by examining the mapping table ctxIdMap[ctxId].

[0189] In one example, following the method described above, the size of the context model index mapping table ctxIdMap[] can be 20 for the encoding of par_level_flag, rem_abs_gt1_flag, and rem_abs_gt2_flag from the luma and chroma components, since the size of ctxPar, ctxGt1, or ctxGt2 is between 1 and 20. For example, after calculating ctxPar, ctxGt1, or ctxGt2 using the method described above, the final context model index is determined via ctxIdMap[ctxPar-1], ctxIdMap[ctxGt1-1], or ctxIdMap[ctxGt2-1]. For example, if ctxIdMap is ctxIdMap[0~19]={1,2,3,4,4,5,6,7,8,9,10,11,12,13,14,15,15,16,17,18}, then for ctxPar=4 and ctxPar=5, they will share the same context model according to the context model mapping table. Specifically, ctxIdMap[4-1] is the same as ctxIdMap[5-1], both having a value of 4.

[0190] In one embodiment, transform coefficients within a given frequency region are grouped when the magnitudes of the transform coefficients covered by the local template of each coefficient lie within a defined interval. Transform coefficients within the same group can share the same context model. This interval can be different for different frequency regions or different syntax elements.

[0191] In one example, for the encoding of the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag, within the frequency region defined by d1>d>d0, each transform coefficient with tmplCpSum1 shares the same context model within the interval (or range) of sum1>=tmplCpsum1>=sum0. The parameters sum0 and sum1 are predefined thresholds. In some examples, sum0 and sum1 may depend on d0 and d1. In other words, the definition of the interval may differ for different frequency regions.

[0192] For example, when d0 = 0 and d1 = 3, sum0 = 1 and sum1 = 2, the table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4] = {0,1,1,2,3}, for the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag. To calculate the context model index, ctxIdBase can first be calculated based on d, for example using expression (8) or (11). Then, the context model index can be calculated by 1 + ctxIdBase + ctxIdTable[min(tmplCpSum1,4)].

[0193] As another example, when d0 = 2 and d1 = 10, sum0 = 2 and sum1 = 3, the table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4] = {0,1,2,2,3}, used for the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag. To calculate the context model index, ctxIdBase is first calculated based on d, and then the context model index is calculated by 1 + ctxIdBase + ctxIdTable[min(tmplCpSum1,4)].

[0194] In one example, for the encoding of the syntax element sig_coeff_flag, within the frequency region of d1>d>=d0, and within the interval of sum1>=sumAbs1>=sum0, each transform coefficient with sumAbs1 can share a syntax model, where sum0 and sum1 are predetermined thresholds depending on d0 and d1.

[0195] For example, when d0 = 0 and d1 = 2, sum0 = 1 and sum1 = 3, the table ctxIdSigTable storing the context model index offset can be designed as ctxIdSigTable[0~5] = {0,1,1,1,2,3}. To calculate the context model index, first calculate ctxIdBase based on d and the state, for example, based on expression (2) or (5). Then, the context model index can be calculated by ctxIdBase + ctxIdSigTable[min(sumAbs1,5)]. In another example, the frequency region is defined as d0 = 2 and d1 = 5, and the interval is defined as sum0 = 1 and sum1 = 3. The calculation of the context model index can also be performed similarly.

[0196] As another example, when d0 = 5 and d1 = maxd (maxd represents the maximum diagonal position in the coefficient block), sum0 = 2 and sum1 = 4, the table ctxIdSigTable storing the context model index offset can be designed as ctxIdSigTable[0~5] = {0,1,2,2,2,3}. To calculate the context model index, first calculate ctxIdBase based on d and the state, and then calculate the context model index by ctxIdBase + ctxIdSigTable[min(sumAbs1,5)].

[0197] In one embodiment, transform coefficients within a given frequency region are grouped when the magnitudes of the transform coefficients covered by the local template of each coefficient lie within a defined interval. Transform coefficients within the same group can share the same context model. The intervals can be the same for different frequency regions or different syntax elements.

[0198] In one example, for the encoding of the syntax element sig_coeff_flag, within the frequency domain where d1>d>=d0, and within the interval sum1>=sumAbs1>=sum0, each transform coefficient with sumAbs1 can share a context model. The parameters d0, d1, sum0, and sum1 can be predefined thresholds.

[0199] For example, for a frequency region defined by d0 = 0 and d1 = 2, or d0 = 2 and d1 = 5, or d0 = 5 and d1 = maxd, an interval can be defined by sum0 = 1 and sum1 = 3. The calculation of the context model index can be performed based on the table ctxIdSigTable, which stores the context model index offsets. The table ctxIdSigTable can be designed as ctxIdSigTable[0~5] = {0,1,1,1,2,3}. Similarly, to calculate the context model index, ctxIdBase is first calculated based on d and the state, and then the context model index is calculated by ctxIdBase + ctxIdSigTable[min(sumAbs1,5)].

[0200] In one example, for encoding syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag within the frequency range of d1>d>=d0, a context model can be shared between each transform coefficient with temlCpSum1 within the interval sum1>=tmplCpSum1>=sum0. The parameters d0, d1, sum0, and sum1 are predefined thresholds.

[0201] For example, for a frequency region where d0 = 0 and d1 = 1, or d0 = 1 and d1 = 3, or d0 = 3 and d1 = 10, or d0 = 10 and d1 = maxd, an interval can be defined by sum0 = 3 and sum1 = 4. The table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4] = {0,1,2,3,3}. To calculate the context model index, ctxIdBase can be calculated first based on d, and then the context model index can be calculated by ctxIdBase + ctxIdTable[min(tmplCpSum1,4)].

[0202] As another example, for d0 = 0 and d1 = 1, or d0 = 1 and d1 = 3, or d0 = 3 and d1 = 10, or d0 = 10 and d1 = maxd, an interval can be defined by sum0 = 1 and sum1 = 2. The table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4] = {0,1,1,2,3}. To calculate the context model index, ctxIdBase can be calculated first based on d, and then the context model index can be calculated by ctxIdBase + ctxIdTable[min(tmplCpSum1,4)].

[0203] In one example, for encoding the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag of transform coefficients from chroma components within the frequency region d1>d>=d0, a context model can be shared between each transform coefficient with tmplCpsum1 within the interval sum1>=tmplCpsum1>=sum0. The parameters d0, d1, sum0, and sum1 are predefined thresholds.

[0204] For example, for a frequency region defined by d0 = 0 and d1 = 1, or d0 = 1 and d1 = maxd, an interval is defined by sum0 = 2 and sum1 = 4. The table ctxIdTable storing the context model index offset can be designed as ctxIdTable[0~4] = {0,1,2,2,2}. To calculate the context model index, ctxIdBase can be calculated first based on d, and then the context model index can be calculated by ctxIdBase + ctxIdTable[min(tmplCpSum1,4)].

[0205] In one embodiment, the range of possible template sizes for the transform coefficients of the transform block is divided into multiple sub-intervals. For a given frequency region, the transform coefficients can be grouped into different groups based on which sub-interval each transform coefficient's template size falls into. For syntax elements derived from color components, transform coefficients within the same group can share the same context model.

[0206] The following describes an exemplary process for grouping transform coefficients according to template size subintervals. A first template size for the first transform coefficient can be determined. Then, the first template size can be limited based on a template size limiting threshold to obtain a limited template size, denoted by tmplMgcli. The limiting operation can be performed as follows:

[0207] tmplMgcli = min(first template size, template size limit threshold).

[0208] Subsequently, the sub-interval indicator can be determined based on the template size of the amplitude limit according to the following formula:

[0209] indi=(tmplMgCli+a)>>b,

[0210] Here, `ind1` represents a sub-interval indicator indicating the first sub-interval to which the first template size belongs, parameter `a` represents the offset, and parameter `b` represents the shift value, where `a` and `b` are non-negative integers. Thus, the first sub-interval is determined based on the amplitude-limited template size. The context model can be determined based on the sub-interval indicator of the first transform coefficients.

[0211] In various examples, template size limiting thresholds, offsets, or shift values ​​can be identified in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), title sequence, tile header, or tile group header, or these values ​​can be predefined. For the luminance and chrominance components, the template size limiting thresholds, offsets, or shift values ​​can be different or the same. For different frequency regions, the offsets or shift values ​​can be different or the same. In one example, the template size limiting threshold is defined as infinity, so the first template size is not limited.

[0212] In some examples of the encoding syntax element `sig_coeff_flag`, the template size of the local template is represented by `sumAbs1` of the local template. The limiting threshold used for `sumAbs1` is represented as `sumAbs1Thr`. Therefore, the template size for limiting is calculated using the following formula:

[0213] sumAbs1=min(sumAbs1,sumAbs1Thr).

[0214] Based on the bounded sumAbs1, the range of sumAbs1 can be divided using (sumAbs1+a)>>b. For example, if sumAbs1Thr = 5, a = 0, and b = 1, the range of sumAbs1 can be divided into three sets (or sub-intervals): {0, 1}, {2, 3}, and {4, 5}, each set corresponding to a sub-interval of the bounded sumAbs1. Specifically, for sumAbs1 of sets {0, 1}, {2, 3}, or {4, 5}, the value calculated from (sumAbs1+a)>>b is 0, 1, or 2, respectively. The calculated value is a sub-interval indicator, which can be used as the basis for determining the context model.

[0215] In one example, for the encoding of the syntax element `sig_coeff_flag` in the luminance component, `sumAbs1Thr` is set to 5, `a` to 1, and `b` to 1. Therefore, the range of the amplitude-limited `sumAbs1` is divided into four sets (or sub-intervals): {0}, {1, 2}, {3, 4}, and {5}. Within `d1>d>d0`, each transform coefficient with amplitude-limited `sumAbs1` in the same set can share a context model. For example, the context model index of `sig_coeff_flag` in luminance is calculated as follows:

[0216] sumAbs1 = min(sumAbs1, 5),

[0217] ctxSig=12*max(0,state-1)+((sumAbs1+1)>>1)+(d<2?8:(d<5?4:0)).

[0218] In one example, for the encoding of the syntax element sig_coeff_flag in the chroma component, sumAbs1Thr is set to 5, a to 1, and b to 2. Therefore, the range of the bounded sumAbs1 is divided into two sets: {0, 1, 2} and {3, 4, 5}. Within d1>d>d0, each transform coefficient with sumAbs1 in the same set shares a context model. For example, the context model index of sig_coeff_flag in chroma can be calculated as follows:

[0219] sumAbs1 = min(sumAbs1, 5),

[0220] ctxSig=4*max(0,state-1)+((sumAbs1+1)>>2)+(d<2?2:0).

[0221] In one example, for the encoding of the syntax element sig_coeff_flag in the chroma component, sumAbs1Thr is set to 3, a to 1, and b to 1. Therefore, the range of the bounded sumAbs1 can be divided into three sets: {0}, {1, 2}, and {3}. Within d1>d>d0, each transform coefficient with bounded sumAbs1 in the same set shares a context model. For example, the context model index of sig_coeff_flag in chroma can be calculated as follows:

[0222] sumAbs1 = min(sumAbs1, 3),

[0223] ctxSig=6*max(0,state-1)+((sumAbs1+1)>>1)+(d<2?3:0).

[0224] In some examples of the encoding syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag, the template size of the local template is represented by tmplCpSum1 of the local template. The clipping threshold used for clipping tmplCpSum1 is represented as tmplCpSum1Thr. Therefore, the clipping tmplCpSum1 can be calculated by the following formula:

[0225] tempCpSum1=min(tempCpSum1,templCpSum1Thr).

[0226] Based on the limited range tmlpCpSum1, the range of the limited range tmplCpSum1 can be divided using (tmplCpSum1+a)>>b. For example, if tmplCpSum1Thr=4, a=0 and b=1, then the range of tmplCpSum1 can be divided into 3 sets (or sub-intervals), namely {0, 1}, {2, 3} and {4}.

[0227] In one example, for encoding the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag in the luma component, the parameters are set to tmplCpSum1Thr = 4, a = 1, and b = 1. Therefore, the range of tmplCpSum1 can be divided into three sets (or sub-intervals): {0}, {1, 2}, and {3, 4}. Within d1 > d > d0, each transform coefficient with tmplCpSum1 in the same set shares a context model. For example, the context model index for par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag in the luma component can be calculated as follows:

[0228] tmplCpSum1=min(tmplCpSum1,tmplCpSum1Thr),

[0229] ctxIdx=1+((tmplCpSum1+1)>>1)+(d==0?9:(d<3?6:(d<10?3:0))).

[0230] In one example, for encoding the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag in the chroma component, the parameters are set to tmplCpSum1Thr = 2, a = 1, and b = 1. Therefore, the range of tmplCpSum1 can be partitioned into two sets {0} and {1, 2}. Within d1 > d > d0, each transform coefficient with tmplCpSum1 in the same set shares a context model. For example, the context model index for par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag in the chroma component can be calculated as follows:

[0231] tmplCpSum1=min(tmplCpSum1,2),

[0232] ctxIdx=1+((tmplCpSum1+1)>>1)+(d==0?2:0).

[0233] In one embodiment, for the encoding of the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag, the possible difference range tmplCpsum1 of the transform coefficients in each different frequency region of the transform block is divided into multiple sub-intervals. A set of transform coefficients in each of the multiple sub-intervals in different frequency regions can share the same context model.

[0234] The following describes an exemplary process for grouping transform coefficients based on tmplCpSum1 and diagonal positions. A first template size for the first transform coefficient can be calculated. The first template size is bounded according to a template size bounding threshold to obtain a bounded template size, denoted by tmplMgcli. Based on the bounded template size and the diagonal positions of the first transform coefficients, the original context model index, denoted by OrgCtxIdx, can be determined. Based on the original context model index, the context model can be determined according to the following formula:

[0235] ctxIdx=OrgCtxIdx-((OrgCtxIdx+m)>>n),

[0236] Where ctxIdx represents the context model index of the context model, m represents the offset, n represents the shift value, and m and n are non-negative integers.

[0237] In various examples, the template size limiting threshold, offset, or shift value can be identified in the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), title sequence, tile header, or tile group header, or the template size limiting threshold, offset, or shift value can be predefined. For the luminance and chrominance color components, the template size limiting threshold, offset, or shift value can be different or the same. In one example, the offset or shift value depends on the template size limiting threshold.

[0238] In one example, for the encoding of the syntax elements par_level_flag, rem_abs_gt1_flag, or rem_abs_gt2_flag in the luma component, tmplCpSum1Thr is set to 4, and the context model index can be calculated as follows:

[0239] tmplCpSum1=min(tmplCpSum1,tmplCpSum1Thr),

[0240] OrgCtxIdx=1+tmplCpSum1+(d==0?15:(d<3?10:(d<10?5:0))),

[0241] ctxIdx=OrgCtxIdx-((OrgCtxIdx-m)>>n).

[0242] For example, parameters m and n are set to m=1 and n=2, respectively. For tmplCpSum1Thr=4, m=1 and n=2, the division of the tmplCpSum1 range in different frequency regions is shown below:

[0243] (i) for frequency region d=0, the range of tmplCpSum1 is divided into sets {0, 1}, {2}, {3} and {4};

[0244] (ii) for frequency region 0<d<3, the range of tmplCpSum1 is divided into sets {0}, {1, 2}, {3} and {4};

[0245] (iii) for frequency region 3 ≤ d<10, the range of tmplCpSum1 is divided into sets {0}, {1}, {2, 3} and {4}; and

[0246] (iv) for frequency region d ≥ 10, the range of tmplCpsum1 is divided into sets {0}, {1}, {2} and {3, 4}.

[0247] In one example, for encoding the syntax element par_level_flag, rem_abs_gt1_flag or rem_abs_gt2_flag in a chroma component, tmplCpSum1Thr is set to 2, and the context model index can be calculated as follows:

[0248] tmplCpSum1=min(tmplCpSum1,tmplCpSum1Thr),

[0249] OrgCtxIdx=1+tmplCpSum1+(d==0? 3:0),

[0250] ctxIdx=OrgCtxIdx-((OrgCtxIdx-m)>>n).

[0251] For example, the parameters m and n are set to m=1 and n=1 respectively. For tmplCpSum1Thr=2, m=1 and n=1, the division of the range of tmplCpSum1 for different frequency regions is as follows:

[0252] (i) for frequency region d=0, the range of tmplCpSum1 is divided into sets {0, 1} and {2}; and

[0253] (ii) for frequency region d>0, the range of tmplCpSum1 is divided into sets {0} and {1, 2}.

[0254] In various embodiments, as an alternative to the diagonal position d defined as d=x+y, where x and y represent coordinates of a transform coefficient, a second diagonal position d' is defined as d'=max(x, y) or d'=min(x, y). The second diagonal position is used in place of the diagonal position d in the embodiments.

[0255] Figure 13A A flowchart of an entropy decoding process (1300) according to one embodiment of this application is shown. The process (1300) can be used to entropy decode several types of transform coefficient syntax elements based on the context model reduction method disclosed herein. In various embodiments, the process (1300) can be executed by processing circuitry, such as processing circuitry in terminal devices (210), (220), (230), and (240), processing circuitry performing the functions of a video decoder (310), processing circuitry performing the functions of a video decoder (410), etc. In some embodiments, the process (1300) is implemented as software instructions, so that the processing circuitry executes the process (1300) when the software instructions are executed. The process begins at step S1301 and proceeds to step S1310.

[0256] In step S1310, a bit stream including encoded bits can be received. Encoded segments can be encoded from segments of various types of syntax elements corresponding to the transform coefficients of transform blocks in the encoded image. For example, various types of syntax elements may include validity flag syntax elements, parity flag syntax elements, syntax elements greater than 1, and syntax elements greater than 2.

[0257] In step S1320, context modeling can be performed to determine a context model for each segment of a grammatical element. During context modeling, the context model reduction method disclosed herein is used, such that, for example, less context model is maintained in memory compared to other related context modeling techniques. In one embodiment, within a given frequency region of a transform block, a set of transform coefficients can share the same context model for a given grammatical element. Each transform coefficient in the transform coefficient set has a template size within a predetermined range. At least two template sizes are different from each other.

[0258] Alternatively, within the frequency region, for different template sizes that may exist within a predetermined range, one of the transform coefficients uses the same context model for a type of syntax element.

[0259] The template size for the corresponding transform coefficients can be defined as the sum of the partially reconstructed absolute transform coefficient levels within the local template (denoted by sumAbs1), or the difference between the sum of the partially reconstructed absolute transform coefficient levels and the number of non-zero coefficients in the local template (denoted by numSig) (denoted by tmplCpsum1). In other examples, the template size can be defined in other ways to represent the size of the transform coefficients or transform coefficient levels.

[0260] A predetermined range can be specified using the diagonal position of the transformation coefficients. In some examples, the predetermined range may be one of several sub-intervals divided from the template size using the partitioning techniques disclosed herein.

[0261] In step S1330, the encoded bits are decoded according to the context model determined in step S1320 to recover segments of syntax elements. Based on the recovered segments, the transform coefficient levels of the transform coefficients can be reconstructed. The process (1300) proceeds to step S1399 and ends at step S1399.

[0262] Figure 13B A schematic diagram of a video decoding apparatus according to one embodiment is shown. Figure 13B As shown, the video decoding device includes:

[0263] The receiving module 1321 is used to receive a bit stream, the bit stream including encoded bits of multiple syntax elements, the syntax elements including various types of transform coefficients corresponding to transform blocks in the encoded image;

[0264] Execution module 1322 is configured to perform context modeling to determine a context model for each segment of grammatical elements, wherein, in a given frequency region of the transform block, for one type of the grammatical element: a set of transform coefficients with different template sizes within a predetermined range share the same context model, or, any possible different template sizes of any transform coefficient use the same context model, the possible different template sizes being within the predetermined range; and

[0265] Decoding module 1323 is used to decode the encoded bits based on the context model determined for the syntax element to determine the multi-segment syntax element.

[0266] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 14 A computer system (1400) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0267] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through decoding, microcode, or other means.

[0268] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0269] Figure 14The components shown for the computer system (1400) are exemplary in nature and are not intended to impose any limitation on the scope or functionality of the computer software used to implement embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any or all of the components illustrated in the exemplary embodiments of the computer system (1400).

[0270] The computer system (1400) may include certain human-machine interface input devices. These devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0271] Human-machine interface input devices may include one or more of the following (only one is depicted): keyboard (1401), mouse (1402), touchpad (1403), touch screen (1410), data glove (not shown), joystick (1405), microphone (1406), scanner (1407), camera (1408).

[0272] The computer system (1400) may also include certain human-machine interface (HMI) output devices. These HMI output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. The HMI output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1410), data gloves (not shown), or joystick (1405), but may also include tactile feedback devices that are not input devices), audio output devices (e.g., speakers (1409), headphones (not shown)), visual output devices (e.g., screens (1410) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each with or without touchscreen input functionality, tactile feedback functionality—some of which may output two-dimensional or higher-dimensional visual outputs through means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0273] The computer system (1400) may also include human-accessible storage devices and their associated media, such as optical media including media (1421) such as CD / DVD ROM / RW (1420) with CD / DVD, thumb drives (1422), removable hard disk drives or solid-state drives (1423), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), etc.

[0274] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this application does not include transmission media, carrier waves, or other transient signals.

[0275] The computer system (1400) may also include interfaces to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicle network and industrial network, a real-time network, a latency-tolerant network, etc. The network also includes LANs such as Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicle and industrial networks (including CANBus), etc. Some networks typically require external network interface adapters to connect to certain general-purpose data ports or peripheral buses (1449) (e.g., a USB port on the computer system (1400)); other systems are typically integrated into the core of the computer system (1400) via system buses as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (1400) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0276] The aforementioned human-machine interface device, human-accessible storage device, and network interface can be connected to the kernel (1440) of the computer system (1400).

[0277] The kernel (1440) may include one or more central processing units (CPU) (1441), graphics processing units (GPUs) (1442), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (1443), hardware accelerators (1444) for specific tasks, etc. These devices, along with read-only memory (ROM) (1445), random access memory (1446), and internal mass storage (e.g., internal non-user-accessible hard disk drives, SSDs, etc.) (1447), can be connected via the system bus (1448). In some computer systems, the system bus (1448) can be accessed via one or more physical connectors for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the kernel's system bus (1448) or connected via a peripheral bus (1449). Peripheral bus architectures include external controller interfaces (PCI), universal serial buses (USB), etc.

[0278] The CPU (1441), GPU (1442), FPGA (1443), and accelerator (1444) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1445) or RAM (1446). Transient data can also be stored in RAM (1446), while permanent data can be stored, for example, in internal mass storage (1447). Fast storage and retrieval of any memory device can be achieved through the use of a cache, which can be closely associated with one or more CPUs (1441), GPUs (1442), mass storage (1447), ROM (1445), RAM (1446), etc.

[0279] The computer-readable medium may have computer code for performing various computer implementation operations. The medium and computer code may be specially designed and constructed for the purposes of this application, or they may be media and code well known and usable by those skilled in the art of computer software.

[0280] By way of example and not limitation, a computer system having an architecture (1400), particularly a kernel (1440), can function as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, as well as specific memory of the kernel (1440) having a non-transitory nature, such as internal kernel mass storage (1447) or ROM (1445). Software implementing various embodiments of this application can be stored in such a device and executed by the kernel (1440). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the kernel (1440), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute a specific process or a specific portion of a specific process described herein, including defining data structures stored in RAM (1446) and modifying such data structures according to a software-defined process. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (1444)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0281] Appendix A: Acronyms

[0282] JEM: Joint Exploration Model

[0283] VVC: Universal Video Coding

[0284] BMS: benchmark set

[0285] MV: Motion Vector

[0286] HEVC: High Efficiency Video Coding

[0287] SEI: Supplementary Enhancement Information

[0288] VUI: Video Usability Information

[0289] GOP: Groups of Pictures

[0290] TU: Transform Units

[0291] PU: Prediction Units

[0292] CTU: Coding Tree Units

[0293] CTB: Coding Tree Blocks

[0294] PB: Prediction Blocks

[0295] HRD: Hypothetical Reference Decoder

[0296] SNR: Signal Noise Ratio

[0297] CPU: Central Processing Unit

[0298] GPU: Graphics Processing Units

[0299] CRT: Cathode Ray Tube

[0300] LCD: Liquid Crystal Display

[0301] OLED: Organic Light-Emitting Diode

[0302] CD: Compact Disc

[0303] DVD: Digital Video Disc

[0304] ROM: Read-Only Memory

[0305] RAM: Random Access Memory

[0306] ASIC: Application-Specific Integrated Circuit

[0307] PLD: Programmable Logic Device

[0308] LAN: Local Area Network

[0309] GSM: Global System for Mobile Communications

[0310] LTE: Long-Term Evolution

[0311] CANBus: Controller Area Network Bus

[0312] USB: Universal Serial Bus

[0313] PCI: Peripheral Component Interconnect

[0314] FPGA: Field Programmable Gate Array

[0315] SSD: Solid-state drive

[0316] IC: Integrated Circuit

[0317] CU: Coding Unit

[0318] While this application has described several exemplary embodiments, various modifications, substitutions, and alternatives to the embodiments are within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, while not explicitly shown or described herein, embody the principles of this application and are thus within its spirit and scope.

Claims

1. A method of video decoding, the method comprising: include: Receive a bit stream, the bit stream comprising encoded bits of multiple segments of syntax elements of transform coefficients in a transform block of an encoded image, the syntax elements indicating the properties of the transform coefficients; For each transform coefficient in a given frequency region of the transform block, context modeling is performed to determine the context model of the plurality of segments of the syntax element; as well as Based on the context model determined for the plurality of segments of the syntax element, the encoded bitstream is decoded to determine the segments of the syntax element; The step of performing context modeling to determine the context model of the plurality of segments of the syntax element for each transform coefficient in a given frequency region of the transform block includes: Determine the template size for the corresponding transform coefficients, where the template size is a value representing the size of the local template for the corresponding transform coefficients; Identify one of a plurality of sub-intervals of possible values ​​of the template size, wherein the determined template size belongs to the identified sub-interval, the possible values ​​of the template size are divided into the plurality of sub-intervals, and the number of the plurality of sub-intervals is less than the number of possible values ​​of the template size; A context model is identified among candidate context models for encoding segments of each syntax element of the corresponding transform coefficients, the identified context model being used to encode segments of a subset of the syntax elements of a subset of the transform coefficients, and the subset of transform coefficients being located in the given frequency region and having a corresponding template size within the identified sub-interval.

2. The method according to claim 1, characterized in that, The local template of the corresponding transform coefficient covers the adjacent transform coefficients, which are processed before the corresponding transform coefficients according to the predetermined scan order of the transform block.

3. The method according to claim 1, characterized in that, The given frequency region is defined based on the diagonal position of each transform coefficient within the given frequency region, wherein the diagonal position is within a predetermined diagonal position range, and The position of the diagonal is determined according to the following formula: d = x + y Where d represents the diagonal position, and x and y are the coordinates of the corresponding transformation coefficients within the transformation block.

4. The method according to claim 3, characterized in that, The given frequency region is a low-frequency region, which is defined based on the diagonal position d being less than or equal to a first diagonal position threshold.

5. The method according to claim 3, characterized in that, The given frequency region is a high-frequency region, which is defined based on the diagonal position d being greater than or equal to the second diagonal position threshold.

6. The method according to claim 3, characterized in that, The transformation coefficients are composed of chromaticity and color components, and the given frequency region is the entire transformation block.

7. The method according to claim 3, characterized in that, The given frequency region is defined by the diagonal position d, which is within the following range: d1 > d > d0. d1 and d0 are predefined diagonal position thresholds.

8. The method according to claim 1, characterized in that, The syntax element of the corresponding transformation coefficient is one of the following types: The first type of syntax element, represented by sig_coeff_flag, indicates whether the absolute transformation coefficient level of the corresponding transformation coefficient is zero or greater than zero, wherein the absolute transformation coefficient level of the corresponding transformation coefficient is represented by absLevel; The second type of syntax element, represented by par_level_flag, indicates the parity check of the absolute transformation coefficient level of the corresponding transformation coefficient when the absolute transformation coefficient level of the corresponding transformation coefficient is not zero. The third type of syntax element, represented by `rem_abs_gt1_flag`, indicates whether the (absLevel-1) >> 1 of the corresponding transform coefficient is greater than 0 when the absolute transform coefficient level of the corresponding transform coefficient is not zero; and The fourth type of syntax element, represented by rem_abs_gt2_flag, indicates whether the absolute transformation coefficient level of the corresponding transformation coefficient is greater than 4 when (absLevel-1)>>1 is greater than 0.

9. The method according to claim 8, characterized in that, The template size of the corresponding transformation coefficients is defined as one of the following: The sum of the absolute transformation coefficients of the partial reconstruction within the local template; and The difference between the sum of the absolute transform coefficient levels of the partially reconstructed model and the number of non-zero coefficients in the local template, the difference being determined according to the following formula: tmplCpSum1=sumAbs1–numSig, Where tmplCpSum1 represents the difference, sumAbs1 represents the sum of the absolute transformation coefficients of the partial reconstruction within the local template, and numSig represents the number of non-zero coefficients in the local template.

10. The method according to claim 9, characterized in that, One of the sub-intervals among the multiple sub-intervals identifying possible values ​​of the template size includes: The template size of the corresponding transform coefficients is limited according to the following formula, based on a template size limiting threshold, to obtain the limited template size represented by tmplMgcli: tmplMgCli = min(the template size, the template size limiting threshold); and The sub-interval indicator is determined based on the template size of the amplitude limit according to the following formula: indi=(tmplMgCli+a)>>b, Wherein, indi represents an indicator used to indicate one of the multiple sub-intervals to which the template size belongs, a represents an offset, b represents a shift value, and a and b are non-negative integers.

11. The method according to claim 10, characterized in that, The template size limiting threshold, the offset, or the shift value are signaled in the sequence parameter set, image parameter set, title sequence, block header, or block group header, or the template size limiting threshold, the offset, or the shift value are predefined.

12. The method according to claim 10, characterized in that, The template size limiting threshold is defined as infinity so that the template size is not limited.

13. The method according to claim 10, characterized in that, The syntax element for the corresponding transform coefficient is one of par_level_flag, rem_abs_gt1_flag, and rem_abs_gt2_flag. The template size for the corresponding transformation coefficients is tmplCpSum1.

14. The method according to claim 10, characterized in that, The syntax element for the corresponding transform coefficients is sig_coeff_flag. The template size for the corresponding transformation coefficients is sumAbs1.

15. A video decoding device, characterized in that, include: A receiving module is configured to receive a bit stream comprising encoded bits of multiple segments of syntax elements of transform coefficients in a transform block of an encoded image, the syntax elements indicating the properties of the transform coefficients. An execution module is configured to perform context modeling for each transform coefficient in a given frequency region of the transform block to determine the context model of the plurality of segments of the syntax element; as well as A decoding module is used to decode the encoded bitstream based on a context model determined for the plurality of segments of the syntax element, so as to determine the segments of the syntax element; The execution module is specifically configured to: determine the template size of the corresponding transform coefficient, wherein the template size is a value representing the size of the local template of the corresponding transform coefficient; identify a sub-interval among multiple sub-intervals of possible values ​​of the template size, wherein the determined template size belongs to the identified sub-interval, the possible values ​​of the template size are divided into the multiple sub-intervals, and the number of the multiple sub-intervals is less than the number of possible values ​​of the template size; and identify a context model among candidate context models for encoding segments of each syntax element of the corresponding transform coefficient, wherein the identified context model is used to encode segments of a subset of the syntax elements of a subset of the transform coefficients, and the subset of transform coefficients is located in the given frequency region and has a corresponding template size within the identified sub-interval.

16. A video encoding method, characterized in that, include: Receive video samples and various types of syntax elements from a video source. The syntax elements include multiple segments and correspond to the transform coefficients of transform blocks in the video samples. For each transform coefficient in a given frequency region of the transform block, context modeling is performed to determine the context model of the plurality of segments of the syntax element; as well as The video samples are encoded into an encoded bitstream based on a context model determined for the plurality of segments of the syntax element, the bitstream including the encoded bits of the plurality of segments of the syntax element as transform coefficients in the transform block; For each transform coefficient in a given frequency region of the transform block, performing context modeling to determine the context model of the plurality of segments of the syntax element includes: Determine the template size for the corresponding transform coefficients, where the template size is a value representing the size of the local template for the corresponding transform coefficients; Identify one of a plurality of sub-intervals of possible values ​​of the template size, wherein the determined template size belongs to the identified sub-interval, the possible values ​​of the template size are divided into the plurality of sub-intervals, and the number of the plurality of sub-intervals is less than the number of possible values ​​of the template size; A context model is identified among candidate context models for encoding segments of each syntax element of the corresponding transform coefficients, the identified context model being used to encode segments of a subset of the syntax elements of a subset of the transform coefficients, and the subset of transform coefficients being located in the given frequency region and having a corresponding template size within the identified sub-interval.

17. A video encoding device, characterized in that, include: A receiving module is used to receive video samples and various types of syntax elements from a video source. The syntax elements include multiple segments and correspond to the transform coefficients of transform blocks in the video samples. An execution module is configured to perform context modeling for each transform coefficient in a given frequency region of the transform block to determine the context model of the plurality of segments of the syntax element; as well as An encoding module is configured to encode the video sample into an encoded bitstream based on a context model determined for the plurality of segments of the syntax element, the bitstream comprising the encoded bits of the plurality of segments of the syntax element as transform coefficients in the transform block; The execution module is further configured to: determine the template size of the corresponding transform coefficient, the template size being a value representing the size of the local template of the corresponding transform coefficient; identify a sub-interval among a plurality of sub-intervals of possible values ​​of the template size, wherein the determined template size belongs to the identified sub-interval, the possible values ​​of the template size are divided into the plurality of sub-intervals, and the number of the plurality of sub-intervals is less than the number of possible values ​​of the template size; identify a context model among candidate context models for encoding segments of each syntax element of the corresponding transform coefficient, the identified context model being used to encode segments of a subset of the syntax elements of a subset of the transform coefficients, and the subset of transform coefficients being located in the given frequency region and having a corresponding template size within the identified sub-interval.

18. A computer device comprising a processor and a memory connected to the processor, characterized in that, The memory stores machine-readable instructions, which the processor executes to perform the method as described in any one of claims 1 to 14 or 16.

19. A non-volatile computer-readable medium, characterized in that, It stores a program executable by a processor, which executes the program to perform the method as described in any one of claims 1 to 14 or 16.

Images