Video decoding method, encoding method, electronic device and storage medium

Abstract

The present application discloses a video decoding method, a coding method, an electronic device and a storage medium. One of the video decoding methods includes receiving a coded video bitstream, the coded video bitstream including a current picture and at least one syntax element corresponding to a transform coefficient of a transform block in the current picture. The method further includes determining an offset value based on an output of a monotonic non-decreasing function performed on a cumulative value sum(x) of a set of partially reconstructed transform coefficients. The method further includes determining a context model index based on a sum of the determined offset value and a base value. The method further includes selecting a context model for the at least one syntax element of the current transform coefficient from a plurality of context models based on the determined context model index.

Description

[0001] References merged

[0002] This disclosure claims priority to U.S. Patent Application No. 16 / 904,000, filed June 17, 2020, entitled “Method and Apparatus for Reducing Context Models for Entropy Coding of Transformer Significant Flag,” which claims priority to U.S. Provisional Application No. 62 / 863,742, filed June 19, 2019, entitled “Method of Reducing Context Models for Entropy Coding of Transformer Significant Flag,” which is incorporated herein by reference in its entirety. Technical Field

[0003] This application describes embodiments that generally relate to video encoding and decoding. Background Technology

[0004] The background description provided herein is intended to present the overall context of this application. The extent of the work of the currently named inventors described in the background section and various aspects of this specification does not imply that it was prior art at the time of filing of this application, nor is it expressly or implied that it was acknowledged as prior art to this application.

[0005] Video encoding and decoding can be performed using inter-frame prediction techniques with motion compensation. Uncompressed digital video can comprise a series of images, each with spatial dimensions, for example, 1920×1080 luminance samples and correlated chrominance samples. The series of images has a fixed or variable image rate (also informally referred to as the frame rate), for example, 60 images per second or 60Hz. Uncompressed video has very high bitrate requirements. For example, a 1080p60 4:2:0 video with 8 bits per sample (1920x1080 luminance sample resolution, 60Hz frame rate) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require over 600 GB of storage space.

[0006] One goal of video encoding and decoding is to reduce redundant information in the input video signal through compression. Video compression can help reduce bandwidth or storage requirements by two or more orders of magnitude in some cases. Lossless and lossy compression, as well as combinations of both, can be used. Lossless compression refers to the technique of reconstructing an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be exactly the same as the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal can be used for the intended application. Lossy compression is widely used in video. The amount of distortion tolerated depends on the application. For example, users of some consumer streaming applications may tolerate higher distortion than users of television applications. The achievable compression ratio reflects that higher allowable / tolerable distortion results in a higher compression ratio.

[0007] Video encoders and decoders can utilize several major categories of techniques, including motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques may include known intra-frame coding techniques. In intra-frame coding, sample values ​​are represented without reference to samples or other data from a previously reconstructed reference image. In some video codecs, an image is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the image can be an intra-frame image. Intra-frame images and their derivatives (e.g., independent decoder refresh images) can be used to reset the decoder state and are therefore used as the first image in the encoded video bitstream and video session, or as still images. Samples from an intra-frame block can be used for transform, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value and the smaller the AC coefficients, the fewer bits are needed to represent the entropy-coded block for a given quantization step size.

[0009] As is known from technologies such as MPEG-2, traditional intra-frame coding does not use intra-frame prediction. However, some newer video compression techniques include those that attempt to derive data blocks from, for example, surrounding sample data and / or metadata, which are obtained during spatially adjacent encoding / decoding and prior to the decoding sequence. This technique is later referred to as "intra-frame prediction." It is important to note that, at least in some cases, intra-frame prediction uses only reference data from the current frame being reconstructed, and not reference data from a reference frame.

[0010] There can be many different forms of intra-prediction. When more than one such technique can be used in a given video coding technique, the techniques used can be coded in intra-prediction modes. In some cases, a mode may have sub-modes and / or parameters, and these modes may be encoded individually or contained in mode codewords. Which codeword is used for a given mode / sub-mode / parameter combination will affect the coding efficiency gain through intra-prediction, and this also applies to entropy coding techniques used to convert codewords into bitstreams.

[0011] H.264 introduced an intra-frame prediction mode, which was improved in H.265 and further refined in newer coding techniques such as Joint Exploration Model (JEM), Universal Video Coding (VVC), and Baseline Set (BMS). Prediction blocks are formed by using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the prediction block in a specific direction. References to the direction used can be encoded in the bitstream or can be predicted themselves.

[0012] Motion compensation can be a lossy compression technique and can involve using sample data blocks obtained from a previously reconstructed image or a portion thereof (the reference image), spatially shifted in a direction indicated by a motion vector (hereinafter referred to as MV), to predict a newly reconstructed image or image portion. In some cases, the reference image can be the same as the image currently being reconstructed. The MV can have two dimensions: an X dimension and a Y dimension, or three dimensions, with the third dimension indicating the reference image in use (the latter can indirectly be a temporal dimension).

[0013] In some video compression techniques, a motion vector (MV) applicable to a given sample data region can be predicted from other MVs, for example, from another sample data region spatially adjacent to the region being reconstructed, whose decoding order precedes that MV. This substantially reduces the amount of data required to encode and decode the MV, thereby eliminating redundancy and enhancing compression. MV prediction can be performed efficiently, for example, because when encoding an input video signal derived from a camera (called natural video), there is statistical likelihood—that is, multiple regions larger than the area applicable to a single MV move in similar directions—so in some cases, MV prediction can be performed using similar motion vectors derived from multiple MVs of adjacent regions. This results in the MV found for a given region being similar or identical to the MV predicted from the surrounding MVs, and after entropy encoding and decoding, it can be represented with fewer bits than would be needed to directly encode and decode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., the MV) derived from the original signal (i.e., the sample stream). In other cases, MV prediction itself can be lossy, for example, because rounding errors occur when calculating predictors from several surrounding MVs.

[0014] H.265 / HEVC (ITU-T H.265 Recommendation, “High Efficiency Video Coding”, December 2016) describes various MV prediction mechanisms. Among the various MV prediction mechanisms provided by H.265, this application describes the technique hereinafter referred to as “spatial combining”.

[0015] Please refer to Figure 1 The current block (101) includes samples that have been discovered by the encoder during the motion search process, and these samples can be predicted based on previous blocks of the same size that have generated spatial offsets. Alternatively, the MV can be derived from metadata associated with one or more reference images, rather than being directly encoded. For example, using the MV associated with any of the five surrounding samples A0, A1 and B0, B1, B2 (corresponding to 102 to 106 respectively), the MV is derived from the metadata of the nearest reference image (in decoding order). In H.265, MV prediction can use predictions from the same reference image that is also being used in adjacent blocks. Summary of the Invention

[0016] According to an exemplary embodiment, a video decoding method performed in a video decoder includes receiving an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image. The method further includes determining an offset value based on the output of a monotonically non-decreasing function performed on a set of at least partially reconstructed transform coefficients, sum(x). The method further includes determining a context model index based on the sum of the determined offset value and a base value. The method further includes selecting a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

[0017] According to an exemplary embodiment, a video decoding method performed in a video decoder includes receiving an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image. The method further includes, for each of a plurality of context model regions, determining the output of a monotonically non-decreasing function, the monotonically non-decreasing function being performed on a cumulative value sum(x) of a set of at least partially reconstructed transform coefficients and the number of context models associated with each context model region. The method further includes determining a context model index based on the output of the monotonically non-decreasing function for each context model region. The method further includes selecting a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

[0018] According to an exemplary embodiment, a video decoder for video decoding includes processing circuitry. The processing circuitry is configured to receive an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image. The processing circuitry is further configured to determine an offset value based on the output of a monotonically non-decreasing function performed on a set of at least partially reconstructed transform coefficients, sum(x). The processing circuitry is further configured to determine a context model index based on the sum of the determined offset value and a base value. The processing circuitry is further configured to select a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

[0019] According to an exemplary embodiment, a video decoder for video decoding includes processing circuitry. The processing circuitry is configured to receive an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image. The processing circuitry is further configured to, for each of a plurality of context model regions, determine the output of a monotonically non-decreasing function, the monotonically non-decreasing function being performed on a cumulative value sum(x) of a set of at least partially reconstructed transform coefficients and the number of context models associated with each context model region. The processing circuitry is further configured to determine a context model index based on the output of the monotonically non-decreasing function for each context model region. The processing circuitry is further configured to select a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

[0020] According to an exemplary embodiment, a non-volatile computer-readable medium stores instructions that, when executed by a computer, cause the computer to perform the method described above. Attached Figure Description

[0021] Other features, properties, and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, wherein:

[0022] Figure 1 This is a schematic diagram of the current block and its surrounding space as potential merge candidates in an example;

[0023] Figure 2 This is a simplified block diagram of a communication system according to one embodiment;

[0024] Figure 3 This is a simplified block diagram of a communication system according to another embodiment;

[0025] Figure 4 This is a simplified block diagram of a decoder according to one embodiment;

[0026] Figure 5 This is a simplified block diagram of an encoder according to one embodiment;

[0027] Figure 6 A block diagram of an encoder according to another embodiment is shown;

[0028] Figure 7 A block diagram of a decoder according to another embodiment is shown;

[0029] Figure 8A An exemplary context-based adaptive binary arithmetic codec (CABAC) entropy encoder according to an embodiment is shown;

[0030] Figure 8B An exemplary CABAC-based entropy decoder according to an embodiment is shown;

[0031] Figure 9 An example of the sub-block scan order according to an embodiment is shown;

[0032] Figure 10 An example of a sub-block scanning method according to an embodiment is shown, from which different types of syntax elements of transform coefficients are generated;

[0033] Figure 11 An example of a local template for contextual selection of the current coefficient is shown;

[0034] Figure 12 This shows the diagonal position of the coefficients or coefficient levels within the coefficient block;

[0035] Figure 13 The illustration shows the context index calculation of the luminance component according to an embodiment;

[0036] Figure 14 The illustration shows the context index calculation of the luminance component according to an embodiment;

[0037] Figure 15 The illustration shows the context index calculation of the luminance component according to an embodiment;

[0038] Figure 16 A flowchart outlining the entropy decoding method according to an embodiment is shown;

[0039] Figure 17 A flowchart outlining the entropy decoding method according to an embodiment is shown;

[0040] Figure 18 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0041] Figure 2 This is a simplified block diagram of a communication system (200) according to an embodiment disclosed in this application. The communication system (200) includes a plurality of terminal devices that can communicate with each other via, for example, a network (250). For example, the communication system (200) includes a first terminal device (210) and a second terminal device (220) interconnected via a network (250). Figure 2In this embodiment, the first terminal device (210) and the second terminal device (220) perform unidirectional data transmission. For example, the first terminal device (210) may encode video data (e.g., a video image stream captured by the terminal device (210)) for transmission over a network (250) to the second terminal device (220). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (220) may receive the encoded video data from the network (250), decode the encoded video data to recover the video data, and display video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0042] In another embodiment, the communication system (200) includes a third terminal device (230) and a fourth terminal device (240) that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (230) and the fourth terminal device (240) may encode video data (e.g., a stream of video images captured by the terminal device) for transmission over a network (250) to the other terminal device. Each of the third terminal device (230) and the fourth terminal device (240) may also receive encoded video data transmitted by the other terminal device and may decode the encoded video data to recover the video data, and may display the video images on an accessible display device based on the recovered video data.

[0043] exist Figure 2 In the embodiments disclosed herein, the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240) may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (250) refers to any number of networks that transmit encoded video data between the first terminal device (210), the second terminal device (220), the third terminal device (230), and the fourth terminal device (240), including, for example, wired (connected) and / or wireless communication networks. The communication network (250) may exchange data in circuit-switched and / or packet-switched channels. The network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (250) may be irrelevant to the operation of this application.

[0044] As an example, Figure 3The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0045] The streaming system may include an acquisition subsystem (313) that may include a video source (301) such as a digital camera, which creates an uncompressed video image stream (302). In an embodiment, the video image stream (302) includes samples captured by a digital camera. The video image stream (302) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (304) (or encoded video bitstream). The video image stream (302) may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (302), the encoded video data (304) (or the encoded video bitstream (304)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (304) (or the encoded video bitstream (304)), which can be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as Figure 3 Client subsystems (306) and (308) can access a streaming server (305) to retrieve copies (307) and (309) of encoded video data (304). Client subsystem (306) may include, for example, a video decoder (310) in an electronic device (330). The video decoder (310) decodes the incoming copy (307) of the encoded video data and produces an output video picture stream (311) that can be displayed on a display (312) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (304), video data (307), and video data (309) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0046] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0047] Figure 4 This is a block diagram of a video decoder (410) according to an embodiment disclosed in this application. The video decoder (410) may be disposed in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., receiving circuitry). The video decoder (410) may be used in place of... Figure 3 The video decoder (310) in the embodiment.

[0048] The receiver (431) may receive one or more encoded video sequences to be decoded by the video decoder (410); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (401), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (431) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (431) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (415) may be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter referred to as "parser (420)"). In some applications, the buffer memory (415) is part of the video decoder (410). In other cases, the buffer memory (415) may be located external to the video decoder (410) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (410) to prevent network jitter, for example, and another buffer (415) may be configured internally for, for example, handling broadcast timing. When the receiver (431) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (415) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (415) may be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (410).

[0049] The video decoder (410) may include a parser (420) to reconstruct symbols (421) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (410) and potential information for controlling a display device (412) (e.g., a display screen), which is not part of the electronic device (430) but may be coupled to it, such as... Figure 4 As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (420) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (420) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (420) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0050] The parser (420) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0051] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (421) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed by the parser (420) from the encoded video sequence. For brevity, the flow of such subgroup control information between the parser (420) and the various units described below is not described.

[0052] In addition to the functional blocks already mentioned, the video decoder (410) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0053] The first unit is the scaler / inverse transform unit (451). The scaler / inverse transform unit (451) receives quantization transform coefficients as symbols (421) and control information from the parser (420), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (451) can output a block containing sample values, which can be input into the aggregator (455).

[0054] In some cases, the output samples of the scaler / inverse transform unit (451) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (452). In some cases, the intra-picture prediction unit (452) uses reconstructed information extracted from the current picture buffer (458) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current picture buffer (458) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (455) adds the predictive information generated by the intra-picture prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451) based on each sample.

[0055] In other cases, the output samples of the scaler / inverse transform unit (451) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (453) can access the reference image memory (457) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (421), these samples can be added by the aggregator (455) to the output of the scaler / inverse transform unit (451) (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation prediction unit (453) can obtain the prediction samples from the address in the reference image memory (457) under motion vector control, and the motion vector is available to the motion compensation prediction unit (453) in the form of the symbols (421), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (457) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0056] The output samples of the aggregator (455) can be employed by various loop filtering techniques in the loop filter unit (456). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (421) from the parser (420) in the loop filter unit (456). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0057] The output of the loop filter unit (456) can be a sample stream, which can be output to a display device (412) and stored in a reference image memory (457) for subsequent inter-frame image prediction.

[0058] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded image corresponding to the current image has been fully reconstructed and the encoded image (by, for example, the parser (420)) is identified as the reference image, the current image buffer (458) can become part of the reference image memory (457), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0059] The video decoder (410) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0060] In this embodiment, the receiver (431) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0061] Figure 5 This is a block diagram of a video encoder (503) according to an embodiment disclosed in this application. The video encoder (503) is disposed in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used to replace... Figure 3 The video encoder (303) in the embodiment.

[0062] The video encoder (503) can obtain data from the video source (501) (not) Figure 5 In one embodiment, a portion of the electronic device (520) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (503). In another embodiment, the video source (501) is a portion of the electronic device (520).

[0063] A video source (501) can provide a sequence of source video samples encoded by a video encoder (503) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (501) can be a storage device storing previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed in sequence. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0064] According to an embodiment, the video encoder (503) can encode and compress images of a source video sequence into an encoded video sequence (543) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (550). In some embodiments, the controller (550) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (550) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) may be used with other suitable functions related to the video encoder (503) optimized for a particular system design.

[0065] In some embodiments, the video encoder (503) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (530) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (533) embedded within the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques considered in this application, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (534). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory (534) are also bit-precisely corresponding between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0066] The operation of the “local” decoder (533) can be combined with, for example, the above-described method. Figure 4 The video decoder (410) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 4 When symbols are available and the entropy encoder (545) and parser (420) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (410), including the buffer (415) and parser (420), may not be fully implemented in the local decoder (533).

[0067] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0068] During operation, in some embodiments, the source encoder (530) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (532) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0069] The local video decoder (533) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (530). The operation of the encoding engine (532) can be a lossy process. When the encoded video data can be decoded by the video decoder (533), Figure 5 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (533) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (534). In this way, the video encoder (503) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0070] The predictor (535) can perform a prediction search against the encoding engine (532). That is, for a new image to be encoded, the predictor (535) can search in the reference image memory (534) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (535) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (535), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (534).

[0071] The controller (550) can manage the encoding operations of the source encoder (530), including, for example, setting parameters and subgroup parameters for encoding video data.

[0072] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (545). The entropy encoder (545) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0073] The transmitter (540) can buffer the encoded video sequence created by the entropy encoder (545) in preparation for transmission via a communication channel (560), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can combine the encoded video data from the video encoder (503) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0074] The controller (550) manages the operation of the video encoder (503). During encoding, the controller (550) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0075] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics.

[0076] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0077] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0078] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0079] The video encoder (503) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (503) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0080] In this embodiment, the transmitter (540) may transmit additional data while transmitting encoded video. The source encoder (530) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0081] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0082] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0083] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0084] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0085] Figure 6 This is a diagram of a video encoder (603) according to another embodiment disclosed in this application. The video encoder (603) is used to receive processing blocks (e.g., prediction blocks) of sample values ​​within a current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (603) is used instead of Figure 3The video encoder (303) in the embodiment.

[0086] In the HEVC embodiment, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (603) uses, for example, rate-distortion (RD) optimization to determine whether to use an intra-frame mode, an inter-frame mode, or a bidirectional prediction mode to encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (603) can use intra-frame prediction techniques to encode the processing block into an already encoded picture; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (603) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into an already encoded picture, respectively. In some video coding techniques, the merging mode can be an inter-frame picture prediction sub-mode, in which motion vectors are derived from one or more motion vector prediction values ​​without relying on already encoded motion vector components outside the prediction values. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the embodiment, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0087] exist Figure 6 In one embodiment, the video encoder (603) includes, as shown below: Figure 6 The inter-frame encoder (630), intra-frame encoder (622), residual calculator (623), switch (626), residual encoder (624), general controller (621) and entropy encoder (625) are shown coupled together.

[0088] An inter-frame encoder (630) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., redundancy information description, motion vectors, merging mode information based on inter-frame coding techniques), and calculate inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image based on encoded video information.

[0089] The intra encoder (622) is used to receive samples of the current block (e.g., the processing block), in some cases compare the block with previously encoded blocks in the same image, generate quantization coefficients after transformation, and in some cases also (e.g., based on intra prediction direction information of one or more intra coding techniques) generate intra prediction information. In an embodiment, the intra encoder (622) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same image.

[0090] A general-purpose controller (621) determines general-purpose control data and controls other components of the video encoder (603) based on the general-purpose control data. In an embodiment, the general-purpose controller (621) determines the mode of a block and provides control signals to a switch (626) based on the mode. For example, when the mode is an intra-frame mode, the general-purpose controller (621) controls the switch (626) to select an intra-frame mode result for use by the residual calculator (623) and controls the entropy encoder (625) to select intra-frame prediction information and add the intra-frame prediction information to the bitstream; and when the mode is an inter-frame mode, the general-purpose controller (621) controls the switch (626) to select an inter-frame prediction result for use by the residual calculator (623) and controls the entropy encoder (625) to select inter-frame prediction information and add the inter-frame prediction information to the bitstream.

[0091] A residual calculator (623) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (622) or the inter encoder (630). A residual encoder (624) is used to operate on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (624) is used to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (622) and the inter encoder (630). For example, the inter encoder (630) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (622) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, and in some embodiments, the decoded image may be buffered in a memory circuit (not shown) and used as a reference image.

[0092] An entropy encoder (625) is used to format the bitstream to produce encoded blocks. The entropy encoder (625) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (625) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0093] Figure 7This is a diagram of a video decoder (710) according to another embodiment disclosed in this application. The video decoder (710) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (710) is used instead of Figure 3 The video decoder (310) in the embodiment.

[0094] exist Figure 7 In this embodiment, the video decoder (710) includes, as follows: Figure 7 The entropy decoder (771), inter-frame decoder (780), residual decoder (773), reconstruction module (774), and intra-frame decoder (772) are shown coupled together.

[0095] An entropy decoder (771) can be used to reconstruct certain symbols from an encoded image, these symbols representing the syntax elements constituting the encoded image. Such symbols may include, for example, a mode for encoding the block (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a combined sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used by the intra-frame decoder (772) or the inter-frame decoder (780) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In an embodiment, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (780); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (772). Residual information may be provided to the residual decoder (773) via inverse quantization.

[0096] The inter-frame decoder (780) is used to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0097] The intra-frame decoder (772) is used to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0098] The residual decoder (773) performs inverse quantization to extract the dequantized transform coefficients and processes the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require some control information (to obtain the quantizer parameters QP), and this information may be provided by the entropy decoder (771) (the data path is not indicated because this is only low-level control information).

[0099] The reconstruction module (774) is used to combine the residual output by the residual decoder (773) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0100] It should be noted that any suitable technology can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In one embodiment, one or more integrated circuits can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (303), video encoder (503), and video encoder (603), as well as the video decoder (310), video decoder (410), and video decoder (710).

[0101] Entropy encoding / decoding can be performed in the final stage of video encoding / decoding (or the first stage of video decoding) after the video signal has been reduced to a series of syntax elements. Entropy encoding / decoding can be a lossless compression scheme that uses statistical properties to compress data such that the number of bits used to represent the data is logarithmically proportional to the probability of the data. For example, by performing entropy encoding / decoding on a set of syntax elements, the bits representing the syntax elements (called binary numbers) can be converted into fewer bits in the bitstream (called encoded bits). Context-based Adaptive Binary Arithmetic Encoding / Decoding (CABAC) is a form of entropy encoding / decoding. In CABAC, a context model is determined to provide a probability estimate for each binary number in a series of binary numbers. The series of binary numbers is based on the context associated with each binary number. The probability estimates are then used to perform a binary arithmetic encoding / decoding process to encode the series of binary numbers into encoded bits in the bitstream. Furthermore, the context model is updated with a new probability estimate based on the encoded binary numbers.

[0102] Figure 8A An exemplary CABAC-based entropy encoder (800A) according to an embodiment is shown. For example, the entropy encoder (800A) can be implemented in... Figure 5 The entropy codec (545) in the example or Figure 6In the example, the entropy encoder (625) is used. The entropy encoder (800A) may include a context modeler (810) and a binary arithmetic encoder (820). In the example, various types of syntax elements are provided as inputs to the entropy encoder (800A). For example, the binary number of a binary value syntax element can be directly input to the context modeler (810), while non-binary value syntax elements can first be binaryized into a binary string before the binary number of the binary string is input to the context modeler (810).

[0103] In the example, the context modeler (810) receives binary numbers of syntax elements and performs a context modeling process to select a context model for each received binary number. For example, it receives binary numbers of binary syntax elements representing transform coefficients in a transform block. Therefore, a context model is determined for this binary number based on factors such as the type of the syntax element, the color component type of the transform component, the position of the transform coefficient, and previously processed adjacent transform coefficients. The context model can provide a probability estimate of this binary number.

[0104] In the example, a set of context models is configured for each type of syntax element. These context models can be arranged in a list (802) of context models stored in memory (801), such as Figure 8A As shown. Each entry in the context model list (802) can represent a context model. An index called the context model index or the context index is assigned to each context model in the list. In addition, each context model can include a probability estimate or a parameter indicating the probability estimate. The probability estimate can indicate the likelihood that the binary number is 0 or 1. For example, during context modeling, the context modeler (810) can compute the context index of the binary number and select a context model from the context model list (802) accordingly based on the context index and assign it to that binary number.

[0105] Furthermore, at the start of the operation of the entropy encoder (800A), the probability estimates in the context model list are initialized. After the context model on the context model list (802) is assigned to a binary number and used to encode that binary number, the context model is subsequently updated based on the value of the binary number with the updated probability estimates.

[0106] In the example, the binary arithmetic encoder (820) receives a binary number and a context model (e.g., a probability estimate) assigned to the binary number, and performs a binary arithmetic encoding / decoding process accordingly. Thus, encoded bits are generated and transmitted in the bitstream.

[0107] Figure 8BAn exemplary CABAC-based entropy decoder (800B) according to an embodiment is shown. For example, the entropy decoder (800B) can be implemented in... Figure 4 The parser (420) in the example or Figure 7 In the example entropy decoder (771), the entropy decoder (800B) may include a binary arithmetic decoder (830) and a context modeler (840). The binary arithmetic decoder (830) receives encoded bits from the bitstream and performs a binary arithmetic decoding process to recover binary numbers from the encoded bits. The context modeler (840) may operate similarly to the context modeler (810). For example, the context modeler (840) may select a context model from a list (804) of context models stored in memory (803) and provide the selected context model to the binary arithmetic decoder (830). However, the context modeler (840) determines the context model based on the binary number recovered from the binary arithmetic decoder (830). For example, based on the recovered binary number, the context modeler (840) may know the type of the syntax elements of the next binary number to be decoded, as well as the values ​​of previously decoded syntax elements. This information is used to determine the context model of the next binary number to be decoded.

[0108] In an embodiment, the residual signal of the transform block is first transformed from the spatial domain to the frequency domain to obtain a block of transform coefficients. Then, quantization is performed to quantize the block of transform coefficients into blocks at the transform coefficient level. In various embodiments, different techniques may be used to convert the residual signal into the transform coefficient level. The blocks at the transform coefficient level are further processed to generate syntax elements, which are then provided to an entropy encoder and encoded into bits of a bitstream. In an embodiment, the process of generating syntax elements from the transform coefficient level is performed as follows.

[0109] First, the block at the transform coefficient level is divided into sub-blocks, for example, with a size of 4×4. These sub-blocks are then processed according to a predefined scan order. Figure 9 An example of the sub-block scanning order is shown, referred to as the reverse diagonal scanning order. As shown, block (910) is divided into 16 sub-blocks (901). The lower right sub-block is processed first, and the upper left sub-block is processed last. In the example, for a sub-block in which all transformation coefficient levels are zero, the sub-block is skipped without processing.

[0110] For each subblock with at least one non-zero transform coefficient level, four scans are performed in each subblock. During each scan, 16 positions in each subblock are scanned in reverse diagonal scan order. Figure 10 An example of the subblock scan method (1000) is shown, from which different types of syntax elements of transform coefficients are generated.

[0111] The sixteen coefficient positions (1010) within the sub-block are in Figure 10 The bottom is shown in one dimension. Positions (1010) are numbered from 0 to 15, reflecting the various scan orders. During the first pass, scan positions (1010) are scanned, and three types of syntax elements (1001 to 1003) can be generated at each scan position (1010):

[0112] (i) The first type of binary syntax element (1001) (called the validity flag and represented by sig_coeff_flag) indicates whether the absolute transform coefficient level (represented by absLevel) of each transform coefficient is zero or greater than zero.

[0113] (ii) A second type of binary syntax element (1002) (called the parity flag and represented by par_level_flag) indicates the parity of the absolute transform coefficient level of each transform coefficient. The parity flag is generated only if the absolute transform coefficient level of each transform coefficient is non-zero.

[0114] (iii) The third type of binary syntax element (1003) (called the larger 1 flag and represented by rem_abs_gt1_flag) indicates whether (absLevel-1) >> 1 is greater than 0 for each transform coefficient. The larger 1 flag is generated only if the absolute transform coefficient level of each transform coefficient is non-zero.

[0115] During the second pass, a fourth type of binary syntax element (1004) may be generated. This fourth type of binary syntax element (1004) is called the Larger 2 flag and is represented by `rem_abs_gt2_flag`. The fourth type of binary syntax element (1004) indicates whether the absolute transform level of each transform coefficient is greater than 4. The Larger 2 flag is generated only if (absLevel-1) >> 1 for each transform coefficient and is greater than 0.

[0116] During the third pass, a fifth-type non-binary syntax element (1005) may be generated. The fifth-type non-binary syntax element (1005) is represented by `abs_remainder` and indicates the residual value of the absolute transform coefficient level of the corresponding transform coefficient greater than 4. The fifth-type non-binary syntax element (1005) is generated only if the absolute transform coefficient level of each transform coefficient is greater than 4.

[0117] During the fourth pass, a sixth-type syntax element (1006) is generated at each scan position (1010). The sixth-type syntax element (1006) has a non-zero coefficient level indicating the sign of each transformation coefficient level.

[0118] Based on the order of each pass and the scan order within each pass, the various types of syntax elements (1001 to 1006) are provided to the entropy encoder. Different entropy encoding schemes are used to encode different types of syntax elements. For example, in an embodiment, the validity flag, parity flag, greater 1 flag, and greater 2 flag can be encoded using a CABAC-based entropy encoder, such as in... Figure 8A As shown in the example. Conversely, the syntax elements generated during the third and fourth passes can be encoded using a CABAC bypass entropy encoder (e.g., a binary arithmetic encoder with a fixed probability estimate for the input binary number).

[0119] Context modeling is performed to determine a context model of the binary number of certain types of transform coefficient syntax elements. In an embodiment, the context model is determined based on the local template and the diagonal position of each current coefficient (e.g., the coefficient currently being processed) that may be combined with other factors.

[0120] Figure 11 An example of a local template (1130) for context selection of the current coefficient is shown. The local template (1130) can cover a set of adjacent positions or coefficients of the current coefficient (1120) within a coefficient block (1110). Figure 11 In the example, the coefficient block (1110) has a size of 8×8 positions and includes coefficient levels at 64 positions. The coefficient block (1110) is divided into 4 sub-blocks, each with a size of 4×4 positions. Figure 11 In the example, the local template (1130) is defined as a 5-position template covering the 5 coefficient levels at the lower right of the current coefficient (1120). When the reverse diagonal scan order is used for multiple scans at scan positions within the coefficient block (1110), adjacent positions within the local template (1130) are processed before the current coefficient (1120).

[0121] During context modeling, information at the coefficient level within the local template (1130) can be used to determine the context model. For this purpose, in some embodiments, a metric called template magnitude is defined to measure or indicate the magnitude of the transform coefficients or the magnitude at the transform coefficient level within the local template (1130). The template magnitude can then be used as the basis for selecting the context model.

[0122] In one example, the template amplitude is defined as the sum of the absolute transform coefficient levels of the partial reconstruction within the local template (1130), denoted by sumAbs1. The absolute transform coefficient level of the partial reconstruction is determined based on the binary representations of the syntax elements sig_coeff_flag, par_level_flag, and rem_abs_gt1_flag for each transform coefficient. These three types of syntax elements are obtained after the first pass of the sub-block during entropy encoder or entropy decoder operations. In this embodiment, the absolute transform coefficient level of the partial reconstruction at position (x, y) is determined according to the following formula:

[0123] Equation (1): absLevel1[x][y]=sig_coeff_flag[x][y]+par_level_flag[x][y]+2*rem_abs_gt1_flag[x][y], where x and y are the coordinates relative to the top left corner of the coefficient block (1110), and absLevel1[x][y] represents the absolute transformation coefficient level of the partial reconstruction at position (x, y).

[0124] In another example, the template amplitude is defined as the difference between the sum of the absolute transform coefficient levels of the local reconstruction and the number of non-zero coefficients in the local template (1130) (denoted by numSig). This difference is denoted by templCpSum1. Therefore, the difference can be determined according to the following formula:

[0125] Equation (2): tmplCpSum1=sumAbs1-numSig.

[0126] In other examples, the template amplitude is defined in other ways to indicate the amplitude of the transform coefficients or the amplitude of the transform coefficient level.

[0127] In some embodiments, to take advantage of the correlation between transform coefficients, in the context selection of the current coefficient, the coefficient is selected by... Figure 11The local template shown covers the previously encoded coefficients. The position with the square crosshair (1120) indicates the current transform coefficient position (x, y), and the position with the diagonal crosshair indicates its five nearest neighbors. Let AbsLevelPass1[x][y] represent the absolute level of the partial reconstruction of the coefficient at position (x, y) after the first pass, d represent the diagonal position of the current coefficient (d = x + y), and sumAbs1 represent the sum of the absolute levels AbsLevelPass1[x][y] of the partially reconstructed coefficients covered by the local template. The syntax element AbsLevelPass1[x][y] can be calculated based on the syntax elements sig_coeff_flag[xC][yC], abs_level_gtx_flag[n][0], par_level_flag[n], and abs_level_gtx_flag[n][1], where abs_level_gtx_flag[n][0] and abs_level_gtx_flag[n][1] can also be referred to as... Figure 10 The rem_abs_gt1_flag and rem_abs_gt2_flag of the coefficient at position n.

[0128] Figure 12 The diagonal positions of the coefficients or coefficient levels within the coefficient block (1210) are shown. In this embodiment, the diagonal position of the scan position (x, y) is defined according to the following formula:

[0129] Equation (3): d = x + y,

[0130] Where d represents the diagonal position, and x and y are the coordinates of each position. The diagonal position d of each coefficient can be used to define different frequency regions within the coefficient block (1210) based on one or two diagonal position thresholds. As two examples, the low-frequency region (1220) is defined as d≤3, while the high-frequency region (1230) is defined as d≥11, as follows. Figure 12 As shown.

[0131] In some embodiments, when encoding / decoding the current coefficient's sig_coeff_flag[x][y], the context model index is selected based on the value of sumAbs1 and the diagonal position d. More specifically, as... Figure 13 As shown, for the luminance component, the context model index is determined according to the following:

[0132] Equation (4): offset = min(sumAbs1,5)

[0133] Equation (5): base = 18 * max(0, state-1) + (d < 2? 12: (d < 5? 6: 0))

[0134] Equation (6): ctxSig = base + offset

[0135] For chroma components, the context model index is determined as follows:

[0136] Equation (7): offset = min(sumAbs1,5)

[0137] Equation (8): base = 12 * max(0, state-1) + (d < 2? 6: 0)

[0138] Equation (9): ctxSig = base + offset,

[0139] Where `base` is the base value, `offset` is the offset value, and `state` specifies the scalar quantizer used, with the operators "?" and ":" as defined in the C programming language. If dependent quantization is enabled, `state` is derived using a state transition procedure. Otherwise, dependent quantization is disabled, and `state` equals 0.

[0140] In some examples, the number of context models used to encode and decode sig_coeff_flag[x][y] is 54 for luminance and 36 for chrominance. Therefore, the total number of context models used to encode and decode sig_coeff_flag[x][y] is 90, which is greater than 21% of the 424 context models in the standardized context modeling scheme. The standardized context modeling scheme is, for example, VVC Draft 5.

[0141] Table 1 illustrates an example of residual encoding / decoding syntax. In Table 1, xC corresponds to the x-coordinate of the current coefficient in the transform block, and yC corresponds to the y-coordinate of the current coefficient in the transform block.

[0142] Table 1

[0143]

[0144]

[0145]

[0146]

[0147]

[0148] As the number of context models increases, the complexity of hardware and software also increases. Therefore, it is desirable to reduce the number of context models without sacrificing encoding and decoding efficiency. In particular, it is desirable to reduce the number of context models used for encoding and decoding transform coefficient significance, as it accounts for more than 21% of the 424 context models in the standardized context modeling scheme in VVC Draft 5.

[0149] The embodiments of the present disclosure may be used alone or combined in any order. Furthermore, each of the method, encoder and decoder according to the embodiments of the present disclosure may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. According to the embodiments of the present disclosure, the term "block" may be interpreted as a prediction block, an encoding and decoding block, or an encoding and decoding unit (i.e., CU).

[0150] According to some embodiments, a region is defined as a set of connected transform coefficient positions. For example, a region is a set of transform coefficient positions (x, y) such that d0≤x+y<d1 for some non-negative integers d0 and d1 referred to as position thresholds. The embodiments of the present disclosure can be applied to entropy encoding and decoding techniques for the transform coefficient significance flag (sig_coeff_flag) with the following parameters:

[0151] (i) N is the number of context models per region. In one example implementation, N is equal to 4. In another example implementation, N is equal to 5.

[0152] (ii) d 0Y and d 1Y are diagonal position thresholds for luma regions. In one example implementation, d 0Y is 2 and d 1Y is 5.

[0153] (iii) d 0C is the diagonal position threshold for chroma regions. In one example implementation, d 0C is 2.

[0154] (iv) f(x) is a monotonically non-decreasing function that maps a set of non-negative integers to a set of non-negative integers.

[0155] (v) When N is 5, the implementation of the function f(x) is defined as

[0156] f(x) = x – (x>>2).

[0157] (vi) When N is 4, the implementation of the function f(x) is defined as

[0158] f(x) = (x+1)>>1.

[0159] According to some embodiments, when encoding / decoding the current coefficient's sig_coeff_flag[x][y], the context model index is selected based on the value of sumAbs1 and the diagonal position d. More specifically, as... Figure 14 As shown, for the luminance component, in some embodiments the context model index is determined according to the following:

[0160] Equation (10): offset = min(f(sumAbs1), N-1)

[0161] Equation (11): base = 3 * N * max(0, state-1) + (d <d0 Y ? 2*N:(d <d1 Y ? N:0))

[0162] Equation (12): ctxSig = base + offset

[0163] For chroma components, the context model index is determined as follows:

[0164] Equation (13): offset = min(f(sumAbs1), N-1)

[0165] Equation (14): base = 2 * N * max(0, state-1) + (d <d0 C ? N:0)

[0166] Equation (15): ctxSig = base + offset

[0167] Where relevant quantization is enabled, `state` specifies the scalar quantizer used, and `state` is derived using a state transition process. If relevant quantization is not enabled, `state` equals 0 in some examples. Furthermore, in some embodiments, such as... Figure 15 As shown, when N is 4 or 5, for lower hardware complexity, the function min(f(sumAbs1),N-1) can also be implemented as f(min(sumAbs1,5)).

[0168] The standardized context modeling scheme in VVC Draft 5 has 90 context models for encoding and decoding the validity of transform coefficients. In embodiments of this disclosure, the number of context models is reduced from 90 to 75 when N equals 5, and from 90 to 60 when N equals 4.

[0169] According to some embodiments, the monotonically non-decreasing function f(x) of a non-negative integer x can be defined as:

[0170] Equation (16): in And b i It is an integer value. Furthermore, a i It can be 0, 1, or -1 to reduce calculations.

[0171] According to some embodiments, the context region depends on the diagonal position d, such that the number of context models for each region can depend on the diagonal position d, further reducing the number of contexts. For example, having (d <d 0Y ), (d 0Y ≤d <d 1Y ) and (d 1Y ≤d <d 2Y The number of context models for each region are N1, N2, and N3, respectively. Specifically, the number of context models can vary based on the value of d. In this case, the context model index can be determined as follows:

[0172] Equation (17): g1(x)=min(f1(x),N1-1)

[0173] Equation (18): g2(x)=min(f2(x),N2-1)

[0174] Equation (19): g3(x)=min(f3(x),N3-1)

[0175] Equation (20): ctxSig=(N1+N2+N3)*max(0,state-1)+(d <d 0Y ? (N2+N3)+g1(sumAbs1):

[0176] (d <d 1Y ? N3+g2(sumAbs1):g3(sumAbs1))),

[0177] Where f1(x), f2(x), and f3(x) are monotonically non-decreasing functions of non-negative integers x. Example values ​​of N1, N2, and N3 can be integer values ​​from 1 to 16. Implementations including equations (17) through (20) provide greater flexibility by reducing the number of contexts with the same bit rate.

[0178] Option embodiments of this disclosure can be applied to entropy encoding / decoding techniques with a validity flag for transform coefficients having the following parameters:

[0179] (i) N is the number of context models for each region. In this implementation, N equals 4.

[0180] (ii)d 0YThis is the diagonal position threshold of the brightness region. In this embodiment, d 0Y It is 5.

[0181] (iii)d 0C This is the diagonal position threshold of the chromaticity region. In this embodiment, d 0C It is 2.

[0182] (iv) When N is 4, the function f(x) of a non-negative integer x can be defined as:

[0183] f(x) = (x+1) >> 1

[0184] According to some embodiments, when encoding / decoding the current coefficient's sig_coeff_flag[x][y], the context model index is selected based on sumAbs1 and the diagonal position d, wherein for the luminance component, the context model index is determined as follows:

[0185] Equation (21): offset = min(f(sumAbs1), N-1)

[0186] Equation (22): base = 2 * N * max(0, state-1) + (d <d 0Y ? N:0)

[0187] Equation (23): ctxSig = base + offset

[0188] For chroma components, the context model index is determined as follows:

[0189] Equation (24): offset = min(f(sumAbs1), N-1)

[0190] Equation (25): base = 2 * N * max(0, state-1) + (d <d 0C ? N:0)

[0191] Equation (26): ctxSig = base + offset

[0192] Here, `state` specifies the scalar quantizer used when correlated quantization is enabled, and `state` is derived using the state transition method. Otherwise, if correlated quantization is not enabled, `state` equals 0.

[0193] In some embodiments, for lower hardware complexity, the function min(f(sumAbs1),N-1) can also be implemented as f(min(sumAbs1,5)).

[0194] The standardized context model scheme in VVC Draft 5 has 90 context models for encoding and decoding the validity of transform coefficients. In the previously disclosed alternative embodiment (i.e., equations (21) to (26)), the number of context models is reduced from 90 to 48 when N equals 4.

[0195] Figure 16 An embodiment of a method performed by a decoder, such as a video decoder (710), is shown. The method begins at step (S1600), where the encoded video bitstream includes at least one syntax element of the current image and transform coefficients corresponding to transform blocks in the current image. As an example, the at least one syntax element could be sig_coeff_flag. The method proceeds to step (S1602), where an offset value is determined based on the output of a monotonically non-decreasing function f(x) performed on the cumulative value sum(x) of at least one set of partially reconstructed transform coefficients. The method proceeds to step (S1604), where a context model index is determined based on the sum of the determined offset value and the base value. As an example, according to... Figure 14 and 15 The method illustrated in one of the figures or according to the alternative embodiments disclosed above (i.e., equations (21) to (26)) determines the context model index. The method proceeds to step (S1606), wherein for at least one grammatical element of the current transform coefficient, a context model is selected from a plurality of context models based on the determined context model index.

[0196] Figure 17An embodiment of a method performed by a decoder, such as a video decoder (710), is shown. The method begins at step (S1700), where the encoded video stream includes at least one syntax element of the current picture and transform coefficients corresponding to transform blocks in the current picture. As an example, the at least one syntax element may be sig_coeff_flag. The method proceeds to step (S1702), where, for each of a plurality of context model regions, the output of a monotonically non-decreasing function is performed on the cumulative value sum(x) of a set of at least partially reconstructed transform coefficients and the number of context models associated with each context model region. For example, the functions g1(x) = min(f1(x), N1-1), g2(x) = min(f2(x), N2-1), and g3(x) = min(f3(x), N3-1) disclosed above can be used for each context model region, where the number of context models in each region (i.e., N1, N2, N3) varies based on the distance from the top left corner of the transform block to the current transform coefficient. The method proceeds to step (S1704), where a context model index is determined based on the output of a monotonically non-decreasing function for each context model region. The method proceeds to step (S1706), where, for at least one grammatical element of the current transform coefficient, a context model is selected from a plurality of context models based on the determined context model index.

[0197] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 18 A computer system (1800) is shown, which is adapted to implement certain embodiments of the disclosed subject matter.

[0198] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through decoding, microcode, etc.

[0199] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0200] Figure 18 The components shown for the computer system (1800) are exemplary in nature and are not intended to limit the scope or functionality of the computer software used to implement the embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of the computer system (1800).

[0201] The computer system (1800) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media, which need not be directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0202] Human-machine interface input devices may include one or more of the following (only one is shown): keyboard (1801), mouse (1802), touchpad (1803), touch screen (1810), data glove (not shown), joystick (1805), microphone (1806), scanner (1807), camera (1808).

[0203] The computer system (1800) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (1810), data gloves (not shown), or joystick (1805), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1809), headphones (not shown)), visual output devices (e.g., screens (1810) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality—some of which may output two-dimensional or more three-dimensional visual outputs by means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0204] The computer system (1800) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (1820) or similar media (1821), thumb drives (1822), removable hard disk drives or solid-state drives (1823), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), and so on.

[0205] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0206] The computer system (1800) may also include interfaces to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicle-mounted network and an industrial network, a real-time network, a latency-tolerant network, and so on. Networks also include Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicle-mounted and industrial networks (including CANBus), and so on. Some networks typically require external network interface adapters for connection to certain general-purpose data ports or peripheral buses (1849) (e.g., the USB port of the computer system (1800); other systems are typically integrated into the core of the computer system (1800) via a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (1800) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0207] The aforementioned human-computer interface device, human-accessible storage device, and network interface can be connected to the core (1840) of the computer system (1800).

[0208] The core (1840) may include one or more central processing units (CPU) (1841), graphics processing units (GPUs) (1842), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (1843), task-specific hardware accelerators (1844), etc. These devices, along with read-only memory (ROM) (1845), random access memory (1846), and internal mass storage (e.g., internal non-user-accessible hard disk drives, solid-state drives, etc.) (1847), can be connected via a system bus (1848). In some computer systems, the system bus (1848) may be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (1848) or connected via a peripheral bus (1849). Peripheral bus architectures include external controller interfaces (PCI), universal serial buses (USB), etc.

[0209] The CPU (1841), GPU (1842), FPGA (1843), and accelerator (1844) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (1845) or RAM (1846). Transient data can also be stored in RAM (1846), while permanent data can be stored, for example, in internal mass storage (1847). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1841), GPUs (1842), mass storage (1847), ROM (1845), RAM (1846), etc.

[0210] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this application, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0211] By way of example and not limitation, a computer system having an architecture (1800), particularly a core (1840), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the non-volatile core (1840), such as internal mass storage (1847) or ROM (1845). Software implementing various embodiments of this application can be stored in such a device and executed by the core (1840). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (1840), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute a specific process or a specific portion of a specific process described herein, including defining data structures stored in RAM (1846) and modifying such data structures according to a software-defined process. Alternatively or as an alternative, a computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (1844)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0212] Appendix A: Acronyms

[0213] JEM: Joint Exploration Model

[0214] VVC: Versatile Video Coding

[0215] BMS: Benchmark set

[0216] MV: Motion Vector

[0217] HEVC: High Efficiency Video Coding

[0218] SEI: Supplementary Enhancement Information

[0219] VUI: Video Usability Information

[0220] GOPs: Groups of Pictures

[0221] TUs: Transform Units

[0222] PUs: Prediction Units

[0223] CTUs: Coding Tree Units

[0224] CTBs: Coding Tree Blocks

[0225] PBs: Prediction Blocks

[0226] HRD: Hypothetical Reference Decoder

[0227] SNR: Signal-to-Noise Ratio

[0228] CPUs: Central Processing Units

[0229] GPUs: Graphics Processing Units

[0230] CRT: Cathode Ray Tube

[0231] LCD: Liquid Crystal Display

[0232] OLED: Organic Light-Emitting Diode

[0233] CD: Compact Disc

[0234] DVD: Digital Video Disc

[0235] ROM: Read-Only Memory

[0236] RAM: Random Access Memory

[0237] ASIC: Application-Specific Integrated Circuit

[0238] PLD: Programmable Logic Device

[0239] LAN: Local Area Network

[0240] GSM: Global System for Mobile Communications

[0241] LTE: Long-Term Evolution

[0242] CANBus: Controller Area Network Bus

[0243] USB: Universal Serial Bus

[0244] PCI: Peripheral Component Interconnect

[0245] FPGA: Field Programmable Gate Array

[0246] SSD: Solid-state drive

[0247] IC: Integrated Circuit

[0248] CU: Coding Unit. Although this application has described several exemplary embodiments, various modifications, arrangements, and equivalent substitutions of the embodiments are all within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are therefore within the spirit and scope of this application.

[0249] (1) A video decoding method executed in a video decoder, the method comprising: receiving an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image; determining an offset value based on the output of a monotonically non-decreasing function performed on a set of at least partially reconstructed transform coefficients, sum(x); determining a context model index based on the determined offset value and a base value; and selecting a context model from a plurality of context models for at least one syntax element of the current transform coefficients based on the determined context model index.

[0250] (2) According to the method of feature (1), one of the base value and offset value is determined based on the number of context models included in the plurality of context models.

[0251] (3) The method according to feature (2), wherein the method further includes: determining whether to enable correlated quantization for the current transform coefficient, and in response to determining to enable correlated quantization for the current transform coefficient, the base value is determined based on the state of the quantizer.

[0252] (4) According to the method of feature (3), wherein the current transformation coefficient is located in the brightness region, and the base value is determined based on a comparison of the distance of the current transformation coefficient from the upper left corner of the transformation block with a first diagonal position threshold.

[0253] (5) According to the method of feature (4), wherein the base value is further determined based on a comparison of the distance with a second diagonal position threshold.

[0254] (6) According to the method of feature (3), wherein the current transformation coefficient is located in the chromaticity region, and the base value is determined based on a comparison of the distance of the current transformation coefficient from the upper left corner of the transformation block with a first diagonal position threshold.

[0255] (7) According to the method of feature (1), wherein the monotonically non-decreasing function is defined as f(x) = x - (x >> 2). (8) According to the method of feature (1), wherein the monotonically non-decreasing function is defined as f(x) = (x + 1) >> 1.

[0256] (9) The method according to feature (1), wherein the current transform coefficients and the set of at least partially reconstructed transform coefficients form a template, the template constituting a continuous set of transform coefficients.

[0257] (10) According to the method of feature (1), wherein the at least one syntax element is a transform coefficient validity flag sig_coeff_flag.

[0258] (11) The method according to feature (1), wherein the encoded video stream comprises a plurality of syntax elements, the plurality of syntax elements comprising the at least one syntax element, and wherein the cumulative value sum(x) of the set of at least partially reconstructed transform coefficients is based on one or more of the plurality of syntax elements.

[0259] (12) A video decoding method executed in a video decoder, the method comprising: receiving an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image; for each of a plurality of context model regions, determining the output of a monotonically non-decreasing function, the monotonically non-decreasing function being performed on a cumulative value sum(x) of a set of at least partially reconstructed transform coefficients and the number of context models associated with each context model region; determining a context model index based on the output of the monotonically non-decreasing function for each context model region; and selecting a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

[0260] (13) According to the method of feature (12), wherein the determination of the context model index is further determined based on a comparison of the distance of the current transform coefficient from the top left corner of the transform block with a first diagonal position threshold and a second diagonal position threshold.

[0261] (14) According to the method of feature (12), wherein the determination of the context model index is further determined based on a comparison of the distance of the current transform coefficient from the top left corner of the transform block with a first diagonal position threshold.

[0262] (15) A video decoder for video decoding, comprising processing circuitry. The processing circuitry is configured to receive an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image; determine an offset value based on the output of a monotonically non-decreasing function performed on a set of at least partially reconstructed transform coefficients, sum(x); determine a context model index based on the sum of the determined offset value and a base value; and select a context model from a plurality of context models for at least one syntax element of the current transform coefficients based on the determined context model index.

[0263] (16) A video decoder according to feature (15), wherein one of the base value and offset value is determined based on the number of context models included in the plurality of context models.

[0264] (17) The video decoder according to feature (16), wherein the processing circuit is further configured to:

[0265] Determine whether to enable relevant quantization for the current transform coefficients, and

[0266] In response to determining that relevant quantization is enabled for the current transform coefficient, the base value is determined based on the state of the quantizer.

[0267] (18) A video decoder according to feature (17), wherein the current transform coefficient is located in the luminance region and the base value is determined based on a comparison of the distance of the current transform coefficient from the upper left corner of the transform block with a first diagonal position threshold.

[0268] (19) The video decoder according to feature (18), wherein the base value is further determined based on a comparison of the distance with a second diagonal position threshold.

[0269] (20) A video decoder for video decoding, comprising: processing circuitry. The processing circuitry is configured to: receive an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image; for each of a plurality of context model regions, determine the output of a monotonically non-decreasing function, the monotonically non-decreasing function being performed on a cumulative value sum(x) of a set of at least partially reconstructed transform coefficients and the number of context models associated with each context model region; determine a context model index based on the output of the monotonically non-decreasing function for each context model region; and select a context model from a plurality of context models for at least one syntax element of the current transform coefficient based on the determined context model index.

Claims

1. A video decoding method, characterized in that, The method includes: Receive an encoded video stream, the encoded video stream including a current image and at least one syntax element of transform coefficients corresponding to transform blocks in the current image; The offset value is determined based on the minimum of the output of a monotonically non-decreasing function performed on the cumulative values ​​of the absolute transformation coefficients of a set of partially reconstructed regions and the number of context models for each region minus 1. The context model index is determined based on the sum of the determined offset and the base value; wherein the calculation of the base value includes at least 2 × N × max(0, state - 1) + (d < d0). C ? N : 0), where N represents the number of context models for each region, state represents the state of the quantizer, d represents the diagonal position, d0 C This indicates the diagonal position threshold of the current chromaticity region; Based on the determined context model index, a context model is selected from multiple context models for at least one syntactic element of the current transform coefficient.

2. The method according to claim 1, wherein, The method further includes: Determine whether to enable relevant quantization for the current transform coefficients, and In response to determining that relevant quantization is enabled for the current transform coefficient, the base value is determined based on the state of the quantizer.

3. The method according to claim 2, wherein, The current transformation coefficient is located in the brightness region, and the base value is determined based on a comparison between the distance of the current transformation coefficient from the upper left corner of the transformation block and a first diagonal position threshold.

4. The method according to any one of claims 1-3, wherein, The current transform coefficients and a set of at least partially reconstructed absolute transform coefficients form a template, the template constituting a continuous set of transform coefficients.

5. The method according to any one of claims 1-3, wherein, The at least one syntax element is the transform coefficient validity flag sig_coeff_flag.

6. The method according to any one of claims 1-3, wherein, The encoded video stream includes multiple syntax elements, and the cumulative value of the set of partially reconstructed absolute transform coefficient levels is determined based on one or more syntax elements from the multiple syntax elements.

7. The method according to claim 1, characterized in that, Further includes: For each of the multiple context model regions, the output of a monotonically non-decreasing function is determined, which is performed on the cumulative value of the absolute transform coefficients of the set of partially reconstructed regions and the number of context models associated with each context model region.

8. The method according to any one of claims 1-3, characterized in that, The offset value is determined based on the minimum value between the output of a monotonically non-decreasing function applied to the cumulative values ​​of the absolute transform coefficients of a set of partially reconstructed data and the number of context models for each region minus 1, including: The offset value is determined according to the following equation: offset = min(f(sumAbs1), N-1), where offset represents the offset value, sumAbs1 represents the cumulative value of a set of partially reconstructed absolute transform coefficient levels, and f(sumAbs1) represents the output of a monotonically non-decreasing function applied to the cumulative value of a set of partially reconstructed absolute transform coefficient levels.

9. The method according to any one of claims 1-3, characterized in that, The value of N is 4, and the calculation of the base value includes at least 2 × 4 × max(0, state - 1) + (d < d0) C ? 4 : 0).

10. A video encoding method, characterized in that, The method includes: Get the current image; Set at least one syntax element corresponding to the transform coefficients of the transform block in the current image; The offset value is determined based on the minimum of the output of a monotonically non-decreasing function performed on the cumulative values ​​of the absolute transformation coefficients of a set of partially reconstructed regions and the number of context models for each region minus 1. The context model index is determined based on the sum of the determined offset and the base value; wherein the calculation of the base value includes at least 2 × N × max(0, state - 1) + (d < d0). C ? N : 0), where N represents the number of context models for each region, state represents the state of the quantizer, d represents the diagonal position, d0 C This indicates the diagonal position threshold of the current chromaticity region; Based on the determined context model index, a context model is selected from multiple context models for at least one syntactic element of the current transform coefficient.

11. An electronic device, characterized in that, include: The processing circuit is configured to perform the method as described in any one of claims 1-10.

12. A non-volatile computer-readable storage medium, characterized in that, The device stores instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-10.

13. A method for storing video streams, characterized in that, The method of claim 10 is used to generate a video stream; and the video stream is stored.

Images