Coefficient coding for conversion skip mode

The coefficient coding scheme addresses inefficiencies in transform skip mode by using neighboring coefficients to enhance entropy coding, reducing bit overhead and improving efficiency in video encoding and decoding processes.

KR103046621B1Active Publication Date: 2026-11-10QUALCOMM INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
KR1020217028600
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-03-10
Filing Date
2020-03-11
Publication Date
2026-11-10
Estimated Expiration
2040-03-11

AI Technical Summary

Technical Problem

Existing video encoding techniques face inefficiencies in transform skip mode due to the lack of effective entropy coding methods, leading to increased bit overhead and potential degradation in video quality.

Method used

A coefficient coding scheme that determines a context offset based on neighboring coefficients for improved entropy decoding and encoding in transform skip mode, enhancing coding efficiency without degrading video quality.

Benefits of technology

The proposed technique reduces bit overhead and improves overall coding efficiency by leveraging correlations between neighbor coefficients in residual blocks, maintaining video quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112021103119350-PCT00035_ABST
    Figure 112021103119350-PCT00035_ABST
Patent Text Reader

Abstract

A device for decoding video data determines, for a residual block of video data encoded using a convert skip mode, a value for a first neighbor coefficient of a coefficient currently being decoded; determines a value for a second neighbor coefficient of a coefficient currently being decoded; determines a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and decodes a value for a coefficient currently being decoded based on the determined context offset.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present application is as follows:

[0002] U.S. patent application 16 / 814,654 filed on March 10, 2020;

[0003] U.S. provisional patent application 62 / 816,745 filed on March 11, 2019; and

[0004] U.S. provisional patent application 62 / 850,453 filed on May 20, 2019

[0005] Claiming the interests of, the full contents of each are incorporated herein by reference.

[0006] Technology field

[0007] The present disclosure relates to video encoding and video decoding. Background Technology

[0008] Digital video capabilities can be integrated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital aids (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite wireless phones, so-called "smartphones," video teleconferencing devices, video streaming devices, etc. Digital video devices implement video coding techniques such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264 / MPEG-4, Part 10, AVC (Advanced Video Coding), ITU-T H.265, HEVC (High Efficiency Video Coding), and extensions of such standards. Video devices may also transmit, receive, encode, decode, and / or store digital video information more efficiently by implementing such video coding techniques.

[0009] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video picture or a part of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction for reference samples in neighboring blocks of the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction for reference samples in neighboring blocks of the same picture, or temporal prediction for reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. means of solving the problem

[0010] In some coding scenarios, a video encoder may encode video data in transform skip mode, where the transform process is not performed; that is, the transform process is skipped. Consequently, residual data is not transformed for blocks encoded in transform skip mode. This disclosure describes a technique for a coefficient coding scheme to transform in transform skip mode. The technique of this disclosure includes an entropy decoding process that transforms binary representations of coefficients into quantized coefficients rather than a series of binary values. The corresponding entropy encoding process, which is generally the inverse process of entropy decoding, is also part of this disclosure.

[0011] In one example, a method for decoding video data includes the steps of: determining a value for a first neighbor coefficient of a coefficient currently being decoded for a residual block of video data encoded using a transform skip mode; determining a value for a second neighbor coefficient of a coefficient currently being decoded; determining a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and decoding a value for a coefficient currently being decoded based on the determined context offset.

[0012] In another example, a method for encoding video data includes the steps of: determining a value for a first neighbor coefficient of a coefficient currently being encoded for a residual block of video data encoded using a transform skip mode; determining a value for a second neighbor coefficient of a coefficient currently being encoded; determining a context offset for a coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and encoding a value for a coefficient currently being encoded based on the determined context offset.

[0013] In another example, a device for decoding video data includes one or more processors implemented in a memory and circuit portion configured to store video data, and the one or more processors are configured to: determine a value for a first neighbor coefficient of a coefficient currently being decoded for a residual block of video data encoded using a convert skip mode; determine a value for a second neighbor coefficient of a coefficient currently being decoded; determine a context offset for a coefficient currently being decoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and decode a value for a coefficient currently being decoded based on the determined context offset.

[0014] In another example, a device for encoding video data includes one or more processors implemented in a memory and circuit portion configured to store video data, and the one or more processors are configured to: determine a value for a first neighbor coefficient of a coefficient currently being encoded for a residual block of video data encoded using a convert skip mode; determine a value for a second neighbor coefficient of a coefficient currently being encoded; determine a context offset for a coefficient currently being encoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and, based on the determined context offset, encode a value for a coefficient currently being encoded.

[0015] In another example, an apparatus for decoding video data includes means for determining a value for a first neighbor coefficient of a coefficient currently being decoded for a residual block of video data encoded using a transform skip mode; means for determining a value for a second neighbor coefficient of a coefficient currently being decoded; means for determining a context offset for a coefficient currently being decoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and means for decoding a value for a coefficient currently being decoded based on the determined context offset.

[0016] In another example, an apparatus for encoding video data includes means for determining a value for a first neighbor coefficient of a coefficient currently being encoded for a residual block of video data encoded using a transform skip mode; means for determining a value for a second neighbor coefficient of a coefficient currently being encoded; means for determining a context offset for a coefficient currently being encoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and means for encoding a value for a coefficient currently being encoded based on the determined context offset.

[0017] In another example, as a computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, cause one or more processors to determine, for a residual block of video data encoded using a convert skip mode, a value for a first neighbor coefficient of a coefficient currently being decoded; to determine a value for a second neighbor coefficient of a coefficient currently being decoded; to determine a context offset for a coefficient currently being decoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and to decode a value for a coefficient currently being decoded based on the determined context offset.

[0018] In another example, as a computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, cause one or more processors to determine, for a residual block of video data encoded using a convert skip mode, a value for a first neighbor coefficient of a coefficient currently being encoded; to determine a value for a second neighbor coefficient of a coefficient currently being encoded; to determine a context offset for a coefficient currently being encoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient; and to encode a value for a coefficient currently being encoded based on the determined context offset.

[0019] Details of one or more examples are disclosed in the accompanying drawings and the following description. Other features, purposes, and advantages will be apparent from the description, drawings, and claims. Brief explanation of the drawing

[0020] FIG. 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure. FIGS. 2a and 2b are conceptual diagrams illustrating an exemplary quadtree binary tree (QTBT) structure and a corresponding coding tree unit (CTU). Figure 3 illustrates an example of neighboring coefficients of a coefficient currently being encoded or decoded. FIG. 4 is a block diagram illustrating an example video encoder that may perform the technique of this disclosure. FIG. 5 is a block diagram illustrating an example video decoder that may perform the technique of this disclosure. Figures 6a and 6b are conceptual diagrams illustrating the range update process in binary arithmetic coding. Figure 7 is a conceptual diagram illustrating the output process in binary arithmetic coding. Figure 8 is a block diagram illustrating a context-adaptive binary arithmetic coding (CABAC) coder in a video encoder. Figure 9 is a block diagram illustrating a CABAC coder in a video decoder. Figure 10 is a flowchart illustrating a process for encoding video data. Figure 11 is a flowchart illustrating a process for decoding video data. Figure 12 illustrates a process for determining the context for coding the sign of the coefficients of a residual block. Specific details for implementing the invention

[0021] Video coding (e.g., video encoding and / or video decoding) typically involves predicting a block of video data from a block of video data already coded in the same picture (e.g., intra-prediction) or from a block of video data already coded in a different picture (e.g., inter-prediction). In some cases, the video encoder also calculates residual data by comparing the predicted block with the original block. Thus, the residual data represents the difference between the predicted block and the original block. To reduce the number of bits required to signal the residual data, the video encoder may transform and quantize the residual data and signal the transformed and quantized residual data from the encoded bitstream. The compression achieved by the transformation and quantization processes may be lossy, which means that the transformation and quantization processes may introduce distortion into the decoded video data.

[0022] A video decoder decodes residual data and adds it to a prediction block to generate a reconstructed video block that matches the original video block more closely than the prediction block alone. Due to losses introduced by the transformation and quantization of the residual data, the first reconstructed block may contain distortions or artifacts. One common type of artifact or distortion is referred to as blockiness, where the boundaries of the blocks used to code the video data are visible.

[0023] To further improve the quality of the decoded video, the video decoder may perform one or more filtering operations on the restored video blocks. Examples of such filtering operations include deblocking filtering, sample adaptive offset (SAO) filtering, and adaptive loop filtering (ALF). Parameters for these filtering operations may be determined by the video encoder and may be explicitly signaled in the encoded video bitstream, or they may be implicitly determined by the video decoder without requiring parameters to be explicitly signaled in the encoded video bitstream.

[0024] In some coding scenarios, a video encoder may encode video data in the transform skip mode described above, where the transform process is not performed; that is, the transform process is skipped. Consequently, residual data is not transformed for blocks encoded in transform skip mode. Residual blocks of video data encoded using transform skip mode may also be referred to as untransformed residual blocks. This disclosure describes a technique for a coefficient coding scheme for transforming transform skip mode. The technique of this disclosure includes an entropy decoding process that transforms binary representations into quantized coefficients rather than a series of binary values. Generally, the corresponding entropy encoding process, which is the inverse process of entropy decoding, is also part of this disclosure. The technique of this disclosure may be applied to any existing video codec, such as High Efficiency Video Coding (HEVC), or standards currently under development, such as Universal Video Coding (VVC), and other future video coding standards.

[0025] The present disclosure proposes a technique, which comprises, for example: determining a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient; and decoding a value for a coefficient currently being decoded based on the determined context offset. Since coefficient values ​​between neighbor coefficients in residual blocks tend to be more correlated with transform skip blocks than with transform blocks, the technique of the present disclosure may yield improved entropy coding, which can improve overall coding efficiency and, for example, reduce the bit overhead required to represent encoded video data without degrading the quality of the decoded video coding.

[0026] FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system (100) capable of performing the techniques of the present disclosure. The techniques of the present disclosure generally relate to coding (encoding and / or decoding) video data. Generally, video data includes any data for processing video. Accordingly, video data may include raw, uncoded video, encoded video, decoded (e.g., restored) video, and video metadata, such as signaling data.

[0027] As illustrated in FIG. 1, the system (100) includes a source device (102) that provides encoded video data to be decoded and displayed by a destination device (116) in this example. In particular, the source device (102) provides the video data to the destination device (116) via a computer-readable medium (110). The source device (102) and the destination device (116) may be any of a wide range of devices, including a desktop computer, a notebook (i.e., laptop) computer, a tablet computer or other mobile device, a set-top box, a telephone handset, a smartphone, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, a broadcast receiver device, etc. In some cases, the source device (102) and the destination device (116) may be equipped for wireless communication and thus may be referred to as wireless communication devices.

[0028] In the example of FIG. 1, the source device (102) includes a video source (104), memory (106), a video encoder (200), and an output interface (108). The destination device (116) includes an input interface (122), a video decoder (300), memory (120), and a display device (118). According to the present disclosure, the video encoder (200) of the source device (102) and the video decoder (300) of the destination device (116) may be configured to apply the coefficient coding techniques described in this disclosure. Thus, the source device (102) represents an example of a video encoding device, while the destination device (116) represents an example of a video decoding device. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device (102) may receive video data from an external video source, such as an external camera. Likewise, the destination device (116) may interface with an external display device rather than including an integrated display device.

[0029] The system (100) illustrated in FIG. 1 is merely an example. In general, any digital video encoding and / or decoding device may perform the counting coding techniques described in this disclosure. The source device (102) and the destination device (116) are merely examples of such coding devices in which the source device (102) generates video data coded for transmission to the destination device (116). This disclosure refers to a "coding" device as a device that performs the coding (encoding and / or decoding) of data. Accordingly, the video encoder (200) and the video decoder (300) represent examples of coding devices, specifically a video encoder and a video decoder, respectively. In some examples, the devices (102, 116) may operate in a substantially symmetric manner such that each of the devices (102, 116) includes video encoding and decoding components. Accordingly, the system (100) may support unidirectional or bidirectional video transmission between video devices (102, 116) for, for example, video streaming, video playback, video broadcasting, or video calling.

[0030] Generally, a video source (104) represents a source of video data (i.e., raw, uncoded video data) and provides a sequential series of pictures of video data (also referred to as “frames”) to a video encoder (200) that encodes data for the pictures. The video source (104) of the source device (102) may include a video capture device such as a video camera, a video archive containing previously captured raw video, and / or a video feed interface for receiving video from a video content provider. As an additional alternative, the video source (104) may generate computer graphics-based data as source video, or as a combination of live video, archived video, and computer-generated video. In each case, the video encoder (200) encodes the captured, pre-captured, or computer-generated video data. The video encoder (200) may rearrange the pictures from the received order (sometimes referred to as “display order”) to the coding order for coding. The video encoder (200) may generate a bitstream containing encoded video data. The source device (102) may then output the encoded video data onto a computer-readable medium (110) through an output interface (108) for reception and / or extraction, for example, by an input interface (122) of a destination device (116).

[0031] The memory (106) of the source device (102) and the memory (120) of the destination device (116) represent general-purpose memories. In some examples, the memories (106, 120) may store raw video data, for example, raw video from a video source (104) and raw, decoded video data from a video decoder (300). Additionally or alternatively, the memories (106, 120) may store software instructions executable by, for example, the video encoder (200) and the video decoder (300), respectively. Although the memory (106) and memory (120) are depicted separately from the video encoder (200) and the video decoder (300) in this example, it should be understood that the video encoder (200) and the video decoder (300) may also include internal memories for functionally similar or equivalent purposes. Additionally, the memories (106, 120) may store encoded video data that is output from the video encoder (200) and input to the video decoder (300), for example. In some examples, parts of the memories (106, 120) may be allocated as one or more video buffers to store raw, decoded, and / or encoded video data, for example.

[0032] The computer-readable medium (110) may represent any type of medium or device capable of transmitting encoded video data from a source device (102) to a destination device (116). In one example, the computer-readable medium (110) represents a communication medium that enables the source device (102) to transmit encoded video data directly to the destination device (116) in real time, for example, through a radio frequency network or a computer-based network. The output interface (108) may modulate a transmission signal containing the encoded video data according to a radio communication protocol, and the input interface (122) may demodulate a received transmission signal according to a communication standard, such as a radio communication protocol. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for enabling communication from a source device (102) to a destination device (116).

[0033] In some examples, the computer-readable medium (110) may include a storage device (112). The source device (102) may output encoded data to the storage device (112) from the output interface (108). Similarly, the destination device (116) may access the encoded data from the storage device (112) through the input interface (122). The storage device (112) may include any of various distributed or locally accessed data storage media, such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0034] In some examples, the computer-readable medium (110) may include a file server (114) or other intermediate storage device that may store encoded video data generated by the source device (102). The source device (102) may output the encoded video data to the file server (114) or other intermediate storage device that may store the encoded video data generated by the source device (102). The destination device (116) may access the stored video data from the file server (114) via streaming or downloading. The file server (114) may be any type of server device capable of storing the encoded video data and transmitting the encoded video data to the destination device (116). The file server (114) may represent a web server (e.g., for a website), an FTP (File Transfer Protocol) server, a content delivery network device, or a NAS (network attached storage) device. The destination device (116) may also access encoded video data from the file server (114) via any standard data connection, including an internet connection. This may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on the file server (114). The file server (114) and the input interface (122) may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0035] The output interface (108) and input interface (122) may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where the output interface (108) and input interface (122) include wireless components, the output interface (108) and input interface (122) may be configured to transmit data, such as encoded video data, according to cellular communication standards such as 4G, 4G-LTE (Long-Term Evolution), LTE Advanced, 5G, etc. In some examples where the output interface (108) includes a wireless transmitter, the output interface (108) and the input interface (122) may be configured to transmit data, such as encoded video data, according to other wireless standards such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee™), the Bluetooth™ standard, etc. In some examples, the source device (102) and / or the destination device (116) may include individual system-on-chip (SoC) devices. For example, the source device (102) may include an SoC device for performing the functionality attributed to the video encoder (200) and / or the output interface (108), and the destination device (116) may include an SoC device for performing the functionality attributed to the video decoder (300) and / or the input interface (122).

[0036] The techniques of the present disclosure may be applied to video coding by supporting any of various multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, internet streaming video transmissions, e.g., DASH (dynamic adaptive streaming over HTTP), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0037] The input interface (122) of the destination device (116) receives an encoded video bitstream from a computer-readable medium (110) (e.g., a communication medium, a storage device (112), a file server (114), etc.). The encoded video bitstream may include signaling information defined by a video encoder (200), which is also used by a video decoder (300), such as syntax elements having values ​​describing the processing and / or features of video blocks or other coded units (e.g., slices, pictures, groups of pictures, sequences, etc.). A display device (118) displays decoded pictures of the decoded video data to a user. The display device (118) may represent any of the various display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0038] Although not illustrated in FIG. 1, in some examples, the video encoder (200) and the video decoder (300) may each be integrated with an audio encoder and / or an audio decoder, and may include suitable MUX-DEMUX units or other hardware and / or software to handle multiplexed streams containing both audio and video in a common data stream. Where applicable, the MUX-DEMUX units may follow the ITU H.223 multiplexer protocol, or other protocols, e.g., the User Datagram Protocol (UDP).

[0039] Each of the video encoder (200) and the video decoder (300) may be implemented as any of various suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. Where the techniques are partially implemented in software, the device may store instructions for the software on a suitable, non-transient computer-readable medium and execute those instructions in hardware using one or more processors to perform the techniques of the present disclosure. Each of the video encoder (200) and the video decoder (300) may be included in one or more encoders or decoders, any one of which may be integrated as part of a combined encoder / decoder (CODEC) in a separate device. A device including a video encoder (200) and / or a video decoder (300) may include an integrated circuit, a microprocessor, and / or a wireless communication device, such as a cellular phone.

[0040] The video encoder (200) and video decoder (300) may operate according to a video coding standard such as ITU-T H.265, also referred to as High Efficiency Video Coding (HEVC), or extensions thereof, such as multi-view and / or scalable video coding extensions. Alternatively, the video encoder (200) and video decoder (300) may operate according to other proprietary or industry standards such as ITU-T H.266, also referred to as Joint Exploration Test Model (JEM) or Versatile Video Coding (VVC). The draft of the VVC standard is described in "Versatile Video Coding (Draft 4)" by Bross et al., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 13th Meeting: Marrakech, MA, 9-18 January 2019, JVET-M1001-v5 (hereinafter "VVC Draft 4"). Another draft of the VVC standard is described in Bross et al., “Versatile Video Coding (Draft 7),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 16th Meeting: Geneva, CH, 1-11 October 2019, JVET-P2001-v14 (hereinafter “VVC Draft 7”). However, the techniques of the present disclosure are not limited to any specific coding standard.

[0041] Generally, the video encoder (200) and video decoder (300) may perform block-based coding of pictures. The term “block” generally refers to a structure containing data to be processed (e.g., to be encoded, to be decoded, or otherwise to be used in the encoding and / or decoding process). For example, a block may contain a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, the video encoder (200) and video decoder (300) may code video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than coding red, green, and blue (RGB) data for samples of a picture, the video encoder (200) and video decoder (300) may code luminance and chrominance components, wherein the chrominance components may include both red and blue chrominance components. In some examples, the video encoder (200) converts the received RGB-formatted data into a YUV representation prior to encoding, and the video decoder (300) converts the YUV representation into an RGB format. Alternatively, pre- and post-processing units (not shown) may perform these conversions.

[0042] The present disclosure may generally refer to the coding of pictures (e.g., encoding and decoding), which includes a process of encoding or decoding data of the pictures. Similarly, the present disclosure may refer to the coding of blocks of pictures, which includes a process of encoding or decoding data for the blocks, such as, for example, prediction and / or residual coding. An encoded video bitstream generally includes a series of values ​​for syntax elements representing coding decisions (e.g., coding modes) and partitioning of the pictures into blocks. Accordingly, references to coding a picture or block should generally be understood as coding values ​​for the syntax elements forming the picture or block.

[0043] HEVC defines various blocks including coding units (CUs), prediction units (PUs), and transformation units (TUs). According to HEVC, a video coder (e.g., a video encoder (200)) partitions coding tree units (CTUs) into CUs according to a quadtree structure. That is, the video coder partitions the CTUs and CUs into four identical, non-overlapping squares, and each node of the quadtree has zero or four child nodes. Nodes without child nodes may be referred to as "leaf nodes," and the CUs of such leaf nodes may contain one or more PUs and / or one or more TUs. The video coder may further partition the PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of the TUs. In HEVC, PUs represent inter-prediction data, while TUs represent residual data. Intra-predicted CUs include intra-predicted information such as intra-mode indications.

[0044] As another example, the video encoder (200) and video decoder (300) may be configured to operate according to JEM or VVC. According to JEM or VVC, the video encoder (e.g., video encoder (200)) partitions the picture into multiple coding tree units (CTUs). The video encoder (200) may partition the CTUs according to a tree structure such as a quadtree binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concepts of multiple partition types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level partitioned according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of binary trees correspond to coding units (CUs).

[0045] In the MTT partitioning structure, blocks may be partitioned using quadtree (QT) partitions, binary tree (BT) partitions, and one or more types of tripletree (TT) (also called ternary tree (TT)) partitions. A triple or ternary tree partition is a partition in which a block is split into three sub-blocks. In some examples, a triple or ternary tree partition partitions the block into three sub-blocks without partitioning the original block through the center. The partitioning types in MTT (e.g., QT, BT, and TT) may be symmetric or asymmetric.

[0046] In some examples, the video encoder (200) and the video decoder (300) may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, while in other examples, the video encoder (200) and the video decoder (300) may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luminance component and another QTBT / MTT structure for both chrominance components (or two QTBT / MTT structures for each chrominance component).

[0047] The video encoder (200) and video decoder (300) may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures per HEVC. For the purposes of explanation, the description of the techniques of the present disclosure is presented with respect to QTBT partitioning. However, it should be understood that the techniques of the present disclosure may also be applied to video coders configured to use quadtree partitioning or other types of partitioning.

[0048] Blocks (e.g., CTUs or CUs) may be grouped in various ways within a picture. As an example, a brick may refer to a rectangular area of ​​rows of CTUs within a specific tile in the picture. A tile may be a rectangular area of ​​CTBs within a specific tile row or a specific tile column in the picture. A tile column refers to a rectangular area of ​​CTUs having a height equal to the height of the picture and a width specified by syntax elements (e.g., in a picture parameter set). A tile row refers to a rectangular area of ​​CTUs having a width equal to the width of the picture and a height specified by syntax elements (e.g., in a picture parameter set).

[0049] In some examples, a tile may be partitioned into multiple bricks, each of which may contain one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick. However, a brick, which is a true subset of a tile, may not be referred to as a tile.

[0050] Bricks in a picture may also be arranged into slices. A slice may be an integer number of bricks in a picture that may be exclusively contained in a single NAL (network abstraction layer) unit. In some examples, a slice contains only a number of complete tiles or a continuous sequence of complete bricks of a single tile.

[0051] The present disclosure may use "NxN" and "N by N," e.g., 16x16 samples or 16 by 16 samples, interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) with respect to vertical and horizontal dimensions. Generally, a 16x16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an NxN CU generally has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU may be arranged in rows and columns. Additionally, CUs do not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, CUs may contain NxM samples, where M is not necessarily the same as N.

[0052] A video encoder (200) encodes video data for CUs and other information representing prediction and / or residual information. The prediction information indicates how the CU will be predicted to form a prediction block for the CU. The residual information generally represents the sample-by-sample differences between the samples of the CU prior to encoding and the prediction block.

[0053] To predict the CU, the video encoder (200) may generally form a prediction block for the CU through inter-prediction or intra-prediction. Inter-prediction generally refers to predicting the CU from data of a previously coded picture, whereas intra-prediction generally refers to predicting the CU from data of the same picture that was previously coded. To perform inter-prediction, the video encoder (200) may generate a prediction block using one or more motion vectors. The video encoder (200) may generally perform motion search to identify a reference block that closely matches the CU in terms of the differences between the CU and the reference block. The video encoder (200) may calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared differences (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder (200) may predict the current CU using unidirectional prediction or bidirectional prediction.

[0054] Some examples of JEM and VVC also provide an affine motion compensation mode that may be considered as an inter-prediction mode. In an affine motion compensation mode, the video encoder (200) may determine two or more motion vectors representing non-translational motion, such as zoom in or out, rotation, perspective motion, or other irregular motion types.

[0055] To perform intra prediction, the video encoder (200) may select an intra prediction mode to generate prediction blocks. Some examples of JEM and VVC provide 67 intra prediction modes, including planar mode and DC mode, as well as various directional modes. Generally, the video encoder (200) selects an intra prediction mode that describes neighboring samples for the current block (e.g., a block of CU) to predict samples of the current block. Such samples may generally be located above, above and to the left of, or to the left of the current block in the same picture as the current block, assuming that the video encoder (200) codes the CTUs and CUs in raster scan order (from left to right, from top to bottom).

[0056] The video encoder (200) encodes data indicating the prediction mode for the current block. For example, for inter-prediction modes, the video encoder (200) may encode motion information for the corresponding mode as well as data indicating which of the various available inter-prediction modes is used. For unidirectional or bidirectional inter-prediction, for example, the video encoder (200) may encode motion vectors using an Advanced Motion Vector Prediction (AMVP) or merging mode. The video encoder (200) may also encode motion vectors for an affine motion compensation mode using similar modes.

[0057] Following a prediction such as an intra-prediction or inter-prediction of a block, the video encoder (200) may calculate residual data for the block. Residual data, such as a residual block, represents the sample-by-sample differences between the block and the prediction block for the block, formed using the corresponding prediction mode. The video encoder (200) may apply one or more transformations to the residual block to generate transformed data in a transformation domain instead of a sample domain. For example, the video encoder (200) may apply a Discrete Cosine Transform (DCT), an Integer Transform, a Wavelet Transform, or a conceptually similar transformation to the residual video data. Additionally, the video encoder (200) may apply a secondary transformation following the first transformation, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transformation, or a Karhunen-Loeve Transform (KLT). The video encoder (200) generates transformation coefficients following the application of one or more transformations.

[0058] Although examples in which transformations are performed are described above, in some examples, the transformation may be skipped. For example, the video encoder (200) may implement a transformation skip mode in which the transformation operation is skipped. In examples where the transformation is skipped, the video encoder (200) may output coefficients corresponding to residual values ​​instead of transformation coefficients. In the following description, the term "coefficient" should be interpreted to include transformation coefficients or coefficients corresponding to residual values ​​that occur as a result of the transformation.

[0059] As noted above, the video encoder (200) may perform quantization of the coefficients following any transformation or when a transformation is skipped to generate coefficients. In some examples, quantization may also be skipped when a transformation is skipped. Quantization generally refers to a process in which coefficients are quantized to provide additional compression, so as to potentially reduce the amount of data used to represent those coefficients. By performing the quantization process, the video encoder (200) may reduce the bit depth associated with some or all of the coefficients. For example, the video encoder (200) may round down n-bit values ​​to m-bit values ​​during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder (200) may perform a bitwise right-shift of the values ​​to be quantized.

[0060] Following quantization, the video encoder (200) may scan coefficients (e.g., resulting from the result of a transformation or due to a transformation skip) to generate a one-dimensional matrix from a two-dimensional matrix containing the quantized coefficients. The scan may be designed to place higher energy (and therefore lower frequency) coefficients at the front of the vector and lower energy (and therefore higher frequency) transformation coefficients at the rear of the vector. In examples where the transformation is skipped, the result of the scan may not be that the higher energy coefficients are at the front of the vector and the lower energy coefficients are at the rear of the vector. In some examples, the video encoder (200) may entropy-encode the quantized coefficients of the vector to generate a serialized vector by utilizing a predefined scan order. In other examples, the video encoder (200) may perform an adaptive scan. After scanning the quantized coefficients to form a one-dimensional vector, the video encoder (200) may entropy-encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder (200) may also entropy-encode the values ​​for syntax elements describing metadata associated with the encoded video data for use by the video decoder (300) when decoding the video data.

[0061] As introduced above, the video encoder (200) encodes residual data in TUs. Depending on the expected characteristics of the residual data in the TUs, the video encoder (200) may encode the TUs in different modes, such as a transform mode or a transform skip mode, and the different modes utilize different coefficient coding schemes. Some coefficient coding schemes encode the TUs using coefficient groups. A coefficient group is generally a subset of coefficients in the TUs. For example, the video encoder (200) may encode a 16x16 TU as four 4x4 coefficient groups.

[0062] To perform CABAC, the video encoder (200) may assign a context within a context model to the symbol to be transmitted. The context may, for example, be related to whether the neighbor values ​​of the symbol are zero values. The probability determination may be based on the context assigned to the symbol.

[0063] The video encoder (200) may additionally generate syntax data, e.g., block-based syntax data, picture-based syntax data, and sequence-based syntax data, to the video decoder (300), e.g., picture header, block header, slice header, or other syntax data, e.g., sequence parameter set (SPS), picture parameter set (PPS), or video parameter set (VPS). The video decoder (300) may likewise decode such syntax data to determine how to decode the corresponding video data.

[0064] In this way, the video encoder (200) may generate a bitstream containing syntax elements describing the partitioning of the encoded video data, for example, into blocks of a picture (e.g., CUs), and prediction and / or residual information for the blocks. Ultimately, the video decoder (300) may receive the bitstream and decode the encoded video data.

[0065] Generally, the video decoder (300) performs a process opposite to that performed by the video encoder (200) to decode the encoded video data of the bitstream. For example, the video decoder (300) may decode values ​​for the syntax elements of the bitstream using CABAC in a manner substantially similar to, but opposite to, the CABAC encoding process of the video encoder (200). The syntax elements may define the CUs of the CTU by defining the partitioning of each CTU according to the partitioning information of the picture into CTUs and the corresponding partition structure, such as the QTBT structure. The syntax elements may additionally define prediction and residual information for blocks of video data (e.g., CUs).

[0066] Residual information may be represented, for example, by quantized coefficients representing residual values ​​or transform coefficients. The video decoder (300) may inversely quantize and inversely transform the quantized transform coefficients of the block to reconstruct a residual block for the block. In examples where the video encoder (200) skipped a transform operation (e.g., transform skip mode), the video decoder (300) may skip an inverse transform operation. The video decoder (300) forms a prediction block (i.e., a predictive block) for the block using a signaled prediction mode (intra- or inter-prediction) and associated prediction information (e.g., motion information for inter-prediction). The video decoder (300) may then combine the prediction block and the residual block (on a sample-by-sample basis) to reconstruct the original block. The video decoder (300) may also perform additional processing, such as performing a deblocking process to reduce visual artifacts along the boundaries of the blocks.

[0067] According to the techniques of the present disclosure, a video encoder (200) and a video decoder (300) may be configured to determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, and to decode a value for a coefficient currently being decoded based on the determined context offset. Since the coefficient values ​​between neighbor coefficients in residual blocks tend to be more correlated with the transform skip blocks than with the transformed blocks, the techniques of the present disclosure may result in improved entropy coding, which can improve overall coding efficiency and, for example, reduce the bit overhead required to represent encoded video data without degrading the quality of the decoded video coding.

[0068] The present disclosure may generally refer to "signaling" specific information, such as syntax elements. The term "signaling" may generally refer to the communication of values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder (200) may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As noted above, the source device (102) may transmit the bitstream to the destination device (116) non-real-time or substantially real-time, as may occur when storing syntax elements in the storage device (112) for subsequent retrieval by the destination device (116).

[0069] FIGS. 2a and 2b are conceptual diagrams illustrating an exemplary quadtree binary tree (QTBT) structure (130) and a corresponding coding tree unit (CTU) (132). Solid lines represent quadtree splits, and dotted lines represent binary tree splits. At each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where 0 indicates a horizontal split and 1 indicates a vertical split. For quadtree splits, it is not necessary to indicate the partitioning type, as quadtree nodes divide blocks horizontally and vertically into four sub-blocks of the same size. Accordingly, the video encoder (200) may encode the syntax elements (e.g., split information) for the region tree level (e.g., the first level) of the QTBT structure (130) (e.g., the solid lines) and the syntax elements (e.g., split information) for the prediction tree level (e.g., the second level) of the QTBT structure (130) (e.g., the dotted lines), and the video decoder (300) may decode them. For the CUs represented by the terminal leaf nodes of the QTBT structure (130), the video encoder (200) may encode video data such as prediction and transformation data, and the video decoder (300) may decode them.

[0070] Generally, the CTU (132) of FIG. 2b may be associated with parameters that define the sizes of blocks corresponding to the nodes of the QTBT structure (130) at the first and second levels. These parameters may include CTU size (indicating the size of the CTU (132) in the samples), minimum quadtree size (MinQTSize, indicating the minimum allowed quadtree leaf node size), maximum binary tree size (MaxBTSize, indicating the maximum allowed binary tree root node size), maximum binary tree depth (MaxBTDepth, indicating the maximum allowed binary tree depth), and minimum binary tree size (MinBTSize, indicating the minimum allowed binary tree leaf node size).

[0071] The root node of the QTBT structure corresponding to the CTU may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to quadtree partitioning. That is, the nodes at the first level are either leaf nodes (no child nodes) or have four child nodes. An example of the QTBT structure (130) illustrates such nodes by including child nodes and parent nodes with solid lines for branches. If the nodes at the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), the nodes may be further partitioned by their respective binary trees. Binary tree splitting of a node may be repeated until the nodes resulting from the split reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). An example of the QTBT structure (130) illustrates such nodes by having dotted lines for branches. Binary tree leaf nodes are referred to as coding units (CUs) used for prediction (e.g., intra-picture or inter-picture prediction) and transformation without any additional partitioning. As discussed above, CUs may also be referred to as "video blocks" or "blocks".

[0072] In one example of a QTBT partitioning structure, the CTU size is set to 128x128 (luminance samples and two corresponding 64x64 chroma samples), MinQTSize is set to 16x16, MaxBTSize is set to 64x64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. Quadtree partitioning is first applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). If a quadtree leaf node is 128x128, the quadtree leaf node will not be further split by the binary tree because the size exceeds MaxBTSize (i.e., 64x64 in this example). Otherwise, quadtree leaf nodes will be further partitioned by the binary tree. Therefore, quadtree leaf nodes are also root nodes for the binary tree and have a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (4 in this example), further splitting is not allowed. A binary tree node with a width equal to MinBTSize (4 in this example) implies that further horizontal splitting is not allowed. Similarly, a binary tree node with a height equal to MinBTSize implies that further vertical splitting is not allowed for that binary tree node. As mentioned above, the leaf nodes of the binary tree are referred to as CUs and are further processed according to prediction and transformation without further partitioning.

[0073] When a block of video data is coded in a transform skip mode, the video encoder (200) skips the transform process for the residual signals before performing the quantization process. The video decoder (300) likewise skips the inverse transform process step after performing the inverse quantization process. The characteristics of the untransformed residual signals are typically significantly different from the characteristics of the transformed signals. For example, the coefficients for the transform skip block tend to be more correlated with their neighboring coefficients when compared to the coefficients for the transformed block. As a result, the level values ​​and sign information for the neighboring coefficients in the transform skip block of the residual data tend to be more correlated when compared to the level values ​​and sign information for the transformed block of the residual data.

[0074] B. Bross, T. Nguyen, P. Keydel, H. Schwarz, D. Marpe, T. Wiegand, "Non-CE8: Unified Transform Type Signalling and Residual Coding for Transform Skip," JVET document JVET-M0464, Marrackech, MA, Jan 2019 describes a proposed process for performing residual coding on blocks coded in transform skip mode. For efficient coding of level and sign information in transform skip mode, the coefficient coding proposed in JVET-M0464 may be modified to utilize signal characteristics for more efficient coding.

[0075] FIG. 3 illustrates an example of three coefficients from transformation skip blocks of residual data. Transform skip blocks also include additional coefficients not illustrated in FIG. 3. In the example of FIG. 3, coefficient value X represents the value of coefficient (140) representing the coefficient currently being coded. Coefficient value (X0) represents the value of coefficient (142), which is the left neighbor coefficient of coefficient (140). Coefficient value (X1) represents the value of coefficient (144), which is the top neighbor coefficient of coefficient (140). In this disclosure, the top neighbor may also be referred to as the upper neighbor.

[0076] The video encoder (200) and video decoder (300) may be configured to perform code coding for coefficients in transform skip blocks. In the technique for transform skip residual coding described in JVET-M0464, coefficients are performed on the transformed blocks from the first (i.e., top-left) to the last (i.e., bottom-right) instead of from last to first. By implementing the techniques of JVET-M0464, the video encoder (200) and video decoder (300) may be configured to context-code code information using CABAC, using the channel type of the residual as the context. That is, the video encoder (200) and video decoder (300) may use one context for the luminal residual and another context for the chroma residual. This disclosure describes code coding techniques that may be used in addition to or instead of the techniques described in JVET-M0464. The following techniques utilize sign information of upper neighbor coefficients (e.g., coefficient (144) in FIG. 3) and left neighbor coefficients (e.g., coefficient (142) in FIG. 3) to derive a sign coding context offset for the coefficient currently being coded (e.g., coefficient (140) in FIG. 3).

[0077] Referring to FIG. 3, X0 is the left neighbor coefficient value and X1 is the top neighbor coefficient value. If both neighbor coefficients are zero or non-zero but have opposite signs, the video encoder (200) and the video decoder (300) may use a context offset of 0 (ctxOffset = 0). Otherwise, if both are non-negative, the video encoder (200) and the video decoder (300) may use a context offset of 1 (ctxOffset = 1). For all other cases, the video encoder (200) and the video decoder (300) may use a context offset of 2 (ctxOffset = 2). This can be described by the following pseudocode:

[0078]

[0079] In some cases, if both neighboring coefficients are zero or non-zero but have opposite signs, the video encoder (200) and the video decoder (300) may use context offset 0 (ctxOffset = 0). Otherwise (if both are positive, or both are negative, or one is zero and the other is non-zero), the video encoder (200) and the video decoder (300) may use context 1. When context 1 is used, a signaled sign value of 0 or 1 means that, depending on the transformation, the sign of the coefficient currently being coded is the same as the sign of one of the non-zero neighbors. This technique can also be extended to coding scenarios where only the value of a previously coded non-zero coefficient may be used for context derivation, where the sign value of 0 or 1 indicates that the sign of the previously coded non-zero coefficient is the same as the sign of the previously coded non-zero coefficient with a single context.

[0080] Separate context sets for the Luma and Chroma components can be used in conjunction with the context offset derivation described above.

[0081] The video encoder (200) and video decoder (300) may also be configured to perform level mapping. In the transform skip residual coding of JVET-M0464, the absolute coefficient levels (absCoeffLevel) are coded using the values ​​sig_coeff_flag, abs_level_gtX_flags, par_level_flag, and abs_remainder to form the final absolute transform coefficient value, where X may be 1,..,5 (or some other cutoff value (C)). Thus, the absCoeffLevel value may be configured by:

[0082]

[0083] Instead of directly expressing absCoeffLevel as in JVET-M0464, the video encoder (200) may be configured to map absCoeffLevel to a modified level. The video decoder (300) may be configured to perform inverse mapping.

[0084] Similar to the code coding context offset derivation technique described above, the video encoder (200) and video decoder (300) may use absCoeffLevel information of left neighbor and upper neighbor coefficients to encode and decode absolute coefficient level values. In this example, X0 is denoted as the absolute coefficient level of the left neighbor coefficient (e.g., coefficient (142) in FIG. 3) for the coefficient currently being coded (e.g., coefficient (140) in FIG. 3), and X1 is denoted as the absolute coefficient level of the upper neighbor coefficient (e.g., coefficient (144) in FIG. 3) for the coefficient currently being coded (e.g., coefficient (140) in FIG. 3). To represent the coefficients as absolute coefficient levels (absCoeff), a mapped absCoeffMod may be coded.

[0085] The operation of the video encoder (200) for deriving a value for absCoeffMod can be illustrated by the following pseudocode:

[0086]

[0087] In some examples, if the absolute value of the coefficient to be coded (absCoeff) is equal to the maximum neighbor predictor (pred), the video encoder (200) sets the modified level (absCoeffMod) to 1. Otherwise, if absCoeff is less than the predictor, the video encoder (200) increments the value to be coded by 1. Otherwise, the video encoder (200) does not modify absCoeff.

[0088] The video encoder (200) may, for example, determine a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, and in response to the fact that the level value of the coefficient currently being encoded is equal to the predicted level value, encode a syntax element having a value equal to 1. In other cases, the video encoder (200) may determine a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, and in response to the fact that the level value of the coefficient currently being encoded is less than the predicted level value, encode a syntax element having a value equal to the level value of the coefficient currently being encoded. In other cases, the video encoder (200) determines a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, and in response to the level value of the coefficient currently being encoded being greater than the predicted level value, it may encode a syntax element having a value equal to the level value of the coefficient currently being encoded minus 1.

[0089] The operation of the video decoder (300) for deriving the value of absCoeff can be illustrated by the following pseudocode:

[0090]

[0091] The video decoder (300) may, for example, determine a predicted level value for a coefficient currently being decoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient, receive a indicated syntax element, and if the syntax element has a value equal to 1, determine that the level value of the coefficient currently being decoded is equal to the predicted level value. In other cases, the video decoder (300) may determine a predicted level value for a coefficient currently being decoded based on the value for a first neighbor coefficient and the value for a second neighbor coefficient, receive a indicated syntax element, and in response to the value for the syntax element being greater than the predicted level value, determine that the level value of the coefficient currently being decoded is equal to the value for the syntax element plus 1. In other cases, the video decoder (300) may determine a predicted level value for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, receive a indicated syntax element, and in response to the value for the syntax element being less than the predicted level value, determine that the level value of the coefficient currently being decoded is equal to the value for the syntax element.

[0092] In some examples, the video encoder (200) and the video decoder (300) determine or derive the context of the syntax element identified herein as abs_level_gt1_flag based on whether the left neighbor and top neighbor coefficient values ​​are zero. The syntax element abs_level_gt1_flag is the syntax element used to code the coefficient values. A value of abs_level_gt1_flag such as 1 may mean, for example, that the absolute level of the coefficient is greater than 1. A value of abs_level_gt1_flag such as 0 may mean, for example, that the absolute level of the coefficient is not greater than 1.

[0093] In one example, the context may be one of multiple, for example, three different contexts. One context may be derived for cases where both the left neighbor coefficient and the upper neighbor coefficient have non-zero values. Another context may be derived for cases where only one of the left neighbor coefficient or the upper neighbor coefficient has a non-zero value. A third context may be derived for cases where both the left neighbor coefficient and the upper neighbor coefficient have zero values. In one example, this context derivation applies only to non-BDPCM (block differential pulse code modulation) modes.

[0094] In some examples, for coding scenarios where there are non-existent or unavailable neighbor values ​​(e.g., when the coefficient being coded is on the left boundary of the block or when there are no left neighbors), the video encoder (200) and video decoder (300) may be configured to use zero values ​​when deriving context.

[0095] In some examples, context derivation may be described as follows:

[0096]

[0097] In these examples, for non-existent / unavailable neighbor values ​​(e.g., left neighbor values ​​on the left boundary of a block), the video encoder (200) and video decoder (300) may be configured to use a zero value for the unavailable values ​​when deriving context.

[0098] FIG. 4 is a block diagram illustrating an exemplary video encoder (200) capable of performing the techniques of the present disclosure. FIG. 4 is provided for illustrative purposes and should not be considered as a limitation to the techniques generally illustrated and described in the present disclosure. For illustrative purposes, the present disclosure describes a video encoder (200) in the context of video coding standards such as the HEVC (H.265) video coding standard and the VVC (H.266) video coding standard under development. However, the techniques of the present disclosure are not limited to these video coding standards and are generally applicable to video encoding and decoding.

[0099] In the example of FIG. 4, the video encoder (200) includes a video data memory (230), a mode selection unit (202), a residual generation unit (204), a conversion processing unit (206), a quantization unit (208), an inverse quantization unit (210), an inverse conversion processing unit (212), a restoration unit (214), a filter unit (216), a decoded picture buffer (DPB) (218), and an entropy encoding unit (220). Any or all of the video data memory (230), the mode selection unit (202), the residual generation unit (204), the conversion processing unit (206), the quantization unit (208), the inverse quantization unit (210), the inverse conversion processing unit (212), the restoration unit (214), the filter unit (216), the DPB (218), and the entropy encoding unit (220) may be implemented in one or more processors or in a processing circuit. Furthermore, the video encoder (200) may include additional or alternative processors or processing circuits to perform these and other functions.

[0100] The video data memory (230) may store video data to be encoded by components of the video encoder (200). The video encoder (200) may receive video data stored in the video data memory (230) from, for example, a video source (104) (Fig. 1). The DPB (218) may act as a reference picture memory that stores reference video data for use in predicting subsequent video data by the video encoder (200). The video data memory (230) and the DPB (218) may be formed by various memory devices, for example, synchronous dynamic random access memory (SDRAM), magnetoresistive RAM (MRAM), DRAM including resistive RAM (RRAM), or any of other types of memory devices. The video data memory (230) and the DPB (218) may be provided by the same memory device or by separate memory devices. In various examples, the video data memory (230) may be on-chip with the other components of the video encoder (200) or off-chip with respect to those components, as illustrated.

[0101] In the present disclosure, a reference to the video data memory (230) should not be interpreted as being limited to memory inside the video encoder (200) unless specifically stated otherwise, or to memory outside the video encoder (200) unless specifically stated otherwise. Rather, a reference to the video data memory (230) should be understood as a reference memory that stores video data received by the video encoder (200) for encoding (e.g., video data for the current block to be encoded). The memory (106) of FIG. 1 may also provide temporary storage of outputs from various units of the video encoder (200).

[0102] The various units illustrated in FIG. 4 are illustrated to aid in understanding the operations performed by the video encoder (200). These units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functionality and are pre-configured for the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, programmable circuits may execute software or firmware that causes the programmable circuits to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations performed by the fixed-function circuits are generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0103] The video encoder (200) may include arithmetic logic units (ALUs), basic function units (EFUs), digital circuits, analog circuits, and / or programmable cores formed from programmable circuits. In examples where the operations of the video encoder (200) are performed using software executed by the programmable circuits, memory (106) (FI. 1) may store object code of software that the video encoder (200) receives and executes, or other memory within the video encoder (200) (not shown) may store such instructions.

[0104] The video data memory (230) is configured to store received video data. The video encoder (200) may extract a picture of video data from the video data memory (230) and provide the video data to the residual generation unit (204) and the mode selection unit (202). The video data in the video data memory (230) may be raw video data to be encoded.

[0105] The mode selection unit (202) includes a motion estimation unit (222), a motion compensation unit (224), and an intra-prediction processing unit (226). The mode selection unit (202) may include additional functional units to perform video prediction according to different prediction modes. As examples, the mode selection unit (202) may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit (222) and / or the motion compensation unit (224)), an affine unit, a linear model (LM) unit, etc.

[0106] The mode selection unit (202) generally adjusts multiple encoding passes to test combinations of encoding parameters and rate-distortion values ​​of the results for such combinations. The encoding parameters may include partitioning of CTUs into CUs, prediction modes for CUs, types of transformation for residual data of CUs, quantization parameters for residual data of CUs, etc. The mode selection unit (202) may ultimately select a combination of encoding parameters that has superior rate-distortion values ​​compared to other tested combinations.

[0107] The video encoder (200) partitions a picture extracted from the video data memory (230) into a series of CTUs and may encapsulate one or more CTUs within a slice. The mode selection unit (202) may partition the CTUs of the picture according to a tree structure, such as the quad-tree structure or QTBT structure of the HEVC described above. As described above, the video encoder (200) may form one or more CUs by partitioning the CTUs according to a tree structure. Such CUs may also generally be referred to as "video blocks" or "blocks".

[0108] Generally, the mode selection unit (202) also controls its components (e.g., motion estimation unit (222), motion compensation unit (224), and intra-prediction unit (226)) to generate a prediction block for the current block (e.g., the current CU, or the overlapping portion of the PU and TU in HEVC). For inter-prediction of the current block, the motion estimation unit (222) may perform motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in the DPB (218). In particular, the motion estimation unit (222) may calculate a value indicating how similar a potential reference block is to a current block, for example, based on the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared differences (MSD), etc. The motion estimation unit (222) may perform these calculations using the differences per sample between the reference block and the current block that are generally considered. The motion estimation unit (222) may also identify a reference block having the lowest value resulting from these calculations, indicating the reference block that matches most closely with the current block.

[0109] The motion estimation unit (222) may form one or more motion vectors (MVs) that define the positions of reference blocks in reference pictures for the position of the current block in the current picture. The motion estimation unit (222) may then provide the motion vectors to the motion compensation unit (224). For example, for unidirectional inter-prediction, the motion estimation unit (222) may provide a single motion vector, whereas for bidirectional inter-prediction, the motion estimation unit (222) may provide two motion vectors. Then, the motion compensation unit (224) may generate a prediction block using the motion vectors. For example, the motion compensation unit (224) may extract data for the reference block using the motion vectors. As another example, if the motion vector has fractional sample precision, the motion compensation unit (224) may interpolate the values ​​for the prediction block according to one or more interpolation filters. Additionally, for bidirectional inter-prediction, the motion compensation unit (224) may extract data for two reference blocks identified by individual motion vectors and combine the extracted data, for example, through averaging or weighted averaging per sample.

[0110] As another example, for intra-prediction or intra-prediction coding, the intra-prediction unit (226) may generate prediction blocks from samples adjacent to the current block. For example, for directional modes, the intra-prediction unit (226) may generally generate prediction blocks by mathematically combining the values ​​of neighboring samples and populating these calculated values ​​in a direction defined across the current block. As yet another example, for DC mode, the intra-prediction unit (226) may calculate the average of neighboring samples for the current block and generate prediction blocks, including the average of these results for each sample of the prediction block.

[0111] The mode selection unit (202) provides the prediction block to the residual generation unit (204). The residual generation unit (204) receives the raw, uncoded version of the current block from the video data memory (230) and the prediction block from the mode selection unit (202). The residual generation unit (204) calculates the sample-by-sample differences between the current block and the prediction block. The sample-by-sample differences of the result define the residual block for the current block. In some examples, the residual generation unit (204) may also determine the differences between sample values ​​in the residual block to generate the residual block using residual differential pulse code modulation (RDPCM). In some examples, the residual generation unit (204) may be formed using one or more subtraction circuits that perform binary subtraction.

[0112] In examples where the mode selection unit (202) partitions the CUs into PUs, each PU may be associated with a luminance prediction unit and a corresponding chroma prediction unit. The video encoder (200) and the video decoder (300) may support PUs of various sizes. As indicated above, the size of the CU may refer to the size of the luminance coding block of the CU, and the size of the PU may refer to the size of the luminance prediction unit of the PU. Assuming that the size of a particular CU is 2Nx2N, the video encoder (200) may support PU sizes of 2Nx2N or NxN for intra-prediction, and may support symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, etc. for inter-prediction. The video encoder (200) and video decoder (300) may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-prediction.

[0113] In examples where the mode selection unit does not further partition the CU into PUs, each CU may be associated with a luminance coding block and a corresponding chroma coding block. As above, the size of the CU may refer to the size of the luminance coding block of the CU. The video encoder (200) and the video decoder (300) may support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0114] For other video coding techniques such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, as some examples, the mode selection unit (202) generates a prediction block for the current block being encoded through individual units associated with the coding techniques. In some examples, such as palette mode coding, the mode selection unit (202) may not generate a prediction block, but instead generate syntax elements indicating a method for restoring the block based on the selected palette. In such modes, the mode selection unit (202) may provide these syntax elements to the entropy encoding unit (220) to be encoded.

[0115] As described above, the residual generation unit (204) receives video data for the current block and the corresponding prediction block. The residual generation unit (204) then generates a residual block for the current block. To generate the residual block, the residual generation unit (204) calculates the differences for each sample between the current block and the prediction block.

[0116] The transformation processing unit (206) applies one or more transformations to a residual block to generate a block of transformation coefficients (referred to herein as a “transformation coefficient block”). The transformation processing unit (206) may form the transformation coefficient block by applying various transformations to the residual block. For example, the transformation processing unit (206) may apply a Discrete Cosine Transform (DCT), a Directional Transform, a Karhunen-Loeve Transform (KLT), or a conceptually similar transformation to the residual block. In some examples, the transformation processing unit (206) may perform multiple transformations on the residual block, such as a first-order transformation and a second-order transformation, such as a rotation transformation. In some examples, the transformation processing unit (206) does not apply transformations to the residual block. In these cases, the transformation processing unit (206) may output a block of coefficients, where the coefficients correspond to residual values ​​instead of transformation coefficients.

[0117] The quantization unit (208) may quantize the transformation coefficients in the transformation coefficient block to create a quantized transformation coefficient block. For a block coded in transformation skip mode, the quantization unit (208) may quantize the coefficients in the coefficient block to create a quantized coefficient block. The quantization unit (208) may quantize or transform the coefficients according to the quantization parameter (QP) value associated with the current block. The video encoder (200) may adjust the degree of applied quantization by adjusting the QP value associated with the CU (e.g., via the mode selection unit (202)). Quantization may introduce a loss of information, and accordingly, the quantized coefficients or transformation coefficients may have lower precision than the transformation coefficients or original coefficients generated by the transformation processing unit (206).

[0118] The inverse quantization unit (210) and the inverse transform processing unit (212) may each apply inverse quantization and inverse transforms to the coefficient block to restore the residual block from the coefficient block. The restoration unit (214) may generate a restored block corresponding to the current block (potentially having some degree of distortion) based on the prediction block generated by the mode selection unit (202) and the restored residual block. For example, the restoration unit (214) may generate the restored block by adding samples of the restored residual block to corresponding samples from the prediction block generated by the mode selection unit (202).

[0119] The filter unit (216) may perform one or more filter operations on the restored block. For example, the filter unit (216) may perform deblocking operations to reduce blockiness artifacts along the edges of the CUs. The operations of the filter unit (216) may be skipped in some examples.

[0120] The video encoder (200) stores the restored blocks in the DPB (218). For example, in examples where the operations of the filter unit (216) are not performed, the restoration unit (214) may store the restored blocks in the DPB (218). In examples where the operations of the filter unit (216) are performed, the filter unit (216) may store the filtered restored blocks in the DPB (218). The motion estimation unit (222) and the motion compensation unit (224) may extract a reference picture from the DPB (218) formed from the restored (and potentially filtered) blocks and inter-predict blocks of subsequently encoded pictures. Additionally, the intra-prediction unit (226) may use the restored blocks from the DPB (218) of the current picture to intra-predict other blocks in the current picture.

[0121] Generally, the entropy encoding unit (220) may entropy encode syntax elements received from other functional components of the video encoder (200). For example, the entropy encoding unit (220) may entropy encode quantized coefficient blocks from the quantization unit (208). As another example, the entropy encoding unit (220) may entropy encode prediction syntax elements (e.g., intra-mode information for intra-prediction or motion information for inter-prediction) from the mode selection unit (202). The entropy encoding unit (220) may perform one or more entropy encoding operations on syntax elements, which are other examples of video data, to generate entropy-encoded data. For example, the entropy encoding unit (220) may perform a context-adaptive variable-length coding (CAVLC) operation, a CABAC operation, a V2V (variable-to-variable) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential-Golomb encoding operation, or other types of entropy encoding operations on the data. In some examples, the entropy encoding unit (220) may operate in a bypass mode where syntax elements are not entropy encoded.

[0122] The video encoder (200) may output a bitstream containing entropy-encoded syntax elements necessary to restore blocks of a picture or slice. In particular, the entropy encoding unit (220) may output the bitstream.

[0123] The operations described above are described with respect to blocks. These descriptions should be understood as operations for luminance coding blocks and / or chroma coding blocks. As described above, in some examples, the luminance coding blocks and chroma coding blocks are the luminance and chroma components of the CU. In some examples, the luminance coding blocks and chroma coding blocks are the luminance and chroma components of the PU.

[0124] In some examples, operations performed on luminance coding blocks do not need to be repeated for chroma coding blocks. As one example, operations to identify the motion vector (MV) for the luminance coding block and the reference picture do not need to be repeated to identify the motion vector (MV) for the chroma blocks and the reference picture. Rather, the MV for the luminance coding block may be scaled to determine the MV for the chroma blocks, and the reference picture may be the same. As another example, the intra-prediction process may be the same for the luminance coding blocks and the chroma coding blocks.

[0125] A video encoder (200) represents an example of a device configured to encode video data, the device comprising a memory configured to store video data and one or more processing units implemented in a circuit, and the one or more processing units are configured to determine, for a residual block of video data encoded using a convert skip mode, a value for a first neighbor coefficient of a coefficient currently being encoded; determine a value for a second neighbor coefficient of a coefficient currently being encoded; determine a context offset for a coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and, based on the determined context offset, encode a value for a coefficient currently being encoded. The first neighbor coefficient may be, for example, one of an upper neighbor coefficient or a left neighbor coefficient, and the second neighbor coefficient may be the other of an upper neighbor coefficient or a left neighbor coefficient.

[0126] In order to encode a value for a coefficient currently being encoded based on a determined context offset, the video encoder (200) may be configured to determine a context based on the determined context offset, determine a sign for a coefficient currently being encoded, and context-encode one or more bins of data based on the determined context to represent a sign for a coefficient currently being encoded. The video encoder (200) may be configured to quantize a residual value for a residual block of video data to determine a value for a coefficient currently being encoded. The video encoder (200) may be configured to determine a prediction block and compare the original block of video data with the prediction block to determine a residual block of video data.

[0127] To determine a context offset for a coefficient currently being encoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video encoder (200) may be configured to select a context offset from three available context offsets based on a value for a first neighbor coefficient and a value for a second neighbor coefficient. The three available context offsets may include a first context offset for cases where the first neighbor coefficient and the second neighbor coefficient have a value equal to zero or have opposite signs; a second context offset for cases where both the first neighbor coefficient and the second neighbor coefficient are positive, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is positive; and a third context offset for cases where both the first neighbor coefficient and the second neighbor coefficient are negative, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative.

[0128] To determine a context offset for a coefficient currently being encoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a first offset value in response to the value for a first neighbor coefficient being equal to zero and the value for a second neighbor coefficient being equal to zero. To determine a context offset for a coefficient currently being encoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a first offset value in response to the value for a first neighbor coefficient being either greater than zero or less than zero, and the other being either greater than zero or less than zero.

[0129] To determine a context offset for a coefficient currently being encoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a first offset value in response to both the value for a first neighbor coefficient and the value for a second neighbor coefficient being non-zero values ​​and having opposite signs. To determine a context offset for a coefficient currently being encoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a second offset value in response to the value for a first neighbor coefficient being zero or greater and the value for a second neighbor coefficient being zero or greater, and at least one of the value for a first neighbor coefficient or the value for a second neighbor coefficient being 1 or greater.

[0130] To determine a context offset for a coefficient currently being encoded based on the values ​​for a first neighbor coefficient and a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a second offset value in response to both the value for a first neighbor coefficient and the value for a second neighbor coefficient being non-negative values. To determine a context offset for a coefficient currently being encoded based on the values ​​for a first neighbor coefficient and a second neighbor coefficient, the video encoder (200) may be configured to set a context offset value to a third offset value in response to the values ​​for a first neighbor coefficient being zero or less and a second neighbor coefficient being zero or less, and the value for a first neighbor coefficient or a second neighbor coefficient being negative 1 (-1) or less. The first, second, and third offset values ​​may be different offset values, for example. A context offset is a value for determining context. Therefore, the first, second, and third offset values ​​may be considered to identify or refer to three different contexts.

[0131] The video encoder (200) also represents an example of a device configured to encode video data, wherein the device comprises a memory configured to store video data and one or more processing units implemented in a circuit, and the one or more processing units are configured to determine, for a residual block of video data currently being encoded using a transform skip mode, an absolute coefficient level for a first neighbor coefficient of the coefficient currently being encoded; an absolute coefficient level for a second neighbor coefficient of the coefficient currently being encoded; an absolute coefficient level for the coefficient currently being encoded; and, based on the absolute coefficient level for the first neighbor coefficient and the absolute coefficient level for the second neighbor coefficient, to encode one or more syntax elements representing an absolute coefficient level for the coefficient currently being encoded. The video encoder (200) may also be configured to quantize residual values ​​for a residual block of video data to determine, for example, a value for the coefficient currently being encoded.

[0132] In some examples, the video encoder (200) is configured to determine a predictor level based on an absolute coefficient level for a first neighbor coefficient and an absolute coefficient level for a second neighbor coefficient, and to encode a syntax element, and a first value for the syntax element indicates that the predictor level is equal to the absolute coefficient level for the coefficient currently being encoded, and a second value for the syntax element indicates that the predictor level is not equal to the absolute coefficient level for the coefficient currently being encoded. In some examples, the video encoder (200) is configured to determine a predictor level based, for example, an absolute coefficient level for a first neighbor coefficient and an absolute coefficient level for a second neighbor coefficient, determine a value for a syntax element based on the absolute coefficient level for the coefficient currently being encoded and the predictor level, and to encode a syntax element. To determine the predictor level, the video encoder (200) may be configured to set a predictor level such as the greater of the absolute coefficient level for the first neighbor coefficient or the absolute coefficient level for the second neighbor coefficient.

[0133] FIG. 5 is a block diagram illustrating an exemplary video decoder (300) that may perform the techniques of this disclosure. FIG. 5 is provided for illustrative purposes and is not limited to techniques generally exemplified and described in this disclosure. For illustrative purposes, this disclosure describes a video decoder (300) according to the techniques of JEM, VVC, and HEVC. However, the techniques of this disclosure may be performed by video coding devices configured with other video coding standards.

[0134] In the example of FIG. 5, the video decoder (300) includes a coded picture buffer (CPB) memory (320), an entropy decoding unit (302), a prediction processing unit (304), an inverse quantization unit (306), an inverse transformation processing unit (308), a restoration unit (310), a filter unit (312), and a decoded picture buffer (DPB) (314). Any or all of the CPB memory (320), the entropy decoding unit (302), the prediction processing unit (304), the inverse quantization unit (306), the inverse transformation processing unit (308), the restoration unit (310), the filter unit (312), and the DPB (314) may be implemented in one or more processors or processing circuits. Furthermore, the video decoder (300) may include additional or alternative processors or processing circuits to perform these and other functions.

[0135] The prediction processing unit (304) includes a motion compensation unit (316) and an intra prediction processing unit (318). The prediction processing unit (304) may include additional units to perform predictions according to different prediction modes. For example, the prediction processing unit (304) may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit (316)), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder (300) may include more, fewer, or different functional components.

[0136] The CPB memory (320) may store video data, such as an encoded video bitstream to be decoded by components of the video decoder (300). The video data stored in the CPB memory (320) may be obtained, for example, from a computer-readable medium (110) (Fig. 1). The CPB memory (320) may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory (320) may store video data other than syntax elements of the encoded picture, such as transient data representing outputs from various units of the video decoder (300). The DPB (314) generally stores decoded pictures that the video decoder (300) may output and / or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. CPB memory (320) and DPB (314) may be formed by any of several memory devices, such as DRAM, SDRAM, MRAM, RRAM, or other types of memory devices. CPB memory (320) and DPB (314) may be provided by the same memory device or by separate memory devices. In several examples, CPB memory (320) may be on-chip with respect to other components of the video decoder (300) or off-chip with respect to those components.

[0137] Additionally or alternatively, in some examples, the video decoder (300) may retrieve coded video data from memory (120) (Fig. 1). That is, the memory (120) may store data as discussed above in CPB memory (320). Likewise, the memory (120) may store instructions to be executed by the video decoder (300) when some or all of the functionality of the video decoder (300) is implemented in software to be executed by the processing circuit of the video decoder (300).

[0138] The various units illustrated in FIG. 5 are illustrated to aid in understanding the operations performed by the video decoder (300). These units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Similar to FIG. 4, fixed-function circuits refer to circuits that provide specific functionality and are pre-configured for the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, programmable circuits may execute software or firmware that causes the programmable circuits to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations performed by fixed-function circuits are generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0139] The video decoder (300) may include ALUs, EFUs, digital circuits, analog circuits, and / or programmable cores formed from programmable circuits. In examples where the operations of the video decoder (300) are performed by software running on the programmable circuits, on-chip or off-chip memory may store instructions (e.g., object code) of the software that the video decoder (300) receives and executes.

[0140] The entropy decoding unit (302) may receive encoded video data from the CPB and entropy decode the video data to reproduce syntax elements. The prediction processing unit (304), the inverse quantization unit (306), the inverse transform processing unit (308), the restoration unit (310), and the filter unit (312) may generate decoded video data based on syntax elements extracted from the bitstream.

[0141] Generally, the video decoder (300) restores the picture on a block-by-block basis. The video decoder (300) may also perform the restoration operation for each block individually (where the block currently being restored, i.e., being decoded, may be referred to as the "current block").

[0142] The entropy decoding unit (302) may entropy decode not only conversion information such as quantization parameters (QP) and / or conversion mode indication(s), but also syntax elements defining the quantized coefficients of the quantized coefficient block. The inverse quantization unit (306) may use the QP associated with the quantized coefficient block to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied by the inverse quantization unit (306). The inverse quantization unit (306) may, for example, perform a bitwise left-shift operation to inverse quantize the quantized coefficients. The inverse quantization unit (306) may thereby form a coefficient block containing the coefficients.

[0143] After the inverse quantization unit (306) forms a coefficient block for the transformed block, the inverse transformation processing unit (308) may apply one or more inverse transformations to the transformation coefficient block to generate a residual block associated with the current block. For example, the inverse transformation processing unit (308) may apply an inverse DCT, an inverse integer transformation, an inverse Karhunen-Loeve transformation (KLT), an inverse rotation transformation, an inverse directional transformation, or other inverse transformations to the transformation coefficient block. For blocks coded in transform skip mode, the inverse transformation processing unit (308) may not perform an inverse transformation, and in these coding scenarios, it may be seen as a pass-through unit that does not process or modify the block of coefficients.

[0144] Furthermore, the prediction processing unit (304) generates a prediction block according to the prediction information syntax elements entropy-decoded by the entropy decoding unit (302). For example, if the prediction information syntax elements indicate that the current block is inter-predicted, the motion compensation unit (316) may generate a prediction block. In this case, the prediction information syntax elements may indicate a motion vector identifying the position of the reference block in the reference picture, as well as the position of the current block in the current picture, as well as the reference picture in the DPB (314) from which the reference block will be extracted. The motion compensation unit (316) may generally perform the inter-prediction process in a manner substantially similar to that described for the motion compensation unit (224) (Fig. 4).

[0145] As another example, if the prediction information syntax elements indicate that the current block is intra-predicted, the intra-predict unit (318) may generate a prediction block according to the intra-predict mode indicated by the prediction information syntax elements. Again, the intra-predict unit (318) may perform the intra-predict process in a manner substantially similar to that generally described for the intra-predict unit (226) (Fig. 4). The intra-predict unit (318) may extract data of neighbor samples for the current block from the DPB (314).

[0146] The restoration unit (310) may restore the current block using the prediction block and the residual block. For example, the restoration unit (310) may restore the current block by adding samples of the residual block to the corresponding samples of the prediction block.

[0147] The filter unit (312) may perform one or more filter operations on the restored blocks. For example, the filter unit (312) may perform deblocking operations to reduce blocking artifacts along the edges of the restored blocks. The operations of the filter unit (312) are not necessarily performed in all examples.

[0148] The video decoder (300) may store the restored blocks in the DPB (314). For example, in examples where the operations of the filter unit (312) are not performed, the restoration unit (310) may store the restored blocks in the DPB (314). In examples where the operations of the filter unit (312) are performed, the filter unit (312) may store the filtered restored blocks in the DPB (314). As discussed above, the DPB (314) may provide reference information to the prediction processing unit (304), such as samples of previously decoded pictures for subsequent motion compensation and samples of the current picture for intra-prediction. Additionally, the video decoder (300) may output the decoded pictures from the DPB for subsequent presentation onto a display device such as the display device (118) of FIG. 1.

[0149] The video decoder (300) also represents an example of a video decoding device comprising a memory configured to store video data and one or more processing units implemented in a circuit, wherein the one or more processing units are configured to determine, for a residual block of video data encoded using a transform skip mode, a value for a first neighbor coefficient of a coefficient currently being decoded; determine a value for a second neighbor coefficient of a coefficient currently being decoded; determine a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and decode a value for a coefficient currently being decoded based on the determined context offset. The first neighbor coefficient may be, for example, one of an upper neighbor coefficient or a left neighbor coefficient, and the second neighbor coefficient may be the other of an upper neighbor coefficient or a left neighbor coefficient. As described above, for residual blocks encoded in a transform skip mode, the coefficient value may correspond to a residual value, such as a quantized residual value or an unquantized residual value.

[0150] To decode a value for a coefficient currently being decoded based on a determined context offset, the video decoder (300) may be configured to determine a context based on the determined context offset; receive one or more bins of data; and context decode one or more bins of data based on the determined context to determine a sign for a coefficient currently being decoded. The video decoder (300) may be configured to inversely quantize a value for a coefficient currently being decoded to determine a residual value for a residual block of video data. The video decoder (300) may determine a decoded residual block based on a value for a coefficient currently being decoded; add the decoded residual block to a prediction block to determine a restored block; perform one or more filtering operations on the restored block to determine a decoded block of video data; and output a decoded picture of video data containing the decoded block of video data.

[0151] To determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video decoder (300) may be configured to select a context offset from three available context offsets based on a value for a first neighbor coefficient and a value for a second neighbor coefficient. The three available context offsets may include a first context offset when both the first neighbor coefficient and the second neighbor coefficient have a value equal to zero or opposite signs; a second context offset when both the first neighbor coefficient and the second neighbor coefficient are positive or when one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is positive; and a third context offset when both the first neighbor coefficient and the second neighbor coefficient are negative or when one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative.

[0152] To determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video decoder (300) may be configured to set a context offset value to a first offset value in response to the value for a first neighbor coefficient being equal to zero and the value for a second neighbor coefficient being equal to zero. To determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video decoder (300) may be configured to set a context offset value to a first offset value in response to the value for a first neighbor coefficient being either greater than zero or less than zero, and the other being either greater than zero or less than zero. To determine the context offset for the coefficient currently being decoded based on the values ​​for the first neighbor coefficient and the values ​​for the second neighbor coefficient, the video decoder (300) may be configured to set the context offset value to the first offset value in response to both the values ​​for the first neighbor coefficient and the values ​​for the second neighbor coefficient being non-zero values ​​and having opposite signs.

[0153] To determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video decoder (300) may be configured to set a context offset value to a second offset value in response to the value for a first neighbor coefficient being zero or greater and the value for a second neighbor coefficient being zero or greater, and at least one of the value for a first neighbor coefficient or the value for a second neighbor coefficient being 1 or greater. To determine a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, the video decoder (300) may be configured to set a context offset value to a second offset value in response to the value for a first neighbor coefficient and the value for a second neighbor coefficient both being non-negative values. To determine a context offset for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, the video decoder (300) may be configured to set the context offset value to a third offset value in response to the value for the first neighbor coefficient being zero or less and the value for the second neighbor coefficient being zero or less, and the value for the first neighbor coefficient or the value for the second neighbor coefficient being negative 1 (-1) or less.

[0154] The video decoder (300) also represents an example of a video decoding device comprising a memory configured to store video data and one or more processing units implemented in a circuit, wherein the one or more processing units may be configured to determine, for a residual block of video data decoded using a transform skip mode, an absolute coefficient level for a first neighbor coefficient of the coefficient currently being decoded; to determine an absolute coefficient level for a second neighbor coefficient of the coefficient currently being decoded; and to determine an absolute coefficient level for the coefficient currently being decoded based on the absolute coefficient level for the first neighbor coefficient and the absolute coefficient level for the second neighbor coefficient. The video decoder (300) may, for example, inversely quantize the coefficient currently being decoded to determine a residual value for a residual block of video data.

[0155] To determine the absolute coefficient level for the coefficient currently being decoded, the video decoder (300) may be configured to determine the predictor level based on the absolute coefficient level for the first neighbor coefficient and the absolute coefficient level for the second neighbor level, and to receive a syntax element, the first value for the syntax element indicates that the predictor level is equal to the absolute coefficient level for the coefficient currently being decoded, and the second value for the syntax element indicates that the predictor level is not equal to the absolute coefficient level for the coefficient currently being decoded. To determine the absolute coefficient level for the coefficient currently being decoded, the video decoder (300) may be configured to determine the predictor level based, for example, the absolute coefficient level for the first neighbor coefficient and the absolute coefficient level for the second neighbor coefficient, to receive a syntax element, and to determine the absolute coefficient level for the coefficient currently being decoded based on the predictor level and the syntax element. To determine the predictor level, the video decoder (300) may set the predictor level to be the same as the larger of the absolute coefficient level for the first neighbor coefficient or the absolute coefficient level for the second neighbor coefficient.

[0156] Figures 6a and 6b show examples of CABAC processes in empty n. In the example (400) of Figure 6a, the range in empty n is the probability (p) of the least likely symbol (LPS) given a specific context state (σ). σIncludes RangeMPS and RangeLPS given by ). Example (400) shows an update of the range at empty n+1 when the value of empty n is equal to the maximum possible symbol (MPS). In this example, the row value of the range remains the same, but the value of the range at empty n+1 is reduced to the value of RangeMPS at empty n. Example (402) in Fig. 6b shows an update of the range at empty n+1 when the value of empty n is not equal to MPS (i.e., equal to LPS). In this example, the row value of the range is moved to the lower range value of RangeLPS at empty n. Additionally, the value of the range at empty n+1 is reduced to the value of RangeLPS at empty n.

[0157] In one example of the HEVC video coding process, the range is represented by 9 bits and the low value by 10 bits. A renormalization process exists to maintain the range and low values ​​with sufficient precision. Renormalization occurs whenever the range is less than 256. Therefore, the range is always 256 or greater after renormalization. Depending on the range and low values, the Binary Arithmetic Coder (BAC) outputs a bitstream of '0' or '1', or updates an internal variable (called BO: bits-outstanding) to maintain future output. Figure 7 shows examples of range-dependent BAC outputs. For example, '1' is output to the bitstream when the range and low values ​​exceed a specific threshold (e.g., 512). '0' is output to the bitstream when the range and low values ​​are below a specific threshold (e.g., 512). When the range and low values ​​are between specific thresholds, nothing is output to the bitstream. Instead, the BO value is incremented, and the next bin is encoded.

[0158] In the CABAC context model of H.264 / AVC and some examples of HEVC, there are 128 states. There are 64 possible LPS probabilities (indicated by state σ) that can be 0 to 63. Each MPS can be 0 or 1. Thus, the 128 states are 64 state probabilities for two possible values ​​(0 or 1) for the MPS. Therefore, states can be indexed by 7 bits.

[0159] LPS ranges (rangeLPS) σ To reduce the computation required to derive ), results for all cases may be pre-calculated and stored as approximations in a look-up table. Thus, an LPS range can be obtained without multiplication by using a simple table lookup. Since this behavior can cause significant latency in many hardware architectures, avoiding multiplication may be important for some devices or applications.

[0160] Instead of multiplication, a 4-column pre-calculated LPS range table may be used. The range is divided into four segments. Segment indices can be derived by the query (range >> 6) & 3. In effect, segment indices are derived by shifting and dropping bits from the actual range. Table 1 below shows the possible ranges and their corresponding indices.

[0161] Table 1 - Range Index

[0162]

[0163] Subsequently, the LPS range table has 64 entries (one for each probabilistic state) multiplied by 4 (one for each range index). Each entry is the Range LPS, i.e., the value obtained by multiplying the range by the LPS probability. An example of a portion of this table is shown in Table 2 below. Table 2 represents probabilistic states 9-12. In one proposal for HEVC, the probabilistic states may be in the range of 0-63.

[0164] Table 2 - RangeLPS

[0165]

[0166] In each segment (i.e., range value), each probability system σ The LPS range of is predefined. That is, the LPS range of the probabilistic state σ is quantized into four values ​​(i.e., one value for each range index). The specific LPS range used at a given point depends on which segment the range belongs to. The number of possible LPS ranges used in the table is a trade-off between the number of table columns (i.e., the number of possible LPS range values) and LPS range precision. Generally speaking, more columns result in smaller quantization errors of the LPS range values ​​but also increase the need for more memory to store the table. A smaller number of columns increases quantization errors but reduces the memory required to store the table.

[0167] As explained above, each LPS probability state has a corresponding probability. The probability p for each state is derived as follows:

[0168]

[0169] Here, the state σ is from 0 to 63. The constant represents the probability change between each context state. In one example, =0.9493, and more precisely, =(0.01875 / 0.5) 1 / 63 .... The probability at state σ = 0 is equal to 0.5 (i.e., p0 = 1 / 2). That is, at context state 0, LPS and MPS are equally possible. The probability at each consecutive state is relative to the previous state It is derived by multiplying. In this way, the context state The probability of an LPS occurring at = 1 is p0 * 0.9493 (0.5 * 0.9493 = 0.47465). In this way, the state As the index of increases, the probability of LPS occurrence decreases.

[0170] CABAC is adaptive because probabilistic states are updated to follow signal statistics (i.e., the values ​​of previously coded bins). The update process is as follows: For a given probabilistic state, the update depends on the state index and the value of the encoded symbol identified as either LPS or MPS. As a result of the update process, a new probabilistic state is derived consisting of a potentially modified LPS probability estimate and, if necessary, a modified MPS value.

[0171] For empty values ​​such as MPS, the given state index may be incremented by 1. This applies to all states except when MPS occurs at state index (62), where the LPS probability is already at its minimum (or equivalently, the maximum MPS probability is reached). In this case, the state index (62) remains fixed until LPS is shown or the last empty value is encoded (state (63) is used for the special case of the last empty value). When LPS occurs, the state index is changed by decreasing the state index by a predetermined amount as shown in the equation below. This rule generally applies to each occurrence of LPS with the following exception. Assuming LPS is encoded in a state with index σ=0, this corresponds to the equal probabilistic case, where the state index remains fixed, but the MPS value will be toggled so that the values ​​of LPS and MPS are interchanged. In all other cases, regardless of which symbol is encoded, the MPS value will not change. The derivation of the transition rule for the LPS probability is given the LPS probability (p old ) and its updated corresponding part (p new It is based on the following relationship between ):

[0172] When MPS occurs,

[0173] When LPS occurs,

[0174] Regarding the actual implementation of the probability estimation process in CABAC, it is important to note that all transition rules may be realized by up to two tables, each having 63 entries of 6-bit unsigned integer values. In some examples, state transitions may be determined by a single table TransIdxLPS[σ], which determines a new updated state index TransIdxLPS[σ] when an LPS is observed for a given state index σ. MPS-driven transitions can be obtained by a simple (saturated) increment of the state index by a fixed value of 1, resulting in an updated state index min(σ+1, 62). Table 3 below is an example of a partial TransIdxLPS table.

[0175] Table 3 - TransIdxLPS

[0176]

[0177] With respect to FIGS. 6A, 6B, and 7, the techniques described above represent only one example of the implementation of CABAC. It should be understood that the techniques of this disclosure are not limited to this described implementation of CABAC. For example, in older BAC approaches (e.g., the BAC approach used in H.264 / AVC), the tables RangeLPS and TransIdxLPS were tuned for low-resolution videos (i.e., Common Intermediate Format (CIF) and Quarter-CIF (QCIF) videos). With future codecs such as HEVC and VVC, a large amount of video content is HD (high definition) and, in some cases, greater than HD. Video content with HD or higher resolution tends to have different statistics than the 10-year-old QCIF sequences used to develop H.264 / AVC. As such, the RangeLPS and TransIdxLPS tables from H.264 / AVC can cause adaptation between states in a manner that is too rapid. That is, particularly when LPS occurs, the transitions between probabilistic states can be too large for the smoother, higher-resolution content of HD video. Therefore, probabilistic models used according to conventional techniques may not be accurate for HD and extra-HD content. Furthermore, as HD video content contains a wider range of pixel values, H.264 / AVC tables do not contain enough entries to account for the more extreme values ​​that may exist in HD content.

[0178] As such, for HEVC and future coding standards such as VVC, RangeLPS and TransIdxLPS tables may be modified to account for the characteristics of these new content. In particular, BAC processes for HEVC and future coding standards may use tables that allow for slower adaptation processes and may account for more extreme cases (i.e., skewed probabilities). Thus, as an example, RangeLPS and TransIdxLPS tables may be modified to achieve these purposes by including more probabilistic states and ranges than are used in BAC for H.264 / AVC or HEVC.

[0179] FIG. 8 is a block diagram of an example entropy encoding unit (220) forming part of a video encoder (200) as illustrated in FIG. 4, which may be configured to perform CABAC according to the technique of this disclosure. A syntax element (418) is input into the entropy encoding unit (220). If the syntax element is already a binary value syntax element (i.e., a syntax element having only values ​​of 0 and 1), the binarization step may be omitted. If the syntax element is a non-binary value syntax element (e.g., a syntax element represented by multiple bits such as transform coefficient levels), the non-binary value syntax element is binarized by a binarizer (420). The binarizer (420) performs mapping the non-binary value sequence element to a sequence of binary determinations. These binary decisions are often referred to as "bins." For example, for count levels, the level values ​​may be decomposed into successive bins, each bin indicating whether the absolute value of the count level is greater than a certain value. For instance, Bin 0 (sometimes referred to as the significance flag) indicates whether the absolute value of the count level is greater than 0. Bin 1 indicates whether the absolute value of the count level is greater than 1, and so on. A unique mapping may also be deployed for each non-binary syntax element.

[0180] Each bin generated by the binarizer (420) is fed to the binary arithmetic coding side of the entropy encoding unit (220). That is, for a predetermined set of syntax elements that are not binary values, each bin type (e.g., bin 0) is coded before the next bin type (e.g., bin 1). Coding may be performed in either regular mode or bypass mode. In bypass mode, the bypass encoding engine (426) forms the arithmetic coding using a fixed probability model, for example, Golomb-Rice or exponential Golomb coding. Bypass mode is generally used for more predictable syntax elements.

[0181] Coding in normal mode may include performing CABAC. Normal mode CABAC is performed to code bin values ​​for which the probability of the bin value is predictable, given the values ​​of previously coded bins. The probability that a bin is LPS is determined by the context modeler (422). The context modeler (422) outputs a bin value and a context model (e.g., a probabilistic state σ). The context model may be an initial context model for a series of bins, or it may be determined based on the coded values ​​of previously encoded bins. As previously mentioned, the context modeler may update the state based on whether the previously coded bin was MPS or LPS.

[0182] After the context model and the probabilistic state σ are determined by the context modeler (422), the normalized encoding engine (424) performs BAC on the bin values. According to the techniques of the present disclosure, the normalized encoding engine (424) performs BAC using a TransIdxLPS table (430) containing more than 64 probabilistic states σ. In one example, the number of probabilistic states is 128. TransIdxLPS is used to determine which probabilistic state is used for the next bin (bin n+1) when the previous bin (bin n) is an LPS. The normalized encoding engine (424) may also use a RangeLPS table (428) to determine a range value for an LPS given a specific probabilistic state σ. However, according to the techniques of the present disclosure, rather than using all possible probability states σ of the TransIdxLPS table (430), probability state indices σ are mapped to grouped indices for use in the RangeLPS table. That is, each index into the RangeLPS table (428) may represent two or more of the total number of probability states. The mapping of probability state indices σ to grouped indices may be linear (e.g., by dividing by 2) or non-linear (e.g., a logarithmic function or a mapping table).

[0183] In other examples of initiation, the difference between continuous probability states is a parameter It can be made smaller by setting it to be greater than 0.9493. In one example, = 0.9689. In other examples of onset, the highest probability (p0) of LPS occurrence may be set lower than 0.5. In one example, p0 may be equal to 0.493.

[0184] According to one or more techniques of the present disclosure, the same value of a variable used to update a probabilistic state in a binary arithmetic coding process (e.g., window size, scaling factor) In contrast to using one or more of the ), and probability update rates), the entropy encoding unit (220) may use different values ​​of variables for different context models and / or different syntax elements. For example, the entropy encoding unit (220) may determine the value of a variable used to update a probability state in a binary arithmetic coding process for one of a plurality of context models, and update the probability state based on the determined value.

[0185] FIG. 9 is a block diagram of an example entropy decoding unit (302) forming part of a video decoder (300) as illustrated in FIG. 5, which may be configured to perform CABAC according to the technique of this disclosure. The entropy decoding unit (302) of FIG. 9 performs CABAC in the inverse manner of the entropy encoding unit (220) described in FIG. 8. Coded bits from a bitstream (448) are input to the entropy decoding unit (302). Based on whether the coded bits were entropy coded using bypass mode or normal mode, the coded bits are supplied to either a context modeler (450) or a bypass decoding engine (452). When the coded bits are coded in bypass mode, the bypass decoding engine (452) may use, for example, Golom-Rice or exponential Golom decoding to extract bins of non-binary syntax elements or syntax elements of binary values.

[0186] If the coded bits are coded in normal mode, the context modeler (450) may determine a probabilistic model for the coded bits, and the normal decoding engine (454) may decode the coded bits to generate bins of syntax elements of non-binary values ​​(or the syntax elements themselves if they are binary values). After the context model and the probabilistic state σ are determined by the context modeler (450), the normal decoding engine (454) performs BAC on the bin values. According to the techniques of the present disclosure, the normal decoding engine (454) performs BAC using a TransIdxLPS table (458) containing more than 64 probabilistic states σ. In one example, the number of probabilistic states is 128, but other numbers of probabilistic states consistent with the techniques of the present disclosure may be defined. The TransIdxLPS table (458) is used to determine which probability state is used for the next bin (bin n+1) when the previous bin (bin n) is an LPS. The normalized decoding engine (454) may also use the RangeLPS table (456) to determine a range value for the LPS given a specific probability state σ. However, according to the techniques of the present disclosure, rather than using all possible probability states σ of the TransIdxLPS table (458), probability state indices σ are mapped to grouped indices for use in the RangeLPS table (456). That is, each index into the RangeLPS table (456) may represent two or more of the total number of probability states. The mapping of probability state indices σ to grouped indices may be linear (e.g., by dividing by 2) or non-linear (e.g., a logarithmic function or a mapping table).

[0187] In other examples of initiation, the difference between continuous probability states is a parameter It can be made smaller by setting it to be greater than 0.9493. In one example, = 0.9689. In another example of the present disclosure, the highest probability (p0) of LPS occurrence may be set lower than 0.5. In one example, p0 may be equal to 0.493.

[0188] After the bins are decoded by the regular decoding engine (454), the debinizer (460) may perform inverse mapping to convert the bins back into values ​​of syntax elements that are not binary values.

[0189] FIG. 10 is a flowchart illustrating an example method for encoding a current block. The current block may include a current CU. Although a video encoder (200) (Fig. 1 and FIG. 4) is described, it should be understood that other devices may be configured to perform a method similar to that of FIG. 10.

[0190] In this example, the video encoder (200) initially predicts the current block (550). For example, the video encoder (200) may form a predicted block for the current block. Then, the video encoder (200) may calculate a residual block for the current block (552). To calculate the residual block, the video encoder (200) may calculate the difference between the original uncoded block and the predicted block for the current block. Then, the video encoder (200) may transform and quantize the coefficients of the residual block (554). In some coding modes, such as transform skip mode, the video encoder (200) may transform and quantize the residual data. Next, the video encoder (200) may scan the quantized coefficients of the residual block (556). During or following a scan, the video encoder (200) may entropy-encode the coefficients (558). For example, the video encoder (200) may encode the coefficients using CAVLC or CABAC. The video encoder (200) may output the entropy-encoded data of the coefficients (560).

[0191] FIG. 11 is a flowchart illustrating an exemplary method for decoding a current block of video data. The current block may include a current CU. Although a video decoder (300) (Fig. 1 and FIG. 5) has been described, it should be understood that other devices may be configured to perform a method similar to that of FIG. 11.

[0192] The video decoder (300) may receive entropy-coded data for the current block, such as entropy-coded prediction information and entropy-coded data for the coefficients of the residual block corresponding to the current block (570). The video decoder (300) may entropy-decode the entropy-coded data to determine the prediction information for the current block and regenerate the coefficients of the residual block (572). The video decoder (300) may predict the current block using, for example, an intra-prediction or inter-prediction mode, such as indicated by the prediction information for the current block, to calculate the prediction block for the current block (574). The video decoder (300) may then back-scan the regenerated coefficients to generate a block of quantized coefficients (576). The video decoder (300) may then back-quantize and back-transform the coefficients to generate a residual block (578). In some coding modes, such as transform skip mode, the video decoder (300) may skip the inverse transform of the coefficients and only perform inverse quantization. The video decoder (300) may ultimately decode the current block by combining the prediction block and the residual block (580).

[0193] FIG. 12 is a flowchart illustrating an example method for combining contexts for coding (e.g., encoding or decoding) the sign of the coefficients of a residual block. The residual block may be, for example, a transformed skipped residual block. The technique of FIG. 12 will be described for a general video coder, which may correspond to a video encoder, such as a video encoder (200) or a video decoder, such as a video decoder (300). However, it should be understood that other devices may be configured to perform a method similar to that of FIG. 12.

[0194] The video coder determines the value (X0) for the first neighbor coefficient of the coefficient currently being coded (600). The video coder determines the value (X1) for the second neighbor coefficient of the coefficient currently being coded (602). If both X0 and X1 are equal to zero (604, yes), the video coder sets the first context for coding the sign of the coefficient currently being coded (606). If at least one of X0 or X1 is not equal to zero (604, no) but X0 and X1 have opposite signs (608, yes), the video coder also sets the first context for coding the sign of the coefficient currently being coded (610).

[0195] If at least one of X0 or X1 is not equal to zero (604, no), X0 and X1 do not have opposite signs (608, no), and one of X0 or X1 is greater than zero (612, yes), the video coder also sets the context for coding the sign of the coefficient currently being coded to a second context different from the first context (614). If at least one of X0 or X1 is not equal to zero (604, no), X0 and X1 do not have opposite signs (608, no), and neither X0 or X1 is greater than zero (612, no), the video coder also sets the context for coding the sign of the coefficient currently being coded to a third context different from the first context or the second context (616).

[0196] It should be recognized that, depending on the example, certain acts or events of any of the techniques described herein may be performed in different sequences and may be added, merged, or removed in their entirety (e.g., not all described acts or events are essential for the execution of the techniques). Furthermore, in certain examples, acts or events may be performed concurrently rather than sequentially, for example, through multi-threaded processing, interrupt processing, or multiple processors.

[0197] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium or transmitted therethrough, or executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a medium of the same type as a data storage medium, or a communication medium including any medium that enables the transmission of a computer program from one place to another, for example, according to a communication protocol. In this way, computer-readable media may generally correspond to (1) non-transient types of computer-readable storage media or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to extract instructions, code and / or data structures for the implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0198] By way of example, not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. Additionally, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other ephemeral media, but instead are oriented toward non-ephemeral types of storage media. Disks and discs, as used herein, include compact discs (CDs), laser discs, optical discs, digital multifunction discs (DVDs), floppy discs, and Blu-ray discs, wherein disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Additionally, the above combinations must be included within the scope of computer-readable media.

[0199] Instructions may be executed by one or more processors, for example, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the terms "processor" and "processing circuit" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Additionally, in some embodiments, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or integrated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0200] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatus, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chip sets). Various components, modules, or units are described in the present disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but are not necessarily required to be realized by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units comprising one or more processors as described above, together with suitable software and / or firmware.

[0201] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

Claim 1 A method for decoding video data, wherein for a residual block of video data encoded using a transform skip mode, a step of determining a value for a first neighbor coefficient of a coefficient currently being decoded; a step of determining a value for a second neighbor coefficient of a coefficient currently being decoded; a step of determining a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, wherein the step of determining a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient comprises: a step of selecting a first context offset in response to both the first neighbor coefficient and the second neighbor coefficient having a value equal to zero or having opposite signs; a step of selecting a second context offset in response to both the first neighbor coefficient and the second neighbor coefficient being positive, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being positive; A method for decoding video data, comprising: determining a context offset for a coefficient currently being decoded, wherein the first neighbor coefficient and the second neighbor coefficient are both negative, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative; and decoding a value for the coefficient currently being decoded based on the determined context offset. Claim 2 A method for decoding video data according to claim 1, wherein the first neighbor coefficient includes one of the top neighbor coefficient or the left neighbor coefficient for the coefficient currently being decoded, and the second neighbor coefficient includes the other of the top neighbor coefficient or the left neighbor coefficient. Claim 3 A method for decoding video data according to claim 1, wherein the step of decoding a value for a coefficient currently being decoded based on the determined context offset comprises: determining a context based on the determined context offset; receiving one or more bins of data; and context decoding one or more bins of data based on the determined context to determine a sign for a coefficient currently being decoded. Claim 4 A method for decoding video data according to claim 3, wherein the step of decoding a value for the coefficient currently being decoded based on the determined context offset further comprises: determining a predicted level value for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; receiving a syntax element; and determining that the level value of the coefficient currently being decoded is equal to the predicted level value in response to the syntax element having a value equal to 1. Claim 5 A method for decoding video data according to claim 3, wherein the step of decoding a value for the coefficient currently being decoded based on the determined context offset comprises: determining a predicted level value for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; receiving a syntax element; and determining that, in response to the value for the syntax element being greater than the predicted level value, the level value of the coefficient currently being decoded is equal to the value for the syntax element plus 1. Claim 6 A method for decoding video data according to claim 3, wherein the step of decoding a value for the coefficient currently being decoded based on the determined context offset comprises: determining a predicted level value for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; receiving a syntax element; and determining that the level value of the coefficient currently being decoded is equal to the value for the syntax element in response to the value for the syntax element being less than the predicted level value. Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 A method for decoding video data according to claim 1, further comprising the step of inversely quantizing the value of the coefficient currently being decoded without inverse transforming it to determine the residual value for the residual block of the video data. Claim 15 A method for decoding video data according to claim 1, further comprising: determining a decoded residual block based on a value for a coefficient currently being decoded; adding the decoded residual block to a prediction block to determine a restored block; performing one or more filtering operations on the restored block to determine a decoded block of video data; and outputting a decoded picture of video data including the decoded block of video data. Claim 16 A method for encoding video data, wherein for a residual block of video data encoded using a transform skip mode, a value for a first neighbor coefficient of a coefficient currently being encoded; a value for a second neighbor coefficient of the coefficient currently being encoded; and a step of determining a context offset for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, wherein the step of determining a context offset for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient comprises: a step of selecting a first context offset in response to the first neighbor coefficient and the second neighbor coefficient both having a value equal to zero or having opposite signs; and a step of selecting a second context offset in response to the first neighbor coefficient and the second neighbor coefficient both being positive, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being positive. A method for encoding video data, comprising: determining a context offset for a coefficient currently being encoded, wherein the first neighbor coefficient and the second neighbor coefficient are both negative, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative; and encoding a value for the coefficient currently being encoded based on the determined context offset. Claim 17 A method for encoding video data according to claim 16, wherein the first neighbor coefficient includes one of the top neighbor coefficient or the left neighbor coefficient, and the second neighbor coefficient includes the other of the top neighbor coefficient or the left neighbor coefficient. Claim 18 A method for encoding video data according to claim 16, wherein the step of encoding a value for the coefficient currently being encoded based on the determined context offset comprises: determining a context based on the determined context offset; determining a sign for the coefficient currently being encoded; and context encoding one or more bins of data based on the determined context to represent the sign for the coefficient currently being encoded. Claim 19 A method for encoding video data according to claim 18, wherein the step of encoding a value for the coefficient currently being encoded based on the determined context offset comprises: determining a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and encoding a syntax element having a value equal to 1 in response to the level value of the coefficient currently being encoded being equal to the predicted level value. Claim 20 A method for encoding video data according to claim 18, wherein the step of encoding a value for the coefficient currently being encoded based on the determined context offset further comprises: determining a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and encoding a syntax element having a value equal to the level value of the coefficient currently being encoded in response to the level value of the coefficient currently being encoded being less than the predicted level value. Claim 21 A method for encoding video data according to claim 18, wherein the step of encoding a value for the coefficient currently being encoded based on the determined context offset comprises: determining a predicted level value for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient; and encoding a syntax element having a value equal to the level value of the coefficient currently being encoded minus 1 in response to the level value of the coefficient currently being encoded being greater than the predicted level value. Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete Claim 29 A method for encoding video data according to claim 16, further comprising the step of quantizing residual values ​​for residual blocks of video data to determine values ​​for coefficients currently being encoded. Claim 30 A method for encoding video data according to claim 16, further comprising: a step of determining a prediction block; and a step of comparing the original block of video data with the prediction block to determine the residual block of the video data. Claim 31 A device for decoding video data comprises: a memory configured to store video data; and one or more processors implemented in a circuit portion, wherein the one or more processors determine, for a residual block of video data encoded using a convert skip mode, a value for a first neighbor coefficient of a coefficient currently being decoded; determine a value for a second neighbor coefficient of a coefficient currently being decoded; and determine a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, wherein to determine a context offset for a coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, the one or more processors select a first context offset in response to the first neighbor coefficient and the second neighbor coefficient both having a value equal to zero or having opposite signs; A device for decoding video data, configured to determine a context offset for a coefficient currently being decoded, wherein the first neighbor coefficient and the second neighbor coefficient are both positive, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is positive; and to determine a context offset for a coefficient currently being decoded, wherein the first neighbor coefficient and the second neighbor coefficient are both negative, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative; and to decode a value for a coefficient currently being decoded based on the determined context offset. Claim 32 A device for decoding video data according to claim 31, wherein the first neighbor coefficient includes one of the top neighbor coefficient or the left neighbor coefficient, and the second neighbor coefficient includes the other of the top neighbor coefficient or the left neighbor coefficient. Claim 33 A device for decoding video data according to claim 31, wherein, in order to decode a value for a coefficient currently being decoded based on the determined context offset, the one or more processors are also configured to: determine a context based on the determined context offset; receive one or more bins of data; and context decode one or more bins of data based on the determined context to determine a sign for a coefficient currently being decoded. Claim 34 delete Claim 35 delete Claim 36 delete Claim 37 delete Claim 38 delete Claim 39 delete Claim 40 delete Claim 41 In claim 31, the one or more processors also: a device for decoding video data configured to inversely quantize a value for a coefficient currently being decoded in order to determine a residual value for a residual block of the video data. Claim 42 A device for decoding video data, wherein, in claim 31, the one or more processors also: determine a decoded residual block based on a value for the coefficient currently being decoded; add the decoded residual block to a prediction block to determine a restored block; perform one or more filtering operations on the restored block to determine a decoded block of video data; and output a decoded picture of video data including the decoded block of video data. Claim 43 A device for decoding video data according to claim 31, wherein the device comprises a wireless communication device, and the wireless communication device comprises a receiver configured to receive encoded video data. Claim 44 In claim 43, the wireless communication device includes a telephone handset, and the receiver is configured to demodulate a signal including the encoded video data according to a wireless communication standard, a device for decoding video data. Claim 45 A device for decoding video data according to claim 31, further comprising a display configured to display decoded video data. Claim 46 In claim 31, the device is a device for decoding video data, comprising one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box. Claim 47 A device for encoding video data, comprising: a memory configured to store video data; and one or more processors implemented in a circuit portion, wherein the one or more processors: determine a value for a first neighbor coefficient of a coefficient currently being encoded for a residual block of video data encoded using a convert skip mode; determine a value for a second neighbor coefficient of the coefficient currently being encoded; and determine a context offset for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, wherein to determine a context offset for the coefficient currently being encoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, the one or more processors: select a first context offset in response to the first neighbor coefficient and the second neighbor coefficient both having a value equal to zero or having opposite signs; A device for encoding video data, configured to determine a context offset for the coefficient currently being encoded, wherein the first neighbor coefficient and the second neighbor coefficient are both positive, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is positive; and to determine a context offset for the coefficient currently being encoded, wherein the first neighbor coefficient and the second neighbor coefficient are both negative, or one of the first neighbor coefficient and the second neighbor coefficient is equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient is negative; and to encode a value for the coefficient currently being encoded based on the determined context offset. Claim 48 A device for encoding video data according to claim 47, wherein the first neighbor coefficient includes one of the top neighbor coefficient or the left neighbor coefficient, and the second neighbor coefficient includes the other of the top neighbor coefficient or the left neighbor coefficient. Claim 49 A device for encoding video data, wherein, in order to encode a value for the coefficient currently being encoded based on the determined context offset, the one or more processors also: determine a context based on the determined context offset; determine a sign for the coefficient currently being encoded; and context-encode one or more bins of data based on the determined context to represent the sign for the coefficient currently being encoded. Claim 50 delete Claim 51 delete Claim 52 delete Claim 53 delete Claim 54 delete Claim 55 delete Claim 56 delete Claim 57 In claim 47, the one or more processors also: a device for encoding video data configured to quantize residual values ​​for residual blocks of video data to determine values ​​for coefficients currently being encoded. Claim 58 In claim 47, the one or more processors are also configured to: determine a prediction block; and compare the original block of video data with the prediction block to determine a residual block of the video data, a device for encoding video data. Claim 59 A device for decoding video data, for a residual block of video data encoded using a convert skip mode, means for determining a value for a first neighbor coefficient of a coefficient currently being decoded; means for determining a value for a second neighbor coefficient of a coefficient currently being decoded; means for determining a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient, wherein the means for determining a context offset for a coefficient currently being decoded based on a value for a first neighbor coefficient and a value for a second neighbor coefficient comprises: means for selecting a first context offset in response to both the first neighbor coefficient and the second neighbor coefficient having a value equal to zero or having opposite signs; An apparatus for decoding video data, comprising: means for selecting a second context offset in response to the first neighbor coefficient and the second neighbor coefficient being both positive, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being positive; means for determining a context offset for a coefficient currently being decoded, comprising the first neighbor coefficient and the second neighbor coefficient being both negative, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being negative; and means for decoding a value for the coefficient currently being decoded based on the determined context offset. Claim 60 A computer-readable storage medium storing instructions, wherein when the instructions are executed by one or more processors, the one or more processors: determine a value for a first neighbor coefficient of a coefficient currently being decoded for a residual block of video data encoded using a convert skip mode; determine a value for a second neighbor coefficient of the coefficient currently being decoded; and determine a context offset for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, wherein to determine a context offset for the coefficient currently being decoded based on the value for the first neighbor coefficient and the value for the second neighbor coefficient, the instructions cause the one or more processors: to select a first context offset in response to the first neighbor coefficient and the second neighbor coefficient both having a value equal to zero or having opposite signs; A computer-readable storage medium storing instructions, wherein a second context offset is selected in response to the first neighbor coefficient and the second neighbor coefficient being both positive, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being positive; and a context offset for the coefficient currently being decoded is determined in response to the first neighbor coefficient and the second neighbor coefficient being both negative, or one of the first neighbor coefficient and the second neighbor coefficient being equal to zero and the other of the first neighbor coefficient and the second neighbor coefficient being negative; and a value for the coefficient currently being decoded is decoded based on the determined context offset.

Citation Information

Patent Citations

  • Coding sign information of video data

    US20170142448A1

  • Coding significant coefficient information in transform skip mode

    KR1020140090658A