Method, apparatus and program for video coding

Abstract

To provide a method and a device, containing a processing circuit for a video decoding.SOLUTION: A processing circuit may decode coding information of a transfer block (TB) from a video bitstream coded. The coding information may indicate a transfer skip in one direction for the TB. The processing circuit may decode a coding value of the current transfer factor in the TB on the basis of the coding value that has been decoded before the previous transfer factor. The current transfer factor and the previous transfer factor may exist in one of the same column and the same row in the TB. One of the same column and the same row may be along the one direction of the transfer skip. The processing circuit may determine the current transfer factor in the TB on the basis of the coding value decoded of the current transfer factor.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] [Incorporation of Related Applications] This application claims the benefit of priority to U.S. Patent Application No. 17 / 339,516, entitled "METHOD AND APPARATUS FOR VIDEO CODING," filed June 4, 2021, which in turn claims the benefit of priority to U.S. Provisional Application No. 63 / 086,280, entitled "CONTEXT DESIGN FOR ENTROPY CODING SIGN MAP FOR ONE-DIMENSIONAL TRANSFORM SKIP," filed October 1, 2020. The entire disclosure of the prior application is incorporated by reference in its entirety.

[0002] [Technical field] This disclosure describes embodiments generally related to video encoding. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The work of the present inventors is not admitted explicitly or implicitly as prior art to the present disclosure, to the extent that such work is described in this background section, nor are any aspects of the description that do not specifically qualify as prior art at the time of filing.

[0004] Video encoding and decoding can be performed using motion-compensated inter-picture prediction. Uncompressed digital video can include a series of pictures, each with spatial dimensions of, for example, 1920x1080 luma samples and associated chroma samples. The series of pictures can have a fixed or variable picture rate (also informally known as frame rate), for example, 60 pictures per second or 60 Hz. Uncompressed video has specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage space.

[0005] One goal of video encoding and decoding is to reduce redundancy in an input video signal through compression. Compression can help reduce the bandwidth and / or storage space requirements, sometimes by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations of these, can be used. Lossless compression refers to techniques in which an exact copy of the original signal can be reconstructed from a compressed version of the original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for the intended application. For video, lossy compression is widely used. The amount of acceptable distortion depends on the application. For example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect higher acceptable distortion / acceptable distortion can result in higher compression ratios.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the picture can be an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and can therefore be used as the first picture in a coded video bitstream and video session or as a still image. Samples of intra-blocks can be transformed, and the transform coefficients can be quantized before entropy coding. Intra-prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and AC coefficients after the transform, the fewer bits are required at a given quantization step size to represent the block after entropy coding.

[0008] For example, conventional intra-coding, such as that known from MPEG-2 generation encoding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to predict intra-prediction from surrounding sample data and / or metadata obtained, for example, during encoding and / or decoding of spatially adjacent and previous blocks of data in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. It should be noted that, at least in some cases, intra-prediction uses reference data only from the current picture being reconstructed, and does not use reference data from reference pictures.

[0009] There may be many forms of intra-prediction. If one or more of such techniques are available in a given video coding technique, the technique used may be coded as an intra-prediction mode. In some cases, a mode may have sub-modes and / or parameters, which may be coded separately or may be included in a mode codeword. The codeword used for a given mode, sub-mode, and / or parameter combination may affect the coding efficiency gain through intra-prediction, which in turn may affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain intra-prediction modes were introduced in H.264, improved in H.265, and further refined in newer coding techniques such as joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). A prediction block can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied into the prediction block according to a direction. A reference to the direction in use can be coded in the bitstream or may itself be predicted.

[0011] Referring to FIG. 1A, shown at the bottom right is a subset of nine known predictor directions from the 33 possible predictor directions in H.265 (corresponding to the 33 angular modes out of the 35 intra modes). The point where the arrows converge (101) represents the sample to be predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples to the upper right and at an angle of 45 degrees from horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the lower left of sample (101) at an angle of 22.5 degrees from horizontal.

[0012] Still referring to FIG. 1A , a square block (104) of 4×4 samples is shown in the upper left (indicated by a thick dashed line). The square block (104) contains 16 samples, each labeled with an “S” and its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Because the block is 4×4 samples in size, S44 is located in the lower right. Also shown are reference samples following a similar numbering scheme. The reference samples are labeled with an R and their Y position (e.g., row index) and X position (column index) relative to the block (104). In both H.264 and H.265, the prediction samples are adjacent to the block being reconstructed, so negative values ​​do not need to be used.

[0013] Intra-picture prediction can work by copying reference sample values ​​from neighboring samples according to the signaled prediction direction. For example, assume that the coded video bitstream includes signaling indicating the prediction direction for this block, consistent with the arrow (102). That is, assume that the samples are predicted from one or more prediction samples to the upper right at an angle of 45 degrees from horizontal. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then, sample S44 is predicted from reference sample R08.

[0014] In some cases, particularly when the orientation is not evenly divisible by 45 degrees, the values ​​of multiple reference samples may be combined, for example by interpolation, to calculate the reference sample.

[0015] As video coding technology advances, the number of possible directions is increasing. In H.264 (2003), nine different directions could be represented. This increased to 33 in H.265 (2013), and as of the time of disclosure, JEM / VVC / BMS can support up to 65 directions. Experiments have been conducted to identify the most likely directions, and certain techniques in entropy coding have been used to represent these likely directions with a small number of bits, accepting a certain penalty for less likely directions. Furthermore, in some cases, the direction itself can be predicted from neighboring directions used in adjacent, already decoded blocks.

[0016] FIG. 1B shows a schematic diagram (180) illustrating 65 intra-prediction directions according to JEM, showing the number of prediction directions increasing over time.

[0017] The mapping of intra-prediction direction bits in a coded video bitstream to represent directions can vary between video coding techniques, ranging, for example, from simple direct mapping of prediction directions to complex adaptive schemes including intra-prediction modes, codewords, most-probable modes, and similar techniques. However, in all cases, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Because the goal of video compression is to reduce redundancy, in well-performing video coding techniques, these less likely directions are represented by more bits than more likely directions.

[0018] Motion compensation is a lossy compression technique, and can be associated with a technique in which blocks of sample data from a previously reconstructed picture or part thereof (reference picture) are used to predict a newly reconstructed picture or part thereof after being spatially shifted in a direction indicated by a motion vector (hereinafter referred to as MV). In some cases, the reference picture can be the same as the picture currently being reconstructed. The MV may have two dimensions, X and Y, or it may have three dimensions, with the third dimension indicating the reference picture in use (the latter may indirectly be the temporal dimension).

[0019] In some video compression techniques, the MV applicable to a particular region of sample data can be predicted from other MVs, e.g., from MVs associated with other regions of sample data that are spatially adjacent to the region being reconstructed and precede that MV in decoding order. This can significantly reduce the amount of data required to encode the MV, thereby removing redundancy and increasing compression. For example, when encoding an input video signal derived from a camera (known as natural video), MV prediction can work effectively because there is a statistical possibility that regions larger than the region to which a single MV is applicable move in a similar direction and therefore can potentially be predicted using similar motion vectors derived from MVs of neighboring regions. As a result, the detected MV for a given region will be similar or identical to the MV predicted from surrounding MVs, and after entropy coding, the MV can be represented with fewer bits than would be used to directly encode the MV. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., an MV) derived from the original signal (i.e., a sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when calculating a predictor from several surrounding MVs.

[0020] H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016) describes various MV prediction mechanisms. Among the many MV prediction mechanisms provided by H.265, a technique called "spatial merge" will be described in this specification.

[0021] Referring to Figure 2, the current block (201) contains samples that the encoder found during the motion search process to be predictable from a spatially shifted previous block of the same size. Instead of directly encoding its MV, the MV can be derived from metadata associated with one or more reference pictures, e.g., using the MV associated with any one of five surrounding samples denoted A0, A1, and B0, B1, B2 (202-206, respectively) from the most recent reference picture (in decoding order). In H.265, MV prediction can use predictors from the same reference picture as neighboring blocks. Summary of the Invention

[0022] Aspects of the present disclosure provide methods and apparatuses for video encoding and / or decoding. In some examples, the apparatus for video decoding includes a processing circuit. The processing circuit may decode coding information for a transform block (TB) from a coded video bitstream. The coding information may indicate a unidirectional transform skip for the TB. The processing circuit may decode a code value of a current transform coefficient in the TB based on a previously decoded code value of a previous transform coefficient. The current transform coefficient and the previous transform coefficient may be in one of the same row and the same column in the TB. The same row and the same column may be along one direction of the transform skip. The processing circuit may determine a current transform coefficient in the TB based on the decoded code value of the current transform coefficient.

[0023] In one embodiment, the one direction is the horizontal direction. One of the same row and the same column is the same row along one direction of the transform skip. The processing circuit may decode the code value of the current transform coefficient in the TB based on the previously decoded code value of the previous transform coefficient in the same row along one direction of the transform skip. In one example, the same row is limited to one of a subset of rows in the TB.

[0024] In one embodiment, the one direction is a vertical direction. One of the same row and the same column is the same column along one direction of the transform skip. The processing circuit may decode the code value of the current transform coefficient in the TB based on the previously decoded code value of the previous transform coefficient in the same column along one direction of the transform skip. In one example, the same column is limited to one of a subset of columns in the TB.

[0025] In one embodiment, the processing circuitry may decode a flag that may indicate whether the code values ​​of transform coefficients in the same row are the same, and the transform coefficients in the same row may include a current transform coefficient and a previous transform coefficient.

[0026] In one embodiment, the processing circuitry may decode a flag that may indicate whether the code values ​​of transform coefficients in the same column are the same, and the transform coefficients in the same column may include a current transform coefficient and a previous transform coefficient.

[0027] The processing circuit may decode the code value of the current transform coefficient in the TB based on the previously decoded code value of the previous transform coefficient, the previously decoded code value of the DC transform coefficient in the upper adjacent block of the TB, and the previously decoded code value of the DC transform coefficient in the left adjacent block of the TB. The DC transform coefficient in the upper adjacent block of the TB and the DC transform coefficient in the left adjacent block of the TB may have a DC spatial frequency.

[0028] In one example, the processing circuit decodes the code value of the current transform coefficient in the TB based on the previously decoded code value of the previous transform coefficient and a weighted average of (i) the previously decoded code value of the DC transform coefficient in the upper adjacent block of the TB and (ii) the previously decoded code value of the DC transform coefficient in the left adjacent block of the TB. The weighting may be based on the number of transform coefficients in the upper adjacent block and the left adjacent block that respectively border the transform coefficient in the TB. The position of the previous transform coefficient in the TB may be adjacent to the position of the current transform coefficient in the TB.

[0029] In one example, the position of the previous transform coefficient in the TB is adjacent to the position of the current transform coefficient in the TB along the scanning direction, which may be based on a scanning order for coding the code values ​​of the transform coefficients in the TB.

[0030] In one embodiment, the processing circuit may decode the code value of a current transform coefficient in the TB based on previously decoded code values ​​of a previous transform coefficient and one or more previously decoded code values ​​of one or more transform coefficients in the same row and one of the same column, respectively.

[0031] Aspects of the present disclosure also provide a non-transitory computer-readable medium having stored thereon instructions that, when executed by a computer, cause the computer to perform a method for video encoding and / or decoding. [Brief explanation of the drawings]

[0032] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Figure 1A] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes. [Figure 1B] FIG. 1 is a diagram of an exemplary intra-prediction direction. [Figure 2] FIG. 1 is a schematic diagram of a current block and its surrounding spatial merge candidates in one example. [Figure 3]FIG. 3 is a simplified block diagram schematic of a communication system (300) according to one embodiment. [Figure 4] FIG. 4 is a simplified block diagram schematic of a communication system (400) according to one embodiment. [Figure 5] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 6] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 7] 10 shows a block diagram of an encoder according to another embodiment; [Figure 8] 10 shows a block diagram of a decoder according to another embodiment; [Figure 9] 1 illustrates an example of a linear transformation basis function according to an embodiment of the present disclosure. [Figure 10A] 10 illustrates an example dependency of availability of various transform kernels based on transform block size and prediction mode, according to an embodiment of the present disclosure. [Figure 10B] 10 illustrates an example transform type selection based on intra-prediction mode for chroma components according to an embodiment of this disclosure. [Figure 11] 1 illustrates an example of a general line graph transform (LGT) characterized by self-loop weights and edge weights, according to one embodiment of the present disclosure. [Figure 12] 1 illustrates an exemplary generalized graph Laplacian (GGL) matrix according to one embodiment of the present disclosure. [Figure 13A] 1 illustrates an example of a transform block (TB) and neighboring blocks according to one embodiment of the present disclosure. [Figure 13B] 1 illustrates an example of a TB according to an embodiment of the present disclosure. [Figure 14] 14 shows a flow chart that outlines a process (1400) according to one embodiment of the present disclosure. [Figure 15] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0033] Figure 3 shows a simplified block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes, for example, multiple terminal devices that can communicate with each other via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of Figure 3, the first pair of terminal devices (310) and (320) perform unidirectional data transmission. For example, the terminal device (310) may encode video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data, reconstruct the video pictures, and display the video pictures according to the reconstructed video data. One-way data transmission may be common, such as in media presentation applications.

[0034] In another example, the communication system 300 includes a second pair of terminal devices 330 and 340 for performing bidirectional transmission of encoded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices 330 and 340 may encode video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices 330 and 340 over the network 350. Each of the terminal devices 330 and 340 may also receive the encoded video data transmitted by the other of the terminal devices 330 and 340, decode the encoded video data to reconstruct the video pictures, and display the video pictures on an accessible display device in accordance with the reconstructed video data.

[0035] In the example of FIG. 3 , the terminal devices 310, 320, 330, and 340 may be depicted as a server, a personal computer, and a smartphone, although the principles of the present disclosure are not limited thereto. Embodiments of the present disclosure may also be applied to laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. The network 350 represents any number of networks that convey coded video data between the terminal devices 310, 320, 330, and 340, including, for example, wired (hardwired) and / or wireless communication networks. The communication network 350 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network 350 are not important to the operation of the present disclosure, unless otherwise described herein.

[0036] 4 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter, which is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media (including CDs, DVDs, memory sticks, etc.), etc.

[0037] The streaming system may include a capture subsystem (413), which may include, for example, a video source (401) (e.g., a digital camera) that generates an uncompressed stream of video pictures (402). In one example, the stream of video pictures (402) includes samples captured by a digital camera. The stream of video pictures (402), depicted as a thick line to emphasize its higher data volume compared to the encoded video data (404) (or encoded video bitstream), may be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403), as described in more detail below, may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter. The encoded video data (404) (or encoded video bitstream (404)), depicted as a thin line to emphasize its lower data volume compared to the stream of video pictures (402), may be stored on a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) in Figure 4, may access the streaming server (405) to obtain copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include, for example, a video decoder (410) within an electronic device (430). The video decoder (410) decodes the input copy (407) of the encoded video data and generates an output stream (411) of video pictures that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) may be encoded according to a particular video encoding / compression standard. Examples of these standards include ITU-T Recommendation H.265.In one example, a developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0038] It should be noted that electronic devices 420 and 430 may include other components (not shown). For example, electronic device 420 may include a video decoder (not shown), and electronic device 430 may include a video encoder (not shown).

[0039] 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used in place of the video decoder (410) in the example of FIG. 4.

[0040] The receiver (531) may receive one or more coded video sequences to be decoded by the video decoder (510), and in the same or other embodiments, may receive one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (531) may receive the coded video data along with other data (e.g., coded audio data and / or auxiliary data streams), which may be forwarded to respective using entities (not shown). The receiver (531) may separate the coded video sequences from the other data. To prevent network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereinafter referred to as "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be external to the video decoder (510) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (510), for example, to prevent network jitter, and in addition, there may be another buffer memory (515) internal to the video decoder (510), for example, to address playback timing. If the receiver (531) is receiving data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isochronous network, the buffer memory (515) may not be required or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be required and may be relatively large, advantageously adaptively sized, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (510).

[0041] The video decoder (510) may include a parser (520) for recovering symbols (521) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (510) and potentially include information for controlling a rendering device, such as a rendering device (512) (e.g., a display screen). The rendering device (512) may not be an integral part of the electronic device (530) but may be coupled to the electronic device (530), as shown in FIG. 5. The rendering device control information may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (520) may extract a set of subgroup parameters for at least one subgroup of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0042] The parser (520) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (515) to generate symbols (521).

[0043] The recovery of the symbols (521) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors. Which units are involved and how may be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not shown for clarity.

[0044] In addition to the functional blocks described above, the video decoder (510) may be conceptually subdivided into multiple functional units, as described below. In a practical implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0045] The first unit is a scalar / inverse transform unit (551), which receives quantized transform coefficients as symbols (521) from the parser (520), along with control information (including which transform to use, block size, quantization coefficients, quantization scaling matrix, etc.). The scalar / inverse transform unit (551) may output blocks containing sample values ​​that can be input to an aggregator (555).

[0046] In some cases, the output samples of the scaler / inverse transform (551) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape as the block being reconstructed using surrounding, already reconstructed information retrieved from the current picture buffer (558). The current picture buffer (558), for example, buffers the partially reconstructed and / or fully reconstructed current picture. In some cases, the aggregator (555) adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551).

[0047] In other cases, the output samples of the scalar / inverse transform unit (551) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (553) may access the reference picture memory (557) to retrieve samples used for prediction. After motion-compensating the retrieved samples according to the symbols (521) associated with the block, these samples may be added by the aggregator (555) to the output of the scalar / inverse transform unit (551) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) available to the motion-compensated prediction unit (553) from which the motion-compensated prediction unit (553) retrieves prediction samples may be controlled by a motion vector, e.g., in the form of a symbol (521) that may have X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​retrieved from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0048] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). The video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called a coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), which may be responsive to meta-information obtained during decoding of previous portions of the coded picture or coded video sequence (in decoding order), as well as to previously reconstructed loop-filtered sample values.

[0049] The output of the loop filter unit (556) may be a sample stream that may be output to a rendering device (512) and stored in a reference picture memory (557) for use in future inter-picture prediction.

[0050] Once a particular coded picture is fully reconstructed, it may be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) may become part of the reference picture memory (557), and a new current picture buffer may be reallocated before beginning reconstruction of the subsequent coded picture.

[0051] The video decoder (510) may perform decoding operations according to a predetermined video compression technique in a standard, such as ITU-T Rec. H.265. A coded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that the coded video sequence conforms to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, a profile may select certain tools from all tools available in the video compression technique or standard as the only tools available for use with that profile. Compliance also requires that the complexity of the coded video sequence be within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstructed sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. In some cases, the limits set by the level may be further constrained through a hypothetical reference decoder (HRD) specification and metadata about HRD buffer management conveyed in the coded video sequence.

[0052] In one embodiment, the receiver (531) may receive additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately recover the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0053] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmitting circuit). The video encoder (603) may be used in place of the video encoder (403) in the example of FIG. 4.

[0054] The video encoder (603) may receive video samples from a video source (601) (which in the example of FIG. 6 is not part of the electronic device (620)), which may capture video images to be encoded by the video encoder (603). In other examples, the video source (601) is part of the electronic device (620).

[0055] The video source (601) may provide a source video sequence to be encoded by the video encoder (603) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 Y CrCB, RGB, etc.), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media presentation system, the video source (601) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, and each pixel may contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0056] According to one embodiment, the video encoder (603) may encode and compress pictures of a source video sequence into a coded video sequence (643) in real time or under any other time constraint required by the application. Achieving an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units, as described below. Coupling is not shown for clarity. Parameters set by the controller (650) may include rate control-related parameters (e.g., picture skip, quantization, lambda value for rate-distortion optimization techniques), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (650) may also be configured with other appropriate functionality associated with the video encoder (603) optimized for a particular system design.

[0057] In some embodiments, the video encoder (603) is configured to operate in an encoding loop. As a very simplified description, in one example, the encoding loop may include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be encoded and reference pictures) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to generate sample data similar to what a (remote) decoder would generate (so that any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Because decoding of the symbol stream yields bit-for-bit accurate results independent of the location (local or remote) of the decoder, the contents in the reference picture memory (634) are also bit-for-bit accurate between the local encoder and the remote encoder. In other words, the prediction part of the encoder "sees" the exact same sample values ​​as the reference picture samples that the decoder "sees" when using the prediction during decoding. This basic principle of reference picture synchronization (including the resulting drift when synchronization cannot be maintained, for example, due to channel error) is used in some related technologies as well.

[0058] The operation of the "local" decoder (633) may be the same as a "remote" decoder, such as the video decoder (510), which has already been described in detail above in connection with Figure 5. However, with brief reference to Figure 5, because symbols are available and the encoding / decoding of symbols into an encoded video sequence by the entropy coder (645) and parser (520) may be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), may not be fully implemented in the local decoder (633).

[0059] An observation that can be made at this point is that any decoder technology, other than analysis / entropy decoding, that exists in the decoder must necessarily exist in substantially the same functional form in the corresponding encoder. For this reason, the subject matter of this disclosure focuses on decoder operation. A description of the encoder technology can be omitted, as it is the reverse of the decoder technology, which is described generically. Only in certain areas is more detailed explanation necessary, and is provided below.

[0060] In some examples, during operation, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (632) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0061] The local video decoder (633) may decode the coded video data of pictures that may be designated as reference pictures based on symbols generated by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data can be decoded by a video decoder (not shown in FIG. 6), the reconstructed video sequence may typically be a replica of the source video sequence, with some errors. The local video decoder (633) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in a reference picture cache (634). In this way, the video encoder (603) may locally store copies of reconstructed reference pictures that have common content as reconstructed reference pictures (without transmission errors) obtained by a far-end video decoder.

[0062] The predictor (635) may perform a predictive search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata (reference picture motion vectors, block shapes, etc.), which may serve as suitable prediction references for the new picture. The predictor (635) may operate sample block-by-pixel block to find suitable prediction references. In some cases, the input picture determined by the search results obtained by the predictor (635) may have prediction references drawn from multiple reference pictures stored in the reference picture memory (634).

[0063] The controller (650) may manage the encoding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0064] The output of all the above functional units may undergo entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0065] The transmitter (640) may buffer the coded video sequence produced by the entropy coder (645) and prepare it for transmission over a communication channel (660), which may be a hardware or software link to a storage device that stores the coded video data. The transmitter (640) may merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or an auxiliary data stream (not shown).

[0066] The controller (650) may manage the operation of the video encoder (603). During encoding, the controller (650) may assign each coded picture a particular coding picture type. The coding picture type may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types:

[0067] An intra picture (I-picture) may be one that can be coded and decoded without using other pictures in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will recognize these variations of I-pictures and their respective uses and characteristics.

[0068] A predicted picture (P picture) may be coded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values ​​of each block.

[0069] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, using up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0070] In general, a source picture may be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks, as determined by the coding assignment applied to the respective picture of the blocks. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial or temporal prediction with reference to a reference picture coded one step earlier. Blocks of a B-picture may be predictively coded via spatial or temporal prediction with reference to a reference picture coded one or two steps earlier.

[0071] The video encoder (603) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (603) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to a syntax specified by the video encoding technique or standard being used.

[0072] In one embodiment, the transmitter (640) may transmit additional data along with the coded video. The source coder (630) may include such data as part of the coded video sequence. The additional data may include other types of redundant data, such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0073] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into blocks. If a block in the current picture is similar to a reference block in a previously encoded and still buffered reference picture in the video, the block in the current picture may be coded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0074] In some embodiments, bidirectional prediction techniques may be used in inter-picture prediction. According to bidirectional prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used, both preceding the current picture in decoding order (but may also be past and future, respectively, in display order) in the video. A block in the current picture may be coded with a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. A block may be predicted by a combination of the first and second reference blocks.

[0075] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0076] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Typically, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the CU's prediction type, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0077] 7 shows a diagram of a video encoder (703) according to another embodiment of this disclosure. The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of the coded video sequence. In one example, the video encoder (703) is used in place of the video encoder (403) of the example of FIG. 4.

[0078] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (703) determines, for example, using rate-distortion optimization, whether the processing block is best coded using intra mode, inter mode, or bidirectional prediction mode. If the processing block is coded in intra mode, the video encoder (703) may use intra prediction techniques to code the processing block into a coded picture. If the processing block is coded in inter mode or bidirectional prediction mode, the video encoder (703) may use inter prediction techniques or bidirectional prediction techniques, respectively, to code the processing block into a coded picture. In certain video coding techniques, merge mode may be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of coded motion vector components other than the motion vector predictor. In certain other video coding techniques, there may be motion vector components applicable to the block of interest. In one example, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0079] In the example of Figure 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), an overall controller (721), and an entropy encoder (725) coupled together as shown in Figure 7.

[0080] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-prediction information (e.g., a description of redundant information due to inter-coding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture decoded based on the coded video information.

[0081] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), optionally compare the block with previously coded blocks in the same picture, generate quantized coefficients after transformation, and optionally generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same picture.

[0082] The overall controller (721) is configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In one example, the overall controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is intra mode, the overall controller (721) controls the switch (726) to select the intra mode result to be used by the residual calculator (723) and controls the entropy encoder (725) to select intra prediction information and include the intra prediction information in the bitstream. If the mode is inter mode, the overall controller (721) controls the switch (726) to select the inter prediction result to be used by the residual calculator (723) and controls the entropy encoder (725) to select inter prediction information and include the inter prediction information in the bitstream.

[0083] The residual calculator (723) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the intra-encoder (722) or inter-encoder (730). The residual encoder (724) operates based on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform to generate decoded residual data. The decoded residual data may be used by the intra-encoder (722) and inter-encoder (730), as appropriate. For example, the inter-encoder (730) may generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-encoder (722) may generate decoded blocks based on the decoded residual data and intra-prediction information. The decoded blocks are processed appropriately to generate decoded pictures, which may be buffered in a memory circuit (not shown) and, in some examples, used as reference pictures.

[0084] The entropy encoder (725) is configured to format the bitstream to include the coded block. The entropy encoder (725) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include global control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that, according to the disclosed subject matter, when encoding a block in a merged sub-mode of either the inter mode or the bi-prediction mode, the residual information is not present.

[0085] 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (810) is used in place of the video decoder (410) of the example of FIG. 4.

[0086] In the example of Figure 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872) coupled together as shown in Figure 8.

[0087] The entropy decoder (871) may be configured to recover, from a coded picture, specific symbols representing the syntax elements of which the coded picture is composed. Such symbols may include, for example, prediction information (e.g., intra-mode, inter-mode, bi-predictive mode, merged submode, or the latter two in other submodes) that can identify the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-predictive mode, merged submode, or the latter two in other submodes), specific samples or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), respectively (e.g., intra-predictive information or inter-predictive information), residual information in the form of quantized transform coefficients, etc. In one example, if the prediction mode is an inter-mode or bi-predictive mode, the inter-predictive information is provided to the inter-decoder (880), and if the prediction type is an intra-predictive type, the intra-predictive information is provided to the intra-decoder (872). The residual information may undergo inverse quantization and be provided to the residual decoder (873).

[0088] The inter decoder (880) is configured to receive the inter prediction information and generate an inter prediction result based on the inter prediction information.

[0089] The intra decoder (872) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0090] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (including quantizer parameters (QP)), which may be provided by the entropy decoder (871) (data path not shown as this may only be low-volume control information).

[0091] The reconstruction module (874) is configured to combine, in the spatial domain, the residual output by the residual decoder (873) and the prediction result (possibly output by an inter-prediction module or an intra-prediction module) to form a reconstruction block, which may be part of a reconstructed picture or part of a reconstructed video. It should be noted that other suitable operations, such as a debooking operation, may also be performed to improve visual quality.

[0092] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technology. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (510), (610), and (810) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0093] Aspects of the present disclosure include context designs for entropy coding code maps for one-dimensional (1D) transform skips. A set of video coding techniques designed for efficient compression of video data can be provided, including context designs for entropy coding code values ​​of 1D transform skip coefficients.

[0094] An embodiment of a linear transform such as that used in AV1 (AOMedia Video 1) is described below: To support extended coding block partitioning, multiple transform sizes (e.g., ranging from 4 points to 64 points in each dimension) and transform shapes (e.g., square, rectangle with width to height ratio of 2:1, 1:2, 4:1, or 1:4) may be used in AV1 and the like.

[0095] The 2D transform process may use hybrid transform kernels, which may include different 1D transforms for each dimension of the coding residual block. The linear 1D transforms may include (a) 4-point, 8-point, 16-point, 32-point, and 64-point DCT-2, (b) 4-point, 8-point, and 16-point asymmetric DSTs (ADSTs) (e.g., DST-4 and DST-7) and their corresponding flipped versions (e.g., a flipped version of ADST, or FlipADST, can be applied to ADSTs in the reverse order), and / or (c) 4-point, 8-point, 16-point, and 32-point identity transforms (IDTXs). Figure 9 shows example linear transform basis functions according to embodiments of the present disclosure. The linear transform basis functions in the example of Figure 9 include basis functions for DCT-2 and asymmetric DSTs (DST-4 and DST-7) with N-point inputs. The linear transform basis functions shown in Figure 9 may be used in AV1.

[0096] The availability of hybrid transform kernels may depend on transform block size and prediction mode. Figure 10A shows an exemplary dependency of the availability of various transform kernels (e.g., transform types shown in the first column and descriptions in the second column) based on transform block size (e.g., sizes shown in the third column) and prediction mode (e.g., intra-prediction and inter-prediction shown in the third column). Exemplary hybrid transform kernels and availability based on prediction mode and transform block size may be used in AV1. Referring to Figure 10A, the "→" and "↓" symbols indicate the horizontal dimension (also referred to as the horizontal direction) and the vertical dimension (also referred to as the vertical direction), respectively.

[0097]

number

[0098]

number

[0099] In one example, as shown in the first column of Figure 10A, the transform type (1010) is indicated by ADST_DCT. As shown in the second column of Figure 10A, the transform type (1010) includes ADST in the vertical direction and DCT in the horizontal direction. According to the third column of Figure 10A, when the block size is 16x16 or less (e.g., 16x16 samples, 16x16 luma samples), the transform type (1010) can be used for intra prediction and inter prediction.

[0100] In one example, as shown in the first column of FIG. 10A, the transform type (1020) is indicated by V_ADST. As shown in the second column of FIG. 10A, the transform type (1020) includes a vertical ADST and a horizontal IDTX (i.e., identity matrix). Therefore, the transform type (1020) (e.g., V_ADST) is performed vertically and not horizontally. According to the third column of FIG. 10A, the transform type (1020) is not available for intra prediction regardless of the block size. If the block size is less than 16x16 (e.g., 16x16 samples, 16x16 luma samples), the transform type (1020) is available for inter prediction.

[0101] In one example, Figure 10A is applicable to the luma component. For the chroma components, the selection of the transform type (or transform kernel) may be performed implicitly. In one example, for intra-prediction residuals, the transform type may be selected according to the intra-prediction mode, as shown in Figure 10B. For inter-prediction residuals, the transform type may be selected according to the transform type selection of the co-located luma block. Thus, in one example, the transform type of the chroma components is not signaled in the bitstream.

[0102] The line graph transform (LGT) may be used in transforms such as linear transforms in AV2 (AOMedia Video 2). In AV2, an 8-bit / 10-bit transform core may be used. In one example, the LGT includes various DCTs, discrete sine transforms (DSTs), as described below. The LGT may include 32-point and 64-point one-dimensional (1D) DSTs.

[0103] A graph is a general mathematical structure containing a set of vertices and edges that can be used to model affinity relationships between objects of interest. A weighted graph, in which a set of weights is assigned to edges and optionally to vertices, may provide a sparse representation for robust modeling of signals / data. LGTs can improve coding efficiency by providing better adaptation to various block statistics. A separable LGT can be designed and optimized by learning a line graph from data to model the statistics of the underlying row- and column-wise block residual signals, and the associated generalized graph Laplacian (GGL) matrix may be used to derive the LGT.

[0104] FIG. 11 illustrates a self-loop weight (e.g., v c1 , v c2 ) and edge weight w c Given a weighted graph G(W,V), the GGL matrix may be defined as follows: L c =D-W+V (Formula 1) where W is the non-negative edge weight w c where D is a diagonal order matrix, and V is the self-loop weight v c1 and v c2 12 shows the matrix L c An example is shown below.

[0105] LGT is the GGL matrix L as follows: c may be derived by eigendecomposition of L c =UΦU T (Formula 2) Here, the columns of the orthogonal matrix U may be basis vectors of the LGT, and Φ may be a diagonal eigenvalue matrix.

[0106] In various examples, particular DCTs and DSTs (e.g., DCT-2, DCT-8, and DST-7) are subsets of the set of LGTs derived from a particular form of GGL. c1 Set to 0 (e.g., v c1 = 0). DST-7 can be derived by c1 W c (e.g., v c1 =w c ) The DCT-8 can be derived by c2 W c (e.g., v c2 =w c ) DST-4 can be derived by c1 2w c (e.g., v c1 =2w c ) DCT-4 can be derived by c2 2w c (e.g., v c2 =2w c ) may be derived.

[0107] In some instances, such as AV2, LGT may be implemented as a matrix multiplication. A 4-point (4p) LGT core c In v c1 2w c Therefore, a 4p LGT core is a DST-4. An 8-point (8p) LGT core is a c In v c1 1.5W c In one example, v c1 W c and v c2 By setting ≠ 0, an LGT core such as a 16-point (16p) LGT core, a 32-point (32p) LGT core, or a 64-point (64p) LGT core may be derived, and the LGT core may become a DST-7.

[0108] According to aspects of the present disclosure, a block may refer to a transform block (TB). In one example, a block or TB includes transform coefficients. A row within a block may refer to a row of transform coefficients within a block (e.g., TB). A column within a block may refer to a column of transform coefficients within a block (e.g., TB).

[0109] The letter "H" or "V" when appended to a transform type may indicate that the transform type applies only to the horizontal or vertical direction, respectively. For example, V_DCT may indicate a 1D transform method that applies a DCT only to the vertical direction, H_DCT may indicate a 1D transform method that applies a DCT only to the horizontal direction, V_ADST may indicate a 1D transform method that applies an ADST or any suitable non-DCT transform only to the vertical direction, and H_ADST may indicate a 1D transform method that applies an ADST or any suitable non-DCT transform only to the horizontal direction. V_FLIPADST may indicate a 1D transform method that applies a FLIPADST only to the vertical direction, and H_FLIPADST may indicate a 1D transform method that applies a FLIPADST only to the horizontal direction. V_LGT may indicate a 1D transform method that applies an LGT only to the vertical direction, and H_LGT may indicate a 1D transform method that applies an LGT only to the horizontal direction. V_FLIPLGT may refer to a 1D transformation method that applies FLIPLGT only in the vertical direction, and H_FLIPLGT may refer to a 1D transformation method that applies FLIPLGT only in the horizontal direction, where FLIPLGT is an inverted version of LGT. Furthermore, V_KLT may refer to a 1D transformation method that applies the Karhunen-Loeve transform (KLT) only in the vertical direction, and H_KLT may refer to a 1D transformation method that applies KLT only in the horizontal direction.

[0110] A set of transform types having a transform skip in the horizontal direction may be referred to as transform set A. Transform set A may include a transform in the vertical direction. The transforms in transform set A may include one or a suitable combination of predetermined transforms such as V_DCT, V_ADST, V_FLIPADST, V_LGT, V_FLIPLGT, V_KLT, etc. In one example, the transforms in transform set A include a linear transform in the vertical direction. In one example, the transforms in transform set A include a linear transform in the vertical direction and a secondary transform in the vertical direction. In one example, transform type set A exhaustively encompasses any or all combinations of predetermined transforms such as DCT, ADST, FLIPADST, LGT, FLIPLGT, and KLT in the vertical direction with a transform skip in the horizontal direction.

[0111] The set of transform types having a transform skip in the vertical direction may be referred to as transform set B. Transform set B may include a transform in the horizontal direction. The transforms in transform set B may include one or a suitable combination of predetermined transforms such as H_DCT, H_ADST, H_FLIPADST, H_LGT, H_FLIPLGT, H_KLT, etc. In one example, the transforms in transform set B include a linear transform in the horizontal direction. In one example, the transforms in transform set B include a linear transform in the horizontal direction and a secondary transform in the horizontal direction. In one example, transform type set B exhaustively encompasses any or all combinations of predetermined transforms such as DCT, ADST, FLIPADST, LGT, FLIPLGT, and KLT in the horizontal direction with a transform skip in the vertical direction.

[0112] In one embodiment, a level map coefficient coding scheme may be used in AV1, etc. The level map coefficient coding scheme differs from related coding schemes that process each 2D transform coefficient sequentially. An example of a level map coefficient coding scheme is described below. For each transform unit (e.g., TB (1301) in FIG. 13A), the level map coefficient coding may first code a skip code, followed by signaling the primary transform kernel type and end-of-block (EOB) position if the transform coding is not skipped. Subsequently, coefficient values ​​(i.e., values ​​of transform coefficients within the transform unit) may be coded using a multi-level map scheme combined with the coding of the code values.

[0113] In one example, the multi-level map includes three level maps coded as three level planes, such as a lower level plane, a mid-level plane, and an upper level plane, with the sign values ​​coded as separate planes. The lower level plane, the mid-level plane, and the upper level plane may correspond to different ranges of coefficient magnitude. In one example, the lower level plane corresponds to a range of 0 to 2, the mid-level plane corresponds to a range of 3 to 14, and the upper level plane corresponds to a range of 15 or greater. The three level planes may be coded as follows: (a) the EOB position is coded; (b) the lower level plane and the mid-level plane are coded together in a backward scan order, where the scan order may indicate a zigzag scan applied on a whole transform unit basis; (c) the sign plane and the upper level plane are coded together in a forward scan order; and (d) the remainder (coefficient level-14) is entropy coded using an Exponential-Golomb code. The context model applied to the lower-level planes may depend on the primary transform direction (e.g., bidirectional, horizontal, and vertical) and the transform size, and up to five neighboring coefficients in the frequency domain may be used to derive the context. Mid-level planes may use similar context models. In one example, the number of context neighboring coefficients is reduced from five to two. Higher-level planes may be coded by exponential-Golomb coding without using a context model.

[0114] 13A illustrates an example of TB(1301) and neighboring blocks (e.g., TB(1302) and (1303)) according to one embodiment of the present disclosure. TB(1301) may include transform coefficients (e.g., t11-t14, t21-t24, t31-t34, and t41-t44). In some examples, transform coefficient t11 ​​is a DC transform coefficient having a DC spatial frequency, and the remaining transform coefficients in TB(1301) are AC transform coefficients having AC spatial frequencies.

[0115] A DC code (or DC code value), for example, the code value of a DC coefficient in a transform unit (e.g., DC transform coefficient t11 ​​in TB(1301)), may be coded using a context modeling technique. In one example, if the DC coefficient is negative, the DC code value is “1,” and if the DC coefficient is positive, the DC code value is “0.” A weighted average of the DC code values ​​of neighboring blocks of a transform unit may be used to derive context information. In one example, the DC code value (e.g., dc_sign(i), where i is 1) of an upper neighboring block (e.g., TB(1302)) and the DC code value (e.g., dc_sign(i), where i is 2) of a left neighboring block (e.g., TB(1303)) are used to derive context information, as described in Equation 3 below. The weighting may depend on the intersection length (or intersection length) (e.g., overlap(i,curr_block), where i is 1 or 2) of the neighboring transform block and the transform unit (e.g., current TB). The overlap length between a first block and a second block may indicate the number of transform coefficients in the first block that border on a transform coefficient in the second block. For example, four transform coefficients (t11-t14) in TB(1301) border on four transform coefficients (a11-a14) in TB(1302), so the overlap length (e.g., overlap(1, curr_block)) between TB(1301) and TB(1302) is 4. Two transform coefficients (t11 and t12) in TB(1301) border on two transform coefficients (112 and 122) in TB(1303), so the overlap length (e.g., overlap(2, curr_block)) between TB(1301) and TB(1303) is 2.

[0116] The derived context information may be used as an index (e.g., dc_ctx) to access three different contexts for coding the DC code, as described in Equation 4. For example, if the weighted average is 0, the context corresponding to index 0 is used; if the weighted average is less than 0, the context corresponding to index 1 is used; and if the weighted average is greater than 0, the context corresponding to index 2 is used. Code values ​​of other coefficients in the transform unit may be coded directly without using a context model. dc_sum=Σ i∈neighbors dc_sign(i)*overlap(i,curr_block) (Formula 3) dc_ctx=0 (if dc_sum=0) =1 (if dc_sum<0) =2 (if dc_sum>0) (Equation 4) The code values ​​of other transform coefficients (eg, AC transform coefficients) in the TB (1301) may be coded directly without using a context model.

[0117] As described above with reference to FIG. 13A, in some examples, coding of the DC code utilizes only the DC codes of the upper and left neighbors to derive a context used to code the DC code of the DC transform coefficient in the current transform block. In one example, coding of the DC code utilizes only the DC codes of the upper and left neighbors to derive context information, and the context information may be used to derive a context for coding the DC code of the DC transform coefficient in the current transform block. For a TB using a transform type including a unidirectional transform skip, either the transform coefficient in the upper row (if a horizontal transform skip is used) or the coefficient in the left column (if a vertical transform skip is used) is the DC coefficient. Therefore, in some examples, applying context modeling to entropy code the code of only the first transform coefficient may be suboptimal because the code values ​​exhibit strong row-wise correlation (in the case of a horizontal transform skip) or column-wise correlation (in the case of a vertical transform skip).

[0118] A 1D transform skip may refer to a transform skip in one dimension (e.g., the horizontal dimension or the vertical dimension), and thus a transform with a 1D transform skip (e.g., a primary transform, a combination of a primary and a secondary transform) is performed in one dimension of a block. A 1D transform may refer to a transform applied to only one dimension of a block, and the transform comprises a 1D transform skip. A 1D transform may refer to a 1D horizontal transform or a 1D vertical transform. A 1D horizontal transform may refer to a transform applied to only the horizontal dimension of a block, and the transform comprises a 1D transform skip in the vertical dimension. A 1D vertical transform may refer to a transform applied to only the vertical dimension of a block, and the transform comprises a 1D transform skip in the horizontal dimension.

[0119] A 2D transform skip may indicate a transform skip in two dimensions (e.g., horizontal and vertical dimensions), such that there is no transform for the block and the transform matrix is ​​an identity matrix. If there is no transform skip, a transform may be performed in two dimensions for the block.

[0120] According to aspects of the present disclosure, a context may be used to entropy code one or more sign values ​​of one or more transform coefficients in a block (e.g., TB, CB, PB, luma block, chroma block, luma TB, chroma TB, etc.) using a 1D transform. The block may be any appropriate block, such as a luma block, a chroma block, etc. The context used may be any appropriate context, such as sign information (e.g., sign values) of other transform coefficients in the block. In one example, the context may be used to entropy code one or more sign values ​​of one or more transform coefficients in the block using a transform type in set A of transforms. In one example, the context may be used to entropy code one or more sign values ​​of one or more transform coefficients in the block using a transform type in set B of transforms.

[0121] According to aspects of the present disclosure, coding information for a block (e.g., a TB) may be decoded from a coded video bitstream. The coding information may indicate a unidirectional transform skip for the block. A code value of a current transform coefficient in the TB may be coded (e.g., encoded or decoded) based on previously coded (e.g., encoded or decoded) code values ​​of previous transform coefficients. The current transform coefficient and the previous transform coefficient may be in one of the same row and the same column in the TB. The same row and the same column may be along one direction of the transform skip. Furthermore, the current transform coefficient in the TB may be determined based on coded (e.g., encoded or decoded) code values ​​of previous transform coefficients. In one example, the code value of the current transform coefficient in the TB is decoded based on previously decoded code values ​​of the previous transform coefficient and one or more previously decoded code values ​​of one or more transform coefficients in the same row and one of the same column.

[0122] In one example, the one direction is the horizontal direction. One of the same row and the same column is the same row along one direction of the transform skip. Therefore, the transform is a 1D vertical transform like the transform type in the transform set A. The code value of the current transform coefficient in the TB may be coded (e.g., coded or decoded) based on the previously coded (e.g., coded or decoded) code value of the previous transform coefficient in the same row along one direction of the transform skip. In one example, the same row is limited to one of a subset of rows in the TB.

[0123] In one embodiment, the context derivation process for entropy coding the sign value of a current transform coefficient in a block using a transform type in the set of transforms A is based on at least one previously coded sign value of at least one previous transform coefficient in the same row. In general, the context used may include sign information (e.g., at least one previously coded sign value) of other transform coefficients (e.g., at least one previous transform coefficient) in the same row.

[0124] 13B shows a TB (1301) transformed with a 1D transform skip in the vertical or horizontal dimension. Thus, in addition to the first transform coefficient (e.g., t11), the remaining transform coefficients in the first column (1321) or first row (1311) may be DC transform coefficients.

[0125] Referring to Figure 13B, TB (1301) is transformed with a transform type in transform set A, e.g., a 1D vertical transform with a 1D transform skip in the horizontal dimension. The first row (1311) includes DC transform coefficients t11-t14. In one example, the code value of one transform coefficient (e.g., t13) in the first row (1311) is entropy coded using a context derivation process based on one or more previously coded transform coefficients (e.g., t11 and / or t12) in the first row (1311). The above description may be appropriately adapted for coding the other transform coefficients in row (1311).

[0126] Rows 1312-1314 contain AC transform coefficients having AC spatial frequencies. The above description may be adapted to one or more of rows 1321-1324, as appropriate. In one example, the code value of one transform coefficient (e.g., t24) in the second row 1312 is entropy coded using a context derivation process based on one or more previously coded transform coefficients (e.g., t21, t22, and / or t24) in the second row 1312. In one example, a default context is used to code the first transform coefficient (e.g., t21) in a row (e.g., row 1312). Alternatively, the use of a context for entropy coding code values ​​is valid only for a subset of rows (e.g., row 1311, rows 1311-1312, etc.) in the block that use transform types in transform set A. In one example, the use of context to entropy code code values ​​is valid only for rows (1311)-(1312), and code values ​​for rows (1313)-(1314) may be coded directly without using a context model.

[0127] In one example, the one direction is the vertical direction. One of the same row and the same column is the same column along one direction of the transform skip. Therefore, the transform is a 1D horizontal transform like the transform type in the transform set B. The code value of a current transform coefficient in the TB may be coded (e.g., coded or decoded) based on a previously coded (e.g., coded or decoded) code value of a previous transform coefficient in the same column along one direction of the transform skip. In one example, the same column is limited to one of a subset of columns in the TB.

[0128] In one embodiment, the context derivation process for entropy coding the sign value of a current transform coefficient in a block using a transform type in the set of transforms B is based on at least one previously coded sign value of at least one previous transform coefficient in the same column. In general, the context used may include sign information (e.g., at least one previously coded sign value) of other transform coefficients (e.g., at least one previous transform coefficient) in the same column.

[0129] Referring back to FIG. 13B , the TB (1301) is transformed with a transform type in transform set B, e.g., a 1D horizontal transform with a 1D vertical transform skip. The first column (1321) includes DC transform coefficients t11, t21, t31, and t41. In one example, the code value of one transform coefficient (e.g., t31) in the first column (1321) is entropy coded using a context derivation process based on one or more previously coded transform coefficients (e.g., t11 and / or t21) in the first column (1321). The above description may be appropriately adapted for coding the other transform coefficients in the column (1321).

[0130] Columns 1322-1324 contain AC transform coefficients having AC spatial frequencies. The above description may be applied to one or more of columns 1322-1324, as appropriate. In one example, the code value of one transform coefficient (e.g., t42) in the second column 1322 is entropy coded using a context derivation process based on one or more previously coded transform coefficients (e.g., t12, t22, and / or t32) in the second column 1322. In one example, a default context is used to code the first transform coefficient (e.g., t12) in the column (e.g., column 1322). Alternatively, the use of a context for entropy coding code values ​​is valid only for a subset of columns in the block that use transform types in transform set B (e.g., column 1321, columns 1321-1322, etc.). In one example, the use of context to entropy code the code values ​​is valid only for columns (1321)-(1322), and the code values ​​for columns (1323)-(1324) may be coded directly without using a context model.

[0131] In one embodiment, the code value of a current transform coefficient in a TB (e.g., TB(1301) in FIG. 13) may be decoded based on a previously decoded code value of a previous transform coefficient (e.g., t12), a previously decoded code value of a DC transform coefficient in an upper-neighboring block of the TB, and a previously decoded code value of a DC transform coefficient in a left-neighboring block of the TB. The DC transform coefficient in the upper-neighboring block of the TB and the DC transform coefficient in the left-neighboring block of the TB have a DC spatial frequency.

[0132] In one example, the code value of the current transform coefficient in the TB may be decoded based on the previously decoded code value of the previous transform coefficient and a weighted average (e.g., as shown in Equation 3) of the previously decoded code value of the DC transform coefficient in the upper neighboring block of the TB and the previously decoded code value of the DC transform coefficient in the left neighboring block of the TB. The weighting may be based on the number of transform coefficients in the upper neighboring block and the left neighboring block that respectively border the transform coefficient in the TB. The position of the previous transform coefficient in the TB may be adjacent to the position of the current transform coefficient in the TB. The position of the previous transform coefficient in the TB may be adjacent to the position of the current transform coefficient in the TB along the scanning direction. The scanning direction may be based on the scanning order for coding the code values ​​of the transform coefficients in the TB.

[0133] A context derivation process such as that described in equations (3)-(4) may be refined for entropy coding sign values ​​of transform coefficients in a block using a transform type in one of transform sets A and B. In one embodiment, the context derivation process for entropy coding sign values ​​of transform coefficients in a block is based not only on one or more neighboring blocks (e.g., a weighted average of DC code values ​​of an upper neighboring block and a left neighboring block), but also on at least one previously coded sign value of at least one neighboring coefficient in the block (if available). In general, the context used for entropy coding sign values ​​of transform coefficients in a block may include sign information of other blocks (e.g., an upper neighboring block, a left neighboring block, etc.), sign information of other transform coefficients in the block, etc. In one example, the at least one neighboring coefficient depends on the scan order used to code the sign value of the block. The at least one neighboring coefficient may be a transform coefficient in the block that was coded before the transform coefficient according to the scan order.

[0134] Referring to FIG. 13B , TB (1301) is transformed by a 1D transform skip in the vertical or horizontal dimension. Therefore, in addition to the first transform coefficient t11, the remaining transform coefficients in the first column (1321) or the first row (1311) may be DC transform coefficients. A context derivation process for entropy coding the code value of one of the DC transform coefficients in TB (1301) may be based on other TBs, such as TBs (1302)-(1303), and at least one previously coded code value of at least one neighboring coefficient in TB (1301). In one example, at least one neighboring coefficient is in the same row or column as one of the DC transform coefficients. In one example, one of the at least one neighboring coefficients is in a different row and column from one of the DC transform coefficients.

[0135] In one example, TB(1301) uses a transform type in transform set A, and row (1311) includes DC transform coefficients t11-t14. The code value of one of DC transform coefficients t11-t14 (e.g., t13) may be based on the DC code value of TB(1302)-(1303), respectively, and the previously coded code value of adjacent coefficient t12 in TB(1301).

[0136] The code value of one of the AC transform coefficients in TB(1301) (e.g., t23) may be based on the DC code values ​​of TB(1302)-(1303) respectively and the previously coded code values ​​of adjacent coefficients in TB(1301) (e.g., t13, t22, etc.).

[0137] The above description may be appropriately adapted to the case where one or more of the neighboring blocks use 1D transform skip. In one embodiment, one of the neighboring blocks (e.g., TB(1303)) uses 1D horizontal skip, and therefore the first row includes DC transform coefficients l11-l12. The context derivation process for entropy coding the code values ​​of the transform coefficients in the block may be based on one or more DC code values ​​of l11-l12, the DC code values ​​of TB(1302), and at least one previously coded code value of at least one neighboring coefficient in TB(1301) (if available).

[0138] In one example, the TB uses 1D transform skip along the horizontal direction, and a flag indicating whether the code values ​​of the transform coefficients in the same row are the same may be decoded. In one example, the transform coefficients in the same row include a current transform coefficient and a previous transform coefficient, and as described above, the code value of the current transform coefficient in the TB may be coded (e.g., coded or decoded) based on the previously coded (e.g., encoded or decoded) code value of the previous transform coefficient.

[0139] In one embodiment, the sign control information, such as a flag, indicates whether the signs (e.g., all signs) of the transform coefficients (e.g., t11-t14) in a row (e.g., row (1311) of a block (e.g., TB (1301)) in FIG. 13B) are the same when the transform type used for the block is one of transform set A. In one example, the sign control information is used only if the number of non-zero transform coefficients in the row is greater than a threshold. In one example, the sign control information is used only if the block dimension (e.g., width) of the block is greater than or equal to a threshold. In one example, each row in the block uses separate sign control information. For example, a separate flag is used to indicate whether the signs (e.g., all signs) of the transform coefficients in each row of the block are the same.

[0140] In one example, the code control information is used (e.g., communicated) for only a subset of the rows in the block, and each piece of code control information is used to indicate whether the signs (e.g., all signs) of the transform coefficients in each row of the block are the same.

[0141] In one example, code control information is used (e.g., signaled) for every N consecutive transform coefficients in a row. In another example, code control information is used (e.g., signaled) for every N consecutive non-zero transform coefficients in a row. Example values ​​of N may include, but are not limited to, 4, 8, and 16.

[0142] In one example, if the code control information indicates that all codes in a row are the same, such as when the flag is true, then no further code information is coded for that row.

[0143] In one example, if the code control information indicates that all codes in a row are the same, such as when a flag is true, then further code control information, such as another flag, is conveyed to indicate the code value for that row.

[0144] In one example, if the code control information indicates that not all codes in the row are the same, such as when the flag is false, code information (eg, respective code values) is coded for each transform coefficient in the row.

[0145] In one example, when the sign control information indicates that not all signs in a row are the same, such as when the flag is false, and the sign information for all non-zero transform coefficients in a row (in scanning order) except for the last non-zero transform coefficient in the row is the same and is transmitted, the sign information for the last non-zero transform coefficient is not transmitted and is derived as the inverse of the sign values ​​of the preceding non-zero transform coefficients in the row. Referring to Figure 13B, in one example, t21-t24 in row (1312) are +4, +1, 0, and -2. The sign values ​​of all non-zero transform coefficients except for the last one (t24) in row (1312) are the same (positive) and are transmitted. The flag is false, indicating that not all sign values ​​of the non-zero transform coefficients in row (1312) are the same. Therefore, the sign information of the last non-zero transform coefficient (t24) may be derived as negative (e.g., the inverse of the sign values ​​of the preceding non-zero transform coefficients (t21 and t22) in the row).

[0146] In one embodiment, the sign control information, such as a flag, indicates whether the signs (e.g., all signs) of the transform coefficients (e.g., t11, t21, t31, and t41) in a column (e.g., column (1321) of a block (e.g., TB (1301)) in FIG. 13B) are the same when the transform type used for the block is one of transform set B. In one example, the sign control information is used only if the number of non-zero transform coefficients in the column is greater than a threshold. In one example, the sign control information is used only if the block dimension (e.g., height) of the block is greater than or equal to a threshold. In one example, each column in the block uses separate sign control information. For example, a separate flag is used to indicate whether the signs (e.g., all signs) of the transform coefficients in each column of the block are the same.

[0147] In one example, the code control information is used (e.g., communicated) for only a subset of columns within the block, and each piece of code control information is used to indicate whether the signs (e.g., all signs) of the transform coefficients in each column of the block are the same.

[0148] In one example, code control information is used (e.g., signaled) for every M consecutive transform coefficients in a column. In another example, code control information is used (e.g., signaled) for every M consecutive non-zero transform coefficients in a column. Example values ​​of M may include, but are not limited to, 4, 8, and 16.

[0149] In one example, if the code control information indicates that all codes in a column are the same, such as when the flag is true, then no further code information is coded for that column.

[0150] In one example, if the code control information indicates that all codes in a column are the same, such as when a flag is true, then further code control information, such as another flag, is conveyed to indicate the code value for that column.

[0151] In one example, if the code control information indicates that not all codes in the column are the same, such as when the flag is false, code information (eg, respective code values) is coded for each transform coefficient in the column.

[0152] In one example, when the sign control information indicates that not all signs in the column are the same, such as when the flag is false, and the sign information for all non-zero transform coefficients in the column (in scanning order) except for the last non-zero transform coefficient is the same and is transmitted, the sign information for the last non-zero transform coefficient is not transmitted and is derived as the inverse of the sign value of the preceding non-zero transform coefficient in the column. Referring to Figure 13B, in one example, t12, t22, t32, and t42 in the column (1322) are +4, +1, 0, and -2. The sign values ​​of all non-zero transform coefficients except for the last one (t42) in the column (1322) are the same (positive) and are transmitted. The flag is false, indicating that not all sign values ​​of the non-zero transform coefficients in the column (1322) are the same. Therefore, the sign information of the last non-zero transform coefficient (t42) may be derived as negative (e.g., the inverse of the sign value of the preceding non-zero transform coefficients (t12 and t22) in the column).

[0153] According to aspects of the present disclosure, the context used for entropy coding the sign values ​​of transform coefficients may depend on the primary transform type and / or the secondary transform kernel (or type). In one example, the context used for entropy coding the sign values ​​of a current block (e.g., TB) depends on whether 1D IDTX or 2D IDTX is used. For example, the transform for the current block is a 1D transform when 1D IDTX is used, and a context or context modeling may be used to entropy code the sign values ​​of multiple transform coefficients in the current block, as described above. For example, when 2D IDTX is used, no transform is applied to the current block. In some examples, the context may be used to entropy code the sign values ​​of multiple transform coefficients in the current block.

[0154] In one example, the context used to entropy code the code values ​​of the current block depends on whether a secondary transform is applied to the current block.

[0155] FIG. 14 shows a flowchart outlining a process (1400) according to one embodiment of the present disclosure. The process (1400) may be used in the reconstruction of a block (e.g., a TB). In various embodiments, the process (1400) is performed by a processing circuit, such as a processing circuit within the terminal devices (310), (320), (330), and (340), a processing circuit performing the functions of the video encoder (403), a processing circuit performing the functions of the video decoder (410), a processing circuit performing the functions of the video decoder (510), a processing circuit performing the functions of the video encoder (603), or the like. In some embodiments, the process (1400) is implemented with software instructions, and thus, the processing circuit performs the process (1400) when the processing circuit executes the software instructions. The process begins at (S1401) and proceeds to (S1410).

[0156] The coding information may indicate a transform skip in one direction (eg, horizontal or vertical) for the block.

[0157] In (S1420), the code value of a current transform coefficient in the block may be decoded based on a previously decoded code value of a previous transform coefficient. The current transform coefficient and the previous transform coefficient may be in one of the same row and the same column in the block. The one of the same row and the same column may be along one direction of the transform skip.

[0158] In one example, when the one direction is horizontal, one of the same row and the same column is the same row along one direction of the transform skip. The code value of the current transform coefficient in the block may be decoded based on the previously decoded code value of the previous transform coefficient in the same row along one direction of the transform skip. In one example, the same row is limited to one of a subset of rows in the block.

[0159] In one example, when the one direction is a vertical direction, one of the same row and the same column is the same column along one direction of the transform skip. The code value of the current transform coefficient in the block may be decoded based on the previously decoded code value of the previous transform coefficient in the same column along one direction of the transform skip. In one example, the same column is limited to one of a subset of columns in the block.

[0160] At step S1430, the current transform coefficient in the block may be determined based on the decoded code value of the current transform coefficient. The process 1400 then proceeds to step S1499, where it ends.

[0161] Process 1400 may be adapted as appropriate. Steps in process 1400 may be modified and / or omitted. Additional steps may be added. Any suitable order of performance may be used.

[0162] In one example, a flag or other code control information is decoded to indicate whether the code values ​​of the transform coefficients in the same row are the same. In one example, the transform coefficients in the same row include a current transform coefficient and a previous transform coefficient.

[0163] In one example, a flag or other code control information is decoded to indicate whether the code values ​​of the transform coefficients in the same column are the same. In one example, the transform coefficients in the same column include a current transform coefficient and a previous transform coefficient.

[0164] In one example, the code value of the current transform coefficient in the TB is decoded based on previously decoded code values ​​of previous transform coefficients and at least one previously decoded code value of a DC transform coefficient in another block (e.g., an adjacent block). For example, the code value of the current transform coefficient in the TB is decoded based on previously decoded code values ​​of previous transform coefficients, previously decoded code values ​​of a DC transform coefficient in an upper adjacent block of the TB, and previously decoded code values ​​of a DC transform coefficient in a left adjacent block of the TB. The DC transform coefficient in the upper adjacent block of the TB and the DC transform coefficient in the left adjacent block of the TB may have a DC spatial frequency.

[0165] The embodiments of the present disclosure may be applied to any suitable blocks, such as luma blocks and / or chroma blocks. The embodiments of the present disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0166] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 15 illustrates a computer system (1500) suitable for implementing certain embodiments of the disclosed subject matter.

[0167] Computer software may be encoded using any suitable machine code or computer language, which may undergo assembly, compilation, linking, or similar mechanisms to generate code including instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through interpretation, microcode execution, etc.

[0168] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0169] 15 for computer system (1500) are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement regarding any one or combination of components shown in the exemplary embodiment of computer system (1500).

[0170] The computer system (1500) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic pictures).

[0171] The input human interface devices may include one or more of a keyboard (1501), a mouse (1502), a trackpad (1503), a touch screen (1510), a data glove (not shown), a joystick (1505), a microphone (1506), a scanner (1507), and a camera (1508).

[0172] The computer system (1500) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1510), data gloves (not shown), or joystick (1505), although haptic feedback devices that do not function as input devices may also be present), audio output devices (speakers (1509), headphones (not shown), etc.), visual output devices (screens (1510), including CRT, LCD, plasma, and OLED screens, each of which may or may not have touchscreen input capability, each of which may or may not have haptic feedback capability, and some of which may be capable of outputting two-dimensional visual output or three- or more-dimensional output through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0173] The computer system (1500) may also include human-accessible storage devices and associated media such as optical media including CD / DVD ROM / RW (1520) with CD / DVD or similar media (1521), thumb drives (1522), removable hard drives or solid state drives (1523), legacy magnetic media such as tape and floppy disks (not shown), specialized ROM / ASIC / PLD-based devices such as security dongles (not shown), etc.

[0174] Additionally, those skilled in the art should understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other non-transitory signals.

[0175] The computer system 1500 may also include an interface 1554 to one or more communications networks 1555. The networks may be, for example, wireless, wired, or optical. The networks may be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include Ethernet, wireless LAN, cellular networks (including GSM, 3G, 4G, 5G, LTE, and the like), TV wired or wireless wide-area digital networks (including cable, satellite, and terrestrial broadcast TV), vehicular, and industrial networks (including CAN Bus), and the like. Particular networks typically require external network interface adapters (e.g., USB ports on the computer system 1500) attached to particular general-purpose data ports or peripheral buses 1549, while other network interface adapters are typically integrated into the core of the computer system 1500 by attaching them to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network to a smartphone computer system), as described below. Using any of these networks, computer system 1500 can communicate with other entities. Such communication may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a specific CANbus device), or bidirectional to other computer systems using, for example, local or wide-area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces.

[0176] The above-mentioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1540) of the computer system (1500).

[0177] The core (1540) may include one or more central processing units (CPUs) (1541), graphics processing units (GPUs) (1542), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1543), task-specific hardware accelerators (1544), graphics adapters (1550), etc. These devices may be connected through a system bus (1548), along with read-only memory (ROM) (1545), random access memory (1546), and internal mass storage (such as an internal non-user-accessible hard drive or SSD) (1547). In some computer systems, the system bus (1548) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1548) or through a peripheral bus (1549). In one example, the screen 1510 may be attached to a graphics adapter 1550. Peripheral bus architectures include PCI, USB, etc.

[0178] The CPU (1541), GPU (1542), FPGA (1543), and accelerator (1544) may execute specific instructions, which in combination may constitute the above-mentioned computer code. The computer code may be stored in ROM (1545) or RAM (1546). Temporary data may be stored in RAM (1546), while persistent data may be stored in, for example, an internal mass storage device (1547). A cache memory, which may be closely associated with one or more of the CPU (1541), GPU (1542), mass storage device (1547), ROM (1545), RAM (1546), etc., may be used to enable fast storage and retrieval in any of the memory devices.

[0179] The computer-readable medium may include computer code for performing various computer-implemented operations, and the medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0180] By way of example and not limitation, the architecture (1500), and in particular a computer system having a core (1540), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with such user-accessible mass storage devices, as well as specific storage of the core (1540) that is non-transitory in nature, such as the core's internal mass storage (1547) or ROM (1545). Software implementing various embodiments of the present disclosure may be stored on such devices and executed by the core (1540). The computer-readable media may include one or more memory devices or chips according to particular needs. The software may cause the core (1540), and in particular the processor (including a CPU, GPU, FPGA, etc.) therein, to perform particular processes or portions of particular processes described herein, including defining data structures stored in RAM (1546) and modifying such data structures according to software-defined operations. Additionally or alternatively, a computer system may provide functionality as a result of hardwired or otherwise embodied logic in circuitry (e.g., accelerator (1544)), which may operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure includes any appropriate combination of hardware and software.

[0181] [Appendix A: Abbreviations] JEM: joint exploration model VVC: versatile video coding BMS: benchmark set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplementary Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: Transform Unit PU: Prediction Unit CTU: Coding Tree Unit CTB: Coding Tree Block PB: Prediction Block HRD: Hypothetical Reference Decoder SNR: Signal Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: Cathode Ray Tube LCD: Liquid-Crystal Display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Areas SSD: solid-state drive IC: Integrated Circuit CU: Coding Unit While this disclosure describes several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be recognized that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. 1. A method for video decoding performed by a decoder, comprising: receiving coding information of a transform block (TB) from a coded video bitstream, the coding information indicating a transform skip in one of a horizontal direction and a vertical direction for the TB, and the coding information including a flag indicating whether code values ​​of transform coefficients in one of the same row and the same column are the same; determining a code value of the current transform coefficient based on a code value of a previous transform coefficient in one of the same row and the same column as the current transform coefficient and the flag; determining the current transform coefficient in the TB based on the code value of the current transform coefficient; Including, the code value of the current transform coefficient in the TB is determined based on the code value of the previous transform coefficient in the same row as the current transform coefficient if the transform skip is in the horizontal direction; The method, wherein the code value of the current transform coefficient in the TB is determined based on the code value of the previous transform coefficient in the same column as the current transform coefficient when the transform skip is in the vertical direction.

2. The method described in claim 1, wherein when the transformation skip is horizontal, the same row is limited to one of a subset of rows in the TB.

3. The method described in claim 1, wherein when the transformation skip is in the vertical direction, the same column is limited to one of a subset of columns in the TB.

4. A method as described in claim 1 or 2, wherein when the transform skip is horizontal, the code value of the current transform coefficient in the TB is determined based on the code value of the previous transform coefficient in the same row as the current transform coefficient and the flag.

5. A method as described in claim 1 or 3, wherein when the transform skip is in the vertical direction, the code value of the current transform coefficient in the TB is determined based on the code value of the previous transform coefficient in the same column as the current transform coefficient and the flag.

6. The step of determining the code value comprises:

2. The method of claim 1, further comprising a step of determining the code value of the current transform coefficient in the TB based on the code value of the previous transform coefficient, the code value of a DC transform coefficient in an upper adjacent block of the TB, and the code value of a DC transform coefficient in a left adjacent block of the TB, wherein the DC transform coefficient in the upper adjacent block of the TB and the DC transform coefficient in the left adjacent block of the TB have a DC spatial frequency.

7. The step of determining the code value comprises:

7. The method of claim 6, further comprising: determining the code value of the current transform coefficient in the TB based on the code value of the previous transform coefficient and a weighted average of (i) the code value of the DC transform coefficient in the upper adjacent block of the TB and (ii) the code value of the DC transform coefficient in the left adjacent block of the TB, wherein the weighting of the weighted average is based on the number of transform coefficients in the upper adjacent block and the left adjacent block that respectively border the transform coefficient in the TB, and the position of the previous transform coefficient in the TB is adjacent to the position of the current transform coefficient in the TB.

8. The method of claim 7, wherein the position of the previous transform coefficient in the TB is adjacent to the position of the current transform coefficient in the TB along a scanning direction, and the scanning direction is based on a scanning order for coding the code values ​​of the transform coefficients in the TB.

9. The step of determining the code value comprises:

2. The method of claim 1, further comprising determining the code value of the current transform coefficient in the TB based on the code value of the previous transform coefficient and one or more code values ​​of one or more transform coefficients in the same row and the same column, respectively.

10. 1. An apparatus for video decoding including a processing circuit, 10. Apparatus, wherein the processing circuitry is configured to perform the method of any one of claims 1 to 9.

11. A program causing at least one processor to carry out the method according to any one of claims 1 to 9.

Images