Video decoding method and video decoding apparatus

Abstract

Aspects of the disclosure provide methods and apparatuses for video encoding / decoding. In some examples, an apparatus includes processing circuitry configured to receive a coded video stream including a current block associated with a transform block, and determine whether transform skipping is used in the transform block. In response to determining that transform skipping is not used in the transform block, the processing circuitry is configured to perform a first transform coefficient sign prediction, wherein (i) a first sign of a first plurality of transform coefficients is predicted based on a first plurality of neighboring samples, and (ii) a second sign of a second plurality of transform coefficients is predicted based on a second plurality of neighboring samples. The processing circuitry is further configured to reconstruct the current block based on the first plurality of transform coefficients and the second plurality of transform coefficients.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 17 / 984,229, filed November 9, 2022, entitled "Prediction of Coefficient Sign with Transformation Skip," and to U.S. Provisional Application No. 63 / 315,348, filed March 1, 2022, also entitled "Prediction of Coefficient Sign with Transformation Skip." The entire contents of the earlier applications are incorporated herein by reference. Technical Field

[0003] This disclosure describes embodiments that are typically associated with video encoding and decoding. Background Technology

[0004] The background description provided in this disclosure is intended to present the overall context of this disclosure. The work of the inventors currently named, within the scope of the work described in this background section and in aspects of the specification that may not be prior art at the time of filing, is neither explicitly nor implicitly acknowledged as prior art to this disclosure.

[0005] Video encoding and decoding can be performed using inter-frame prediction with motion compensation. Uncompressed digital video can comprise a series of pictures, each with spatial dimensions, for example, 1920×1080 luma samples and associated chroma samples. This series of pictures can have a fixed or variable picture rate (informally also called frame rate), such as 60 pictures per second or 60Hz. Uncompressed video has very high bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (at a frame rate of 60Hz and a luma sample resolution of 1920×1080) requires close to 1.5 Gbit / s of bandwidth. One hour of such video would require more than 600 GB of storage space.

[0006] One objective of video encoding and decoding is to reduce redundancy in the input video signal through compression. Compression helps reduce the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. Lossless compression, lossy compression, and combinations thereof can be employed. Lossless compression refers to a technique that reconstructs an exact copy of the original signal from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough that the reconstructed signal is useful for the intended application. In the case of video, lossy compression is widely used. The amount of distortion tolerated depends on the application. For example, users of some consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio can be reflected in the fact that higher allowable / tolerable distortion results in a higher compression ratio.

[0007] Video encoders and decoders can utilize techniques from several major categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0008] Video codec techniques can include those called intra-frame coding. In intra-frame coding, sample values ​​are represented without referencing samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into sample blocks. When all sample blocks are encoded in intra-frame mode, the picture can be an intra-frame picture. Intra-frame pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and thus can be used as the first picture in the encoded video stream and video session, or as still images. Samples of an intra-frame block can be exposed to a transform, and the transform coefficients can be quantized before entropy coding. Intra-frame prediction can be a technique that minimizes the sample values ​​in the pre-transform domain. In some cases, the smaller the transformed DC value, the smaller the AC coefficients, and the fewer bits are needed to represent the entropy-coded block at a given quantization step size.

[0009] Traditional intra-frame coding techniques, such as MPEG-2 generative coding, do not use intra-frame prediction. MPEG stands for Moving Picture Experts Group. However, some newer video compression techniques include attempts to use surrounding sample data and / or metadata obtained during the encoding and / or decoding of blocks that are spatially adjacent to the data and precede it in decoding order. This technique is hereby referred to as "intra-frame prediction." It is important to note that, at least in some cases, intra-frame prediction uses only reference data from the current picture being reconstructed, and not reference data from a reference picture.

[0010] There can be many different forms of intra-frame prediction. When more than one such technique can be used in a given video coding technique, the technique in use can be encoded in an intra-frame prediction mode. In some cases, a mode can have sub-modes and / or parameters, and these sub-modes and / or parameters can be encoded separately or included in the mode codeword. Which codeword is used for a given combination of modes, sub-modes, and / or parameters can affect the coding efficiency gain through intra-frame prediction, and thus affect the entropy encoding / decoding technique used to translate the codeword into a bitstream.

[0011] H.264 introduced specific intra-frame prediction modes, which were improved in H.265 and further refined in newer coding techniques such as the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Sets (BMS). Predicted value blocks can be formed using neighboring sample values ​​belonging to already available samples. Sample values ​​from neighboring samples are copied into the predicted value blocks based on orientation. References to the orientation used in the bitstream can be encoded, or the orientation itself can be predicted. Summary of the Invention

[0012] This disclosure provides methods and apparatus for video encoding / decoding.

[0013] Some exemplary aspects provide prediction of the transform coefficient sign when transform skipping is applied in one or two dimensions.

[0014] In some examples, the apparatus for video encoding / decoding includes processing circuitry. This processing circuitry is configured to receive an encoded video stream including a current block associated with a transform block and determine whether transform skipping was used in the transform block. In response to determining that no transform skipping was used in the transform block, the processing circuitry is configured to perform a first transform coefficient symbol prediction, wherein (i) a first symbol of a first plurality of transform coefficients is predicted based on a first plurality of neighboring samples, and (ii) a second symbol of a second plurality of transform coefficients is predicted based on a second plurality of neighboring samples. The processing circuitry is further configured to reconstruct the current block based on the first plurality of transform coefficients and the second plurality of transform coefficients.

[0015] In some aspects, the processing circuitry is configured to perform a second transform coefficient sign prediction on the transform block in response to determining that a transform skip was used in the transform block. The second transform coefficient sign prediction is different from the first transform coefficient sign prediction.

[0016] In some aspects, in response to the use of transform skipping in the transform block, the first transform coefficient sign prediction is disabled.

[0017] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip has been used. In response to the use of a one-dimensional horizontal transform skip in the transform block, the signs of coefficients in different columns of the transform block are predicted independently via a second transform coefficient sign prediction. The processing circuitry is configured to predict the coefficient signs of the column using only reconstructed samples adjacent to the column.

[0018] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip is used. In response to the use of a one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in the first column of the transform block is predicted by a second transform coefficient sign prediction based on one or more reconstructed columns to the left of the transform block and one or more reconstructed pixels above the first column of the transform block.

[0019] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip is used. In response to the use of a one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in a column other than the first column of the transform block is predicted by second transform coefficient sign prediction using one or more reconstructed pixels in one or more rows above the transform block.

[0020] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip has been used. In response to the use of a one-dimensional vertical transform skip in the transform block, the signs of coefficients in different rows of the transform block are predicted independently via a second transform coefficient sign prediction. The processing circuitry is configured to predict the coefficient signs of the row using only reconstructed samples adjacent to the row.

[0021] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip is used. In response to the use of a one-dimensional vertical transform skip in the transform block, the sign of the coefficients in the first row of the transform block is predicted by a second transform coefficient sign prediction based on one or more reconstructed rows above the transform block and one or more reconstructed pixels to the left of the transform block.

[0022] In some aspects, the processing circuitry is configured to determine whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip is used. In response to the use of a one-dimensional vertical transform skip in the transform block, the sign of the coefficients in a row other than the first row of the transform block is predicted using one or more reconstructed pixels from one or more columns to the left of the transform block via a second transform coefficient sign prediction.

[0023] In some aspects, in response to the use of one-dimensional transform skipping in the transform block, the prediction of the first transform coefficient sign is disabled.

[0024] In some respects, the first total number of symbols predicted by the second transform coefficient symbol prediction when a one-dimensional transform skip is applied in a transform block is different from the second total number of symbols predicted by the first transform coefficient symbol prediction when no transform skip is applied in any direction in a transform block.

[0025] In some aspects, the processing circuitry is configured to determine whether a two-dimensional transform skip has been used. In response to the use of a two-dimensional transform skip in a transform block, the processing circuitry is configured to predict coefficient signs only for pixels in the first row or first column of the transform block, using one or more reconstructed pixels in one or more rows or columns adjacent to the transform block.

[0026] In some aspects, the processing circuitry is configured to determine whether forward skip coding (FSC) has been used. In response to the use of FSC in the transform block, the processing circuitry is configured to predict coefficient signs only for pixels in the first row or first column of the transform block, using one or more reconstructed pixels in one or more rows or columns adjacent to the transform block.

[0027] In some aspects, the processing circuitry is configured to determine whether a two-dimensional transform skip has been used. In response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the top-left pixel of the transform block is predicted by a second transform coefficient sign prediction based on: one or more reconstructed pixels above the top-left pixel in one or more rows above the transform block; and one or more reconstructed pixels to the left of the top-left pixel in one or more columns to the left of the transform block.

[0028] In some aspects, the processing circuitry is configured to determine whether a two-dimensional transform skip was used. In response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels of the first row of the transform block is predicted by second transform coefficient sign prediction based on: one or more first reconstructed pixels in one or more rows above the transform block and the pixels; and one or more second reconstructed pixels to the left or right of one or more first reconstructed pixels in one or more rows.

[0029] In some aspects, the processing circuitry is configured to determine whether a two-dimensional transform skip has been used. In response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels of the first column of the transform block is predicted by second transform coefficient sign prediction based on: one or more first reconstructed pixels in one or more columns to the left of the transform block and the pixels; and one or more second reconstructed pixels above or below one or more first reconstructed pixels in one or more columns.

[0030] In some aspects, in response to the use of two-dimensional transform skipping in the transform block, first transform coefficient sign prediction is not enabled.

[0031] In some aspects, first transform coefficient symbol prediction is not enabled for intra-coded blocks that use the forward skip coding (FSC) mode.

[0032] In some aspects, first transform coefficient symbol prediction is enabled for inter-frame coded blocks based on the fact that the inter-frame coded blocks do not use forward skip coding (FSC) mode.

[0033] In some examples, the apparatus for video encoding / decoding includes processing circuitry. The processing circuitry is configured to determine whether a transform skip was used in a transform block. The processing circuitry is further configured to perform transform coefficient sign prediction based on the determination that a transform skip was used in the transform block. The processing circuitry is further configured to determine transform coefficients based on the determination that a transform skip was used in the transform block. The processing circuitry is further configured to reconstruct the current block based on the determined transform coefficients. Transform coefficient sign prediction is a first transform coefficient sign prediction performed on the transform block in response to determining that no transform skip was used in the transform block.

[0034] In some aspects, transform coefficient sign prediction is a second transform coefficient sign prediction performed on a transform block in response to determining that a transform skip was used in the transform block, the second transform coefficient sign prediction being different from the first transform coefficient sign prediction.

[0035] In some aspects, in response to the use of transform skipping in the transform block, the first transform coefficient sign prediction is disabled.

[0036] In some aspects, in response to the use of a one-dimensional horizontal transformation skip in the transform block, the signs of the coefficients in different columns of the transform block are predicted independently by a second transform coefficient sign prediction, and performing the second transform coefficient sign prediction includes: predicting the signs of the coefficients in that column using only the reconstructed samples adjacent to the column.

[0037] In some aspects, in response to the use of a one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in the first column of the transform block is predicted by second transform coefficient sign prediction based on one or more reconstructed columns to the left of the transform block and one or more reconstructed pixels above the first column of the transform block.

[0038] In some aspects, in response to the use of a one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in a column other than the first column of the transform block is predicted by the second transform coefficient sign prediction using one or more reconstructed pixels in one or more rows above the transform block.

[0039] In some aspects, in response to the use of a one-dimensional vertical transform skip in the transform block, the signs of coefficients in different rows of the transform block are predicted independently by a second transform coefficient sign prediction, and performing the second transform coefficient sign prediction includes predicting the coefficient signs of the row using only the reconstructed samples adjacent to the row.

[0040] In some aspects, in response to the use of a one-dimensional vertical transform skip in the transform block, the sign of the coefficients in the first row of the transform block is predicted by second transform coefficient sign prediction based on one or more reconstructed rows above the transform block and one or more reconstructed pixels to the left of the transform block.

[0041] In some aspects, in response to the use of a one-dimensional vertical transform skip in the transform block, the sign of the coefficients in a row other than the first row of the transform block is predicted by using one or more reconstructed pixels from one or more columns to the left of the transform block, through second transform coefficient sign prediction.

[0042] In some aspects, in response to the use of one-dimensional transform skipping in the transform block, transform coefficient sign prediction is disabled.

[0043] In some respects, the first total number of symbols predicted by the second transform coefficient symbol prediction when a one-dimensional transform skip is applied in a transform block is different from the second total number of symbols predicted by the first transform coefficient symbol prediction when no transform skip is applied in any direction in a transform block.

[0044] In some aspects, in response to the use of two-dimensional transform skip or forward skip coding (FSC) in the transform block, performing second transform coefficient symbol prediction includes: using one or more reconstructed pixels in one or more rows or columns adjacent to the transform block, predicting coefficient symbols only for pixels in the first row or first column of the transform block.

[0045] In some aspects, in response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the top-left pixel of the transform block is predicted by second transform coefficient sign prediction based on the following: one or more reconstructed pixels above the top-left pixel in one or more rows above the transform block, and one or more reconstructed pixels to the left of the top-left pixel in one or more columns to the left of the transform block.

[0046] In some aspects, in response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels in the first row of the transform block is predicted by second transform coefficient sign prediction based on the following: one or more first reconstructed pixels in one or more rows above the transform block and the pixels, and one or more second reconstructed pixels to the left or right of one or more first reconstructed pixels in one or more rows.

[0047] In some aspects, in response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels in the first column of the transform block is predicted by second transform coefficient sign prediction based on the following: one or more first reconstructed pixels in one or more columns to the left of the transform block and the pixels, and one or more second reconstructed pixels above or below one or more first reconstructed pixels in one or more columns.

[0048] In some aspects, in response to the use of two-dimensional transform skipping in the transform block, transform coefficient sign prediction is not enabled.

[0049] In some aspects, transform coefficient symbol prediction is not enabled for intra-coded blocks that use the forward skip coding (FSC) mode.

[0050] In some aspects, transform coefficient symbol prediction is not enabled for inter-coded blocks that use the forward skip coding (FSC) mode.

[0051] In some aspects, transform coefficient symbol prediction is enabled for inter-coded blocks that do not use the forward skip coding (FSC) mode.

[0052] This disclosure also provides a non-transitory computer-readable medium storing instructions that, when executed by a computer for video encoding / decoding, cause the computer to perform any method for video encoding / decoding. Attached Figure Description

[0053] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0054] Figure 1 This is a simplified block diagram of a communication system according to an embodiment.

[0055] Figure 2 This is a simplified block diagram of a communication system according to an embodiment.

[0056] Figure 3 This is a simplified block diagram of the decoder according to an embodiment.

[0057] Figure 4 This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0058] Figure 5 A block diagram of an encoder according to another embodiment is shown.

[0059] Figure 6 A block diagram of a decoder according to another embodiment is shown.

[0060] Figure 7A An exemplary coding tree structure according to an embodiment of this disclosure is shown.

[0061] Figure 7B An exemplary coding tree structure according to an embodiment of this disclosure is shown.

[0062] Figure 7C An exemplary multi-type tree splitting pattern according to an embodiment of this disclosure is shown.

[0063] Figure 7D An exemplary quadtree with a nested multi-type tree coding block structure is shown according to an embodiment of the present disclosure.

[0064] Figure 8 An example of a line graph transform (LGF) according to an embodiment is shown.

[0065] Figure 9 An example of the transform block and the pixel positions in its adjacent top two rows and left two columns, according to an embodiment, is shown.

[0066] Figure 10A A flowchart illustrating an example of an overview process according to some embodiments of this disclosure is shown.

[0067] Figure 10B A flowchart illustrating an example of an overview process according to some embodiments of this disclosure is shown.

[0068] Figure 11 This is a schematic diagram of a computer system according to an embodiment. Detailed Implementation

[0069] Figure 1 This is a simplified block diagram of a communication system (100) according to an embodiment of this application. The communication system (100) includes a plurality of terminal devices that can communicate with each other via, for example, a network (150). For example, the communication system (100) includes a first terminal device (310) and a second terminal device (320) interconnected via a network (150). Figure 1 In this embodiment, the first terminal device (110) and the second terminal device (120) perform unidirectional data transmission. For example, the first terminal device (110) may encode video data (e.g., a video image stream captured by the terminal device (110)) for transmission over a network (150) to the second terminal device (120). The encoded video data is transmitted in the form of one or more encoded video streams. The second terminal device (120) may receive the encoded video data from the network (150), decode the encoded video data to recover the video data, and display video images based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0070] In another embodiment, the communication system (100) includes a third terminal device (130) and a fourth terminal device (140) for performing bidirectional transmission of encoded video data, which may be performed, for example, during a video conference. For bidirectional data transmission, each of the third terminal device (130) and the fourth terminal device (140) may encode video data (e.g., a video image stream captured by the terminal device) for transmission over a network (150) to the other terminal device. Each of the third terminal device (130) and the fourth terminal device (140) may also receive encoded video data transmitted by the other terminal device and may decode the encoded video data to recover the video data, and may display the video images on an accessible display device based on the recovered video data.

[0071] exist Figure 1 In the embodiments disclosed herein, the first terminal device (110), the second terminal device (120), the third terminal device (130), and the fourth terminal device (140) may be servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (150) refers to any number of networks that transmit encoded video data between the first terminal device (110), the second terminal device (120), the third terminal device (130), and the fourth terminal device (140), including, for example, wired (connected) and / or wireless communication networks. The communication network (150) may exchange data in circuit-switched and / or packet-switched channels. The network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network (150) may be irrelevant to the operation of this application.

[0072] As an example, Figure 2 The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media such as compact disks (CDs), digital video discs (DVDs), and memory sticks.

[0073] The streaming system may include an acquisition subsystem (213) that may include a video source (201) such as a digital camera, which creates an uncompressed video image stream (202). In an embodiment, the video image stream (202) includes samples captured by a digital camera. The video image stream (202) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (204) (or encoded video bitstream). The video image stream (202) may be processed by an electronic device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (202), the encoded video data (204) (or the encoded video bitstream (204)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (204) (or the encoded video bitstream (204)), which can be stored on a streaming server (205) for future use. One or more streaming client subsystems, such as Figure 2 Client subsystems (206) and (208) can access a streaming server (205) to retrieve copies (207) and (209) of encoded video data (204). Client subsystem (206) may include, for example, a video decoder (210) in an electronic device (230). The video decoder (210) decodes the incoming copy (207) of the encoded video data and produces an output video picture stream (211) that can be displayed on a display (212) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (204), video data (207), and video data (209) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0074] It should be noted that the electronic devices (220) and (230) may include other components (not shown). For example, the electronic device (220) may include a video decoder (not shown), and the electronic device (230) may also include a video encoder (not shown).

[0075] Figure 3This is a block diagram of a video decoder (310) according to an embodiment disclosed in this application. The video decoder (310) may be disposed in an electronic device (330). The electronic device (330) may include a receiver (331) (e.g., receiving circuitry). The video decoder (310) may be used in place of... Figure 2 The video decoder (210) in the embodiment.

[0076] The receiver (331) may receive one or more encoded video sequences to be decoded by the video decoder (310); in the same embodiment or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the other encoded video sequences. The encoded video sequences may be received from a channel (301), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (331) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (331) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (315) may be coupled between the receiver (331) and the entropy decoder / parser (320) (hereinafter referred to as "parser (320)"). In some applications, the buffer memory (315) is part of the video decoder (310). In other cases, the buffer memory (315) may be located external to the video decoder (310) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (310) to prevent network jitter, for example, and another buffer (315) may be configured internally for, for example, handling broadcast timing. When the receiver (331) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (315) may not be necessary, or the buffer may be made smaller. Of course, a buffer (315) may also be required for use on packet networks such as the Internet. This buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (310).

[0077] The video decoder (310) may include a parser (320) to reconstruct symbols (321) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (310) and potential information for controlling a display device (312) (e.g., a display screen), which is not part of the electronic device (330) but may be coupled to it, such as... Figure 3As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (320) may parse / decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include a Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (320) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0078] The parser (320) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (315) to create symbols (321).

[0079] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (321) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (320). For brevity, the flow of such subgroup control information between the parser (320) and the various units described below is not described.

[0080] In addition to the functional blocks already mentioned, the video decoder (310) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0081] The first unit is the scaler / inverse transform unit (351). The scaler / inverse transform unit (351) receives quantization transform coefficients as symbols (321) and control information from the parser (320), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (351) can output a block containing sample values, which can be input into the aggregator (355).

[0082] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) uses reconstructed information extracted from the current picture buffer (358) to generate surrounding blocks of the same size and shape as the block being reconstructed. For example, the current picture buffer (358) buffers partially reconstructed and / or fully reconstructed current images. In some cases, the aggregator (355) adds the predictive information generated by the intra-picture prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) based on each sample.

[0083] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (353) can access the reference image memory (357) to extract samples for prediction. After motion compensation of the extracted samples according to symbols (321), these samples can be added by the aggregator (355) to the output of the scaler / inverse transform unit (351) (referred to as residual samples or residual signals in this case) to generate output sample information. The motion compensation prediction unit (353) can obtain the predicted samples from the address in the reference image memory (357) under motion vector control, and the motion vector is available to the motion compensation prediction unit (353) in the form of symbols (421), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (357) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0084] The output samples of the aggregator (355) can be employed by various loop filtering techniques in the loop filter unit (356). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (321) from the parser (320) in the loop filter unit (356). However, in other embodiments, the video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0085] The output of the loop filter unit (356) can be a sample stream, which can be output to a display device (312) and stored in a reference image memory (357) for subsequent inter-frame prediction.

[0086] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded images corresponding to the current image have been fully reconstructed and the encoded images (by, for example, the parser (320)) are identified as reference images, the current image buffer (358) can become part of the reference image memory (357), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0087] The video decoder (310) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under the configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0088] In this embodiment, the receiver (331) may receive supplemental (redundant) data along with the encoded video. The supplemental data may be a portion of the encoded video sequence. The supplemental data may be used by the video decoder (310) to properly decode the data and / or more accurately reconstruct the original video data. The supplemental data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0089] Figure 4 This is a block diagram of a video encoder (403) according to an embodiment disclosed in this application. The video encoder (403) is disposed in an electronic device (420). The electronic device (420) includes a transmitter (440) (e.g., a transmission circuit). The video encoder (403) can be used to replace... Figure 2 The video encoder (203) in the embodiment.

[0090] The video encoder (403) can obtain data from the video source (401) (not) Figure 4 In one embodiment, a portion of the electronic device (420) receives video samples, and the video source can capture video images to be encoded by the video encoder (403). In another embodiment, the video source (401) is a portion of the electronic device (420).

[0091] A video source (401) can provide a sequence of source video samples encoded by a video encoder (403) in the form of a digital video sample stream. The digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (401) can be a storage device storing previously prepared video. In a video conferencing system, the video source (401) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed sequentially. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0092] According to an embodiment, the video encoder (403) can encode and compress images of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller (450). In some embodiments, the controller (450) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (450) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (450) may be used with other suitable functions related to the video encoder (403) optimized for a particular system design.

[0093] In some embodiments, the video encoder (403) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (430) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (433) embedded within the video encoder (403). The decoder (433) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (because in the video compression techniques considered in this application, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (434). Since decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory (434) also correspond bit-precisely between the local encoder and the remote encoder. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" when using the prediction during decoding. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0094] The operation of the “local” decoder (433) can be combined with, for example, the above-mentioned... Figure 3 The video decoder (310) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 3 When symbols are available and the entropy encoder (445) and parser (320) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (310), including the buffer (315) and parser (320), may not be fully implemented in the local decoder (433).

[0095] It can be observed that any decoder technique other than parsing / entropy decoding present in the decoder must also exist in the corresponding encoder in essentially the same functional form. For this reason, this application focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0096] During operation, in some embodiments, the source encoder (430) may perform motion-compensated predictive coding. Motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (432) encodes the differences between pixel blocks in the input image and pixel blocks in the reference image, which may be selected as a predictive reference for the input image.

[0097] The local video decoder (433) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (430). The operation of the encoding engine (432) can be a lossy process. When the encoded video data can be decoded by the video decoder (433), Figure 4 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (433) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in a reference image cache (434). In this way, the video encoder (403) can locally store a copy of the reconstructed reference image that shares the same content as the reconstructed reference image that will be obtained by the remote video decoder (there are no transmission errors).

[0098] The predictor (435) can perform a prediction search against the encoding engine (432). That is, for a new image to be encoded, the predictor (435) can search in the reference image memory (434) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (435) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (435), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (434).

[0099] The controller (450) can manage the encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0100] The outputs of all the above functional units can be entropy encoded in the entropy encoder (445). The entropy encoder (445) performs lossless compression on the symbols generated by various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0101] The transmitter (440) can buffer the encoded video sequence created by the entropy encoder (445) in preparation for transmission via a communication channel (460), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (440) can combine the encoded video data from the video encoder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0102] The controller (450) manages the operation of the video encoder (403). During encoding, the controller (450) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0103] An intra-frame picture (I-frame) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are familiar with variations of I-frames and their corresponding applications and characteristics.

[0104] Predictive images (P-frames) can be images that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0105] Bidirectional predictive images (B-frames) can be images that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses up to two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0106] The source image is typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and each block is encoded sequentially. These blocks can be predictively coded with reference to other (already coded) blocks, and the other blocks are determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks in an I-frame can be non-predictively coded, or blocks can be predictively coded with reference to already coded blocks in the same image (spatial prediction or intra-frame prediction). Pixel blocks in a P-frame can be predictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks in a B-frame can be predictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0107] The video encoder (403) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (403) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0108] In this embodiment, the transmitter (440) may transmit additional data along with the encoded video while transmitting the encoded video. The source encoder (430) may transmit such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0109] The acquired video can be presented as multiple source images (video images) in a time-series format. Intra-frame prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is divided into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector may have a third dimension that identifies the reference image.

[0110] In some embodiments, bidirectional prediction techniques can be used in inter-frame prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, a block can be predicted using a combination of the first and second reference blocks.

[0111] In addition, merging mode techniques can be used in inter-frame prediction to improve coding efficiency.

[0112] According to some embodiments disclosed in this application, predictions such as inter-frame prediction and intra-frame prediction are performed on a block-by-block basis. For example, according to the High Efficiency Video Coding (HEVC) standard, images in a video image sequence are divided into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more Prediction Units (PUs). Typically, each PU includes a Luminance Prediction Block (PB) and two Chroma PBs. In embodiments, prediction operations during encoding (encoding / decoding) are performed on a block-by-block basis. Taking a Luminance Prediction Block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0113] Figure 5 This is a diagram of a video encoder (503) according to another embodiment disclosed in this application. The video encoder (503) is used to receive processing blocks (e.g., prediction blocks) of sample values ​​within the current video image in a video image sequence, and to encode the processing blocks into an encoded image that is part of an encoded video sequence. In this embodiment, the video encoder (503) is used instead of Figure 2 The video encoder (203) in the embodiment.

[0114] In the HEVC embodiment, the video encoder (503) receives a matrix of sample values ​​for a processing block, such as an 8×8 sample prediction block. The video encoder (503) uses, for example, rate-distortion (RD) optimization to determine whether to use intra-frame mode, inter-frame mode, or bidirectional prediction mode to encode the processing block. When encoding the processing block in intra-frame mode, the video encoder (503) can use intra-frame prediction techniques to encode the processing block into the encoded picture; and when encoding the processing block in inter-frame mode or bidirectional prediction mode, the video encoder (503) can use inter-frame prediction or bidirectional prediction techniques to encode the processing block into the encoded picture, respectively. In some video coding techniques, the merging mode can be an inter-frame prediction sub-mode, in which motion vectors are derived from one or more motion vector prediction values ​​without relying on encoded motion vector components outside the prediction values. In some other video coding techniques, motion vector components applicable to the subject block may exist. In the embodiment, the video encoder (503) includes other components, such as a mode decision module (not shown) for determining the processing block mode.

[0115] exist Figure 5 In one embodiment, the video encoder (503) includes, as shown below: Figure 5 The inter-frame encoder (530), intra-frame encoder (522), residual calculator (523), switch (526), ​​residual encoder (524), general controller (521) and entropy encoder (525) are shown coupled together.

[0116] The inter-frame encoder (530) is configured to receive samples of the current block (e.g., the processing block), compare the block with one or more reference blocks in a reference image (e.g., blocks in previous and later images), generate inter-frame prediction information (e.g., redundancy information description, motion vectors, and merging mode information based on the inter-frame prediction information), and calculate inter-frame prediction results (e.g., predicted blocks) based on the inter-frame prediction information using any suitable technique. In some embodiments, the reference image is a decoded reference image based on encoded video information.

[0117] The intra encoder (522) is used to receive samples of the current block (e.g., the processing block), compare the block with encoded blocks in the same image in some cases, generate quantization coefficients after transformation, and in some cases also (e.g., based on intra prediction direction information of one or more intra coding techniques) generate intra prediction information. In an embodiment, the intra encoder (522) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same image.

[0118] A general-purpose controller (521) is used to determine general-purpose control data and, based on the general-purpose control data, control other components of the video encoder (503). In an embodiment, the general-purpose controller (521) determines the mode of a block and, based on the mode, provides control signals to a switch (526). For example, when the mode is intra-frame mode, the general-purpose controller (521) controls the switch (526) to select an intra-frame mode result for use by the residual calculator (523) and controls the entropy encoder (525) to select intra-frame prediction information and add the intra-frame prediction information to the bitstream; and when the mode is inter-frame mode, the general-purpose controller (521) controls the switch (526) to select an inter-frame prediction result for use by the residual calculator (523) and controls the entropy encoder (525) to select inter-frame prediction information and add the inter-frame prediction information to the bitstream.

[0119] A residual calculator (523) is used to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (522) or the inter encoder (530). A residual encoder (524) is used to operate on the residual data to encode the residual data to generate transform coefficients. In an embodiment, the residual encoder (524) is used to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then quantized to obtain quantized transform coefficients. In various embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is used to perform an inverse transform and generate decoded residual data. The decoded residual data can be used appropriately by the intra encoder (522) and the inter encoder (530). For example, the inter encoder (530) can generate a decoded block based on the decoded residual data and inter-frame prediction information, and the intra encoder (522) can generate a decoded block based on the decoded residual data and intra-frame prediction information. The decoded blocks are processed appropriately to generate a decoded image, and in some embodiments, the decoded image may be buffered in a memory circuit (not shown) and used as a reference image.

[0120] An entropy encoder (525) is used to format the bitstream to produce encoded blocks. The entropy encoder (525) generates various information according to a suitable standard such as the HEVC standard. In an embodiment, the entropy encoder (525) is used to obtain general control data, selected prediction information (e.g., intra-frame prediction information or inter-frame prediction information), residual information, and other suitable information in the bitstream. It should be noted that, according to the disclosed subject matter, residual information is not present when blocks are encoded in a merged sub-mode of inter-frame mode or bidirectional prediction mode.

[0121] Figure 6This is a diagram of a video decoder (610) according to another embodiment disclosed in this application. The video decoder (610) is used to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In the embodiment, the video decoder (610) is used instead of Figure 2 The video decoder (210) in the embodiment.

[0122] exist Figure 6 In the embodiment, the video decoder (610) includes, as follows: Figure 6 The entropy decoder (671), inter-frame decoder (680), residual decoder (673), reconstruction module (674), and intra-frame decoder (672) are shown coupled together.

[0123] An entropy decoder (671) can be used to reconstruct certain symbols from an encoded image, representing the syntax elements constituting the encoded image. Such symbols may include, for example, modes used for encoding blocks (e.g., intra-frame mode, inter-frame mode, bidirectional prediction mode, a merged sub-mode of the latter two, or another sub-mode), prediction information (e.g., intra-frame prediction information or inter-frame prediction information) that can respectively identify certain samples or metadata used by the intra-frame decoder (672) or the inter-frame decoder (680) for prediction, residual information in the form of, for example, quantized transform coefficients, and so on. In an embodiment, when the prediction mode is inter-frame or bidirectional prediction mode, inter-frame prediction information is provided to the inter-frame decoder (680); and when the prediction type is intra-frame prediction type, intra-frame prediction information is provided to the intra-frame decoder (672). Residual information may be provided to the residual decoder (673) via inverse quantization.

[0124] The inter-frame decoder (680) is used to receive inter-frame prediction information and generate inter-frame prediction results based on the inter-frame prediction information.

[0125] The intra-frame decoder (672) is used to receive intra-frame prediction information and generate prediction results based on the intra-frame prediction information.

[0126] The residual decoder (673) performs inverse quantization to extract the dequantized transform coefficients and processes the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (673) may also require some control information (to obtain the quantizer parameters QP), which can be provided by the entropy decoder (671) (the data path is not indicated because this is only low-level control information).

[0127] The reconstruction module (674) combines the residual output by the residual decoder (673) with the prediction result (which may be output by the inter-frame prediction module or the intra-frame prediction module) in the spatial domain to form a reconstructed block, which may be a part of a reconstructed image, which in turn may be a part of a reconstructed video. It should be noted that other suitable operations, such as deblocking, may be performed to improve visual quality.

[0128] It should be noted that any suitable technology can be used to implement the video encoder (203), video encoder (403), and video encoder (503), as well as the video decoder (210), video decoder (310), and video decoder (610). In one embodiment, one or more integrated circuits (ICs) can be used to implement the video encoder (203), video encoder (403), and video encoder (503), as well as the video decoder (210), video decoder (310), and video decoder (610). In another embodiment, one or more processors executing software instructions can be used to implement the video encoder (203), video encoder (403), and video encoder (503), as well as the video decoder (210), video decoder (310), and video decoder (610).

[0129] This disclosure includes video coding methods applicable to various video coding technologies and / or standards, including but not limited to HEVC, VVC, and AV1. AV1 stands for AOMedia Video 1 (AV1). AOMedia stands for Alliance for Open Media (AOMedia).

[0130] like Figure 7A As shown in the first coding tree structure (700), VP9 uses a 4-way partitioning tree starting from 64×64 levels and going down to 4×4 levels, with some additional restrictions on 8×8 blocks. Figure 7A In this context, a partition designated as R can be called recursive because the same partition tree is repeated at lower scales until we reach the lowest 4×4 level.

[0131] like Figure 7B As shown in the second coding tree structure (702), AV1 not only extends the partition tree to a 10-way structure, but also increases the maximum size (referred to as the superblock in VP9 / AV1 terminology) from 128×128. It should be noted that this includes 4:1 or 1:4 rectangular partitions that are not present in VP9. Figure 7BThe partition type with 3 sub-partitions in the second row is called a "T-shaped" partition. No rectangular partition can be further subdivided. In addition to the coded block size, the coded tree depth is defined to indicate the split depth from the root feature. Specifically, for example, the coded tree depth of a 128×128 root node is set to 0, and the coded tree depth increases by 1 after the tree block is further split once.

[0132] Unlike VP9, ​​which enforces a fixed transform unit size, AV1 allows luma coding blocks to be divided into transform units of various sizes, which can be represented by recursive partitioning down to a maximum of two levels. To incorporate AV1's extended coding block partitioning, squares ranging from 4×4 to 64×64, and transform sizes of 2:1 or 1:2 and 4:1 or 1:4 are supported. For chroma blocks, only the largest possible transform unit is allowed.

[0133] A CTU can be split into multiple CUs using a quadtree structure represented as a coding tree to accommodate various local features, such as in HEVC. The decision of whether to use inter-frame (temporal) prediction or intra-frame (spatial) prediction to encode image regions is made at the CU level. Depending on the PU splitting type, each CU can be further split into one, two, or four PUs. Within a PU, the same prediction processing is applied, and relevant information is sent to the decoder based on the PU. After obtaining residual blocks by applying a prediction process based on the PU splitting type, the CUs can be divided into multiple TUs according to another quadtree structure similar to the CU coding tree. A key feature of the HEVC structure is that it has multiple partitioning concepts including CUs, PUs, and TUs. In HEVC, for inter-frame prediction blocks, CUs or TUs can only be squares, while PUs can be squares or rectangles. In HEVC, a coding block can be further split into four square sub-blocks, and a transformation is performed on each sub-block (i.e., TU). Each TU can be further recursively split (using quadtree splitting) into smaller TUs, which is called a Residual Quad-Tree (RQT).

[0134] At image boundaries, such as in HEVC, implicit quadtree splitting can be used so that blocks will maintain quadtree splitting until they are the right size for the image boundaries.

[0135] For example, in VVC, a quadtree with nested multi-type trees using binary and ternary splitting structures can replace the concept of multiple partitioning unit types. That is, it eliminates the separation of CU, PU, ​​and TU concepts, unless necessary for CUs that are too large for the maximum transform length, and supports greater flexibility in CU partitioning shapes. In the coding tree structure, CUs can be squares or rectangles. ACTUs are initially partitioned by a quadtree (also called a quadtree) structure. Quadtree leaf nodes can be further partitioned using multi-type tree structures. Figure 7C As shown in the third coding tree structure (704), the multi-type tree structure includes four split types. For example, the multi-type tree structure includes vertical binary split (SPLIT_BT_VER), horizontal binary split (SPLIT_BT_HOR), vertical ternary split (SPLIT_TT_VER), and horizontal ternary split (SPLIT_TT_HOR). The multi-type tree leaf nodes are called CUs, and unless the CU is too large for the maximum transform length, this split is used for prediction and transform processing without any further partitioning. This means that in most cases, in a quadtree with a nested multi-type tree coding block structure, the CU, PU, ​​and TU have the same block size. An anomaly occurs when the maximum supported transform length is less than the width or height of the color component of the CU. Figure 7D The image shows an example of block partitioning for a CTU, illustrating an exemplary quadtree with a nested multi-type tree-encoded block structure.

[0136] For example, in VVC, the maximum supported luminance transformation size can be 64×64, and the maximum supported chrominance transformation size can be 32×32. When the width or height of the CB is greater than the maximum transformation width or height, the CB is automatically split in the horizontal and / or vertical directions to meet the transformation size limit in that direction.

[0137] For example, in VTM7, the coding tree scheme supports the ability for luma and chroma to have separate block tree structures. In some cases, for P and B slices, the luma and chroma CTBs within a single CTU must share the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is partitioned into CUs via one coding tree structure, and the chroma CTB is partitioned into chroma CUs via another coding tree structure. This means that a CU in an I slice can include a coding block of one luma component or two chroma components, or consist of a coding block of one luma component or two chroma components, and a CU in a P or B slice can always include a coding block of all three chroma components, or consist of a coding block of all three chroma components, unless the video is monochrome.

[0138] To support extended coding block partitioning, such as in AV1, various transformation sizes (e.g., a range from 4 to 64 points per dimension) and transformation shapes (e.g., squares, rectangles with aspect ratios of 2:1 or 1:2, and rectangles with aspect ratios of 4:1 or 1:4) can be used.

[0139] Two-dimensional transformation processes can involve the use of hybrid transform kernels (e.g., consisting of different one-dimensional transforms for each dimension of the encoded residual block). The primary one-dimensional transforms can include at least one of the following: a) 4-point, 8-point, 16-point, 32-point, or 64-point DCT-2; b) 4-point, 8-point, or 16-point asymmetric DSTs (DST-4, DST-7) and their inverted versions; or c) 4-point, 8-point, 16-point, or 32-point identity transforms. Table 1 lists the basis functions for DCT-2 and asymmetric DSTs used, for example, in AV1, where “→” and “↓” indicate horizontal and vertical dimensions, and “√” and “×” indicate the availability of kernels for the block size and prediction mode. IDTX (or IDT) stands for identity transform.

[0140] Table 1. Exemplary AV1 main transformation basis functions (DCT-2, DST-4, and DST-7 for N-point inputs)

[0141]

[0142]

[0143] The availability of hybrid transform kernels can be based on transform block size and prediction mode. Table 2 lists exemplary dependencies.

[0144] Table 2. Availability of AV1 Hybrid Transform Kernels Based on Prediction Mode and Block Size

[0145]

[0146]

[0147] For chroma components, transform type selection is performed implicitly. For intra-frame prediction residuals, the transform type is selected based on the intra-frame prediction mode, as specified in Table 3. For inter-frame prediction residuals, the transform type can be selected based on the transform type selection of the co-located luma block. Therefore, for chroma components, transform type signaling in the bitstream is not required.

[0148] Table 3 Transform type selection for chroma component intra-frame prediction residuals

[0149] DC_PRED DCT DCT V_PRED ADST DCT H_PRED DCT ADST D45_PRED DCT DCT D135_PRED ADST ADST D113_PRED ADST DCT D157_PRED DCT ADST D203_PRED DCT ADST D67_PRED ADST DCT SMOOTH_PRED ADST ADST SMOOTH_V_PRED ADST DCT SMOOTH_H_PRED DCT ADST PAETH_PRED ADST ADST

[0150] Starting with the replacement and expansion (by introducing 32-point and 64-point) of the aforementioned one-dimensional DST, for example in AV2, line graph transformations (LGTs) have been introduced.

[0151] A graph is a general mathematical structure consisting of a set of vertices and edges, or a set of vertices and edges, used to model relationships between objects of interest. In practice, weighted graphs (where a set of weights is assigned to edges and possibly to vertices) provide a sparse representation for robust modeling of signals and / or data. LGTs can improve coding efficiency by providing better adaptation to different block statistics. Separable LGTs are designed and optimized by learning line graphs from data to model the basic row and column statistics of blocks in the residual signal, where the associated Generalized Graph Laplacian (GGL) matrix is ​​used to derive the LGT. Figure 8 It shows the self-circulating weight v c1 v c2 and edge weight w c An exemplary LGT (800) is characterized.

[0152] Given a weighted graph G(W, V), the GGL matrix can be defined as:

[0153] L c =D-W+V,

[0154] Where W is the weight of the non-negative edge w c Or determined by the non-negative edge weight w c The adjacency matrix is ​​formed by the following matrix: D is the angle matrix, and V represents the weighted self-loop v. c1 v c2 A diagonal matrix. Matrix L c It can be represented as:

[0155]

[0156] Then it can be accessed via GGL L c The LGT is derived from the eigenvalue decomposition:

[0157] L c =UΦU T

[0158] In this model, the columns of the orthogonal matrix U are the basis vectors of the LGT, and Φ is the diagonal eigenvalue matrix. In fact, the DCT and DST of DCT-2, DCT-8, and DST-7 are LGTs derived from certain forms of GGL, as shown below:

[0159] DCT-2 can be configured by setting v c1 =0 is used to derive this.

[0160] DST-7 can be configured via v c1 =w c To deduce this.

[0161] DCT-8 can be configured via v c2 =w c To deduce this.

[0162] DST-4 can be configured via v c1 =2w c To deduce this.

[0163] DCT-4 can be configured via v c2 =2w c To deduce this.

[0164] For example, the LGT in AV2 can be implemented as matrix multiplication. A 4p LGT kernel can be implemented using L... c Setting v in the middle c1 =2w c To export, which means it is a DST-4.8p LGT kernel that can be exported via L c Setting v in the middle c1 =1.5w c To export. 16pLGT, 32pLGT, and 64pLGT kernels can be exported via L... c Setting v in the middle c1 =w c This means it is exported as DST-7.

[0165] In the example of residual coding in AV1, for each transform unit, AV1 coefficient coding begins with a signal indicating a skipped symbol, followed by the transform kernel type and end-of-block (eob) position when the skipped symbol is zero. Each coefficient value is then mapped to multiple level maps and symbols.

[0166] After the eob position is encoded, the low-level and mid-level graphs are encoded in reverse scan order. The former indicates whether the coefficient amplitude is between 0 and 2, while the latter indicates whether the range is between 3 and 14. Next, following the forward scan order, the signs of the coefficients and the residuals of coefficients greater than 14 are encoded using Exp-Golomb codes.

[0167] Regarding the use of context modeling, lower-level graph coding combines transform magnitude and direction with information from up to five neighboring coefficients. On the other hand, mid-level graph coding follows a similar approach to lower-level graph coding, except that the number of neighboring coefficients is reduced to two. Without any context model, the residual level's Exp-Golomb codes and AC coefficients are symbolically encoded, while the DC coefficients are symbolically encoded using the DC symbols of their neighboring transform units.

[0168] In examples of residual coding used for transform skip, such as in VVC, CUs encoded in transform skip mode (TSM) can use a modified transform coding process. The improvements can be summarized as follows: (a) all sub-blocks and positions within sub-blocks are scanned in a forward scan order; (b) the position of the last valid coefficient is not indicated by a signal; (c) the syntax element `coded_sub_block_flag` is not encoded for the last sub-block; (d) context modeling of the syntax `sig_coeff_flag`, `abs_level_gt1`, and `par_level_flag` is modified; and (e) the symbol flags are context-coded based on their left and top adjacent values.

[0169] During the development of AV2, a new mode called Forward Skip Coding (FSC) was introduced to modify the transform coding process of IDTX (2D transform skip). The modifications introduced by FSC are functionally similar to the changes introduced into the VVC transform skip mode mentioned above, and can be summarized as follows: (a) scanning all coding blocks and positions within coding blocks in a forward scan order; (b) skipping the EOB syntax; (c) using reduced context for coefficient levels; and (d) context encoding of symbol flags based on the left, bottom, and bottom-left corners.

[0170] For intra-frame blocks, when FSC mode is selected, the transform type is not indicated by signaling for transform blocks. Instead, at the coded block level, transform type signaling is tied to the FSC mode. Inter-frame blocks do not indicate the FSC mode by signaling, but the FSC method is implicitly selected if the transform type is IDTX and the screen content flag is enabled.

[0171] For entropy coding of transform coefficients, coefficient symbols are typically encoded separately using a bypass mode and amplitude / level (the absolute value of the coefficient). Separate encoding means that each coefficient symbol may require one bit for encoding, which is expensive. To improve the efficiency of entropy coding of coefficient symbols, symbol prediction methods have been proposed, such as JVET-D0031 (JVET stands for Joint Video Exploration Team - Joint Video Expert Team). In one example, instead of using a signal to indicate the symbol value, the context is used to entropy-code a flag indicating whether the predicted symbol is the same as the actual symbol. Because larger level values ​​lead to more accurate predicted symbol values, the context value depends on the level of the coefficient (the absolute value of the coefficient).

[0172] First, a set of transformation coefficients is identified such that the associated sign needs to be predicted. Then, a set of hypotheses are created for predicting the sign values ​​of these coefficients. For example, for three coefficients, the number of hypotheses can be at most 8 (2^3). To predict the sign values, there is a cost value associated with each hypothesis, and the hypothesis with the minimum cost is used to specify the predicted sign values ​​of the coefficients covered by that hypothesis.

[0173] Figure 9 An example of pixel positions in the transform block (900) and its adjacent top two rows (902) and left two columns (904) is shown. The cost of each hypothesis is calculated as follows. After the normal reconstruction process (dequantization, inverse transform), a reconstructed block is generated associated with a given hypothesis (hypothesis reconstruction), and the boundary samples of the reconstructed block, i.e., p, are derived. 0,y and p x,0 For each reconstructed pixel p at the left boundary of the reconstructed block 0,y A simple linear prediction is performed using the two previously reconstructed adjacent pixels on the left to obtain its predicted pred. 0,y =(2p) -1,y -p -2,y The predicted and reconstructed pixel p 0,y The absolute difference between them is added to the cost of the hypothesis. A similar process is performed on the pixels in the top row of the reconstructed block, adding each predicted value... x,0 =(2p) x,-1 -p x,-2 ) and reconstructed pixel p x,0 The absolute differences are added together. Therefore, the cost calculation process for each coefficient sign prediction hypothesis is as follows:

[0174]

[0175] In some embodiments, when transform skipping (identity transform) is applied in one or two dimensions of a block, or if the FSC mode is used to encode the block, the reconstructed neighboring pixels and pixels in the current block may be unrelated. Therefore, it may be necessary to modify the method used for coefficient sign prediction to improve coding efficiency.

[0176] Embodiments of this disclosure include, for example, transform coefficient sign prediction when transform skipping is applied in one or two dimensions. This method can be used individually or in any combination in any order. Furthermore, each of the method (or embodiment), encoder, and decoder can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0177] In this disclosure, the term "block" can refer to coefficients in a transform block. The term "row" can refer to a row of coefficients in a transform block. The term "column" can refer to a column of coefficients in a transform block. Aspects of this disclosure can also be applied to other embodiments, for example, replacing IDTX with a transform skip, or vice versa.

[0178] The set of transformation types that have transformation skips in the horizontal direction can be called set A. In one example, set A includes exhaustively a combination of one-dimensional transformations, and in some cases, a combination of all one-dimensional transformations, where the transformation kernel is a matrix. Examples of one-dimensional transformations include, but are not limited to, DCT, ADST, FLIPADST, LGT, FLIPLGT, KLT, all trigonometric transformation types (DCT types 1–8, DST types 1–8) and their derivations in the vertical direction and transformation skips in the horizontal direction.

[0179] The set of transformation types that have transformation skips in the vertical direction can be called set B. In one example, set B includes exhaustively a combination of one-dimensional transformations, and in some cases, a combination of all one-dimensional transformations, where the transformation kernel is a matrix. Examples of one-dimensional transformations include, but are not limited to, DCT, ADST, FLIPADST, LGT, FLIPLGT, KLT, all trigonometric transformation types (DCT types 1–8, DST types 1–8) and their derivations in the horizontal direction and transformation skips in the vertical direction.

[0180] This method can be applied to luminance and / or chrominance blocks. The sign coefficients can be predicted differently based on whether an identity transformation / transformation skip is used in the transform block. In some examples, when a one-dimensional identity transformation / transformation skip is used in the transform block, the signs of N coefficients in different columns (e.g., in the case of horizontal transformation skip) or different rows (e.g., in the case of vertical transformation skip) are predicted differently compared to the sign prediction process when no identity transformation / transformation skip is applied.

[0181] In some exemplary embodiments, when a one-dimensional identity transformation / transformation skip is used in a transform block, the signs of N coefficients in different columns (e.g., in the case of horizontal transformation skip) or different rows (e.g., in the case of vertical transformation skip) are predicted independently using adjacent reconstructed samples.

[0182] In one embodiment, symbolic prediction is disabled when a one-dimensional identity transformation is applied.

[0183] In one embodiment, when transform skipping is applied only in the horizontal direction, the signs of N coefficients in the first column are predicted using reconstructed pixels from one or more columns to the left of the first column and one or more rows above the first column, for example, K pixels in column A to the left of the first column and row B at the top of the first column in the transform block. Exemplary values ​​for A include, but are not limited to, values ​​greater than or equal to 2. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for B include, but are not limited to, values ​​greater than or equal to 2.

[0184] For example, refer to Figure 9 When the transform skip is applied only in the horizontal direction, in order to predict the coefficient p in the first column of the transform block (900) 0,1 The symbol is used to represent the reconstructed pixels (pixels p) in the two columns to the left of the first column in the transform block. -1,0 to p -1,3 and p -2,0 to p -2,3 ) and directly in p 0,1 The two pixels p in the top two rows above 0,-1 and p 0,-2 .

[0185] In one embodiment, when transform skipping is applied only in the vertical direction, the signs of the N coefficients in the first row are predicted using reconstructed pixels from one or more rows above the top row of the transform block and one or more columns to the left of the first row, for example, K pixels in row A above the top row and column B to the left of the first row. Exemplary values ​​for A include, but are not limited to, values ​​greater than or equal to 2. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for B include, but are not limited to, values ​​greater than or equal to 2.

[0186] For example, refer to Figure 9 In order to predict the coefficient p in the first row of the transform block (900) 1,0 The symbol uses the reconstructed pixels (pixels p) from the top two rows of the first row. 0,-1 to p 3,-1 and p 0,-2 to p 3,-2 ) and two adjacent pixels p in the left two columns -2,0 and p -1,0 .

[0187] In one embodiment, when a transform skip is applied in the horizontal direction, the reconstructed pixels in the M rows above the top row of the transform block are used to predict the signs of N coefficients in each column except the first column. Exemplary values ​​for M include, but are not limited to, values ​​greater than or equal to 2.

[0188] In one embodiment, when a transform skip is applied in the vertical direction, the reconstructed pixels in the M columns to the left of the first column in the transform block are used to predict the signs of N coefficients in each row except the first row. Exemplary values ​​for M include, but are not limited to, values ​​greater than or equal to 2.

[0189] In one embodiment, the number of coefficient symbols predicted when a one-dimensional transform skip is applied differs from the number of coefficient symbols predicted when a transform skip is not applied in any direction. In one example, the number of coefficient symbols predicted when a one-dimensional transform skip is applied is less than the number of coefficient symbols predicted when a transform skip is not applied in any direction. In another example, the number of coefficient symbols predicted when a one-dimensional transform skip is applied may be greater than the number of coefficient symbols predicted when a transform skip is not applied in any direction.

[0190] In some embodiments, the coefficient sign of only N pixels in the first row / column of the transform block is predicted. The coefficient sign can be predicted based on reconstructed pixels from one or more columns to the left of the transform block and / or one or more rows above it. In one example, for certain modes, such as when using 2D transform skipping or if the FSC mode is used for encoding the block, the coefficient sign of only N pixels in the first row / column of the transform block is predicted. For example, the coefficient sign of only N pixels in the first row / column of the transform block can be predicted using K neighboring pixels in the top A row or left B column adjacent to the transform block. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for A include, but are not limited to, values ​​greater than or equal to 2. Exemplary values ​​for B include, but are not limited to, values ​​greater than or equal to 2.

[0191] In one embodiment, when a transform skip is applied in both directions, the coefficient sign of the top-left pixel can be predicted based on pixels in one or more rows above the top row of the transform block and / or pixels in one or more columns to the left of the first column of the transform block. In one example, the coefficient sign of the top-left pixel can be predicted using K neighboring pixels in M ​​rows above the top row and L neighboring pixels in N columns to the left of the first column. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for M include, but are not limited to, values ​​greater than or equal to 2. Exemplary values ​​for L include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for N include, but are not limited to, values ​​greater than or equal to 2.

[0192] For example, refer to Figure 9The transform block (900) in the middle uses two pixels from the top two rows respectively, i.e., p 0,-1 and p 0,-2 And two pixels from the left two columns, namely p -1,0 and p- 2,0 To predict pixel p 0,0 The coefficient sign.

[0193] In one embodiment, when transform skipping is applied in both directions, the coefficient sign of N pixels in the first row is predicted based on pixels in one or more rows above the top row of the transform block and / or pixels in one or more columns to the left of the first column of the transform block. In one example, the coefficient sign of N pixels in the first row is predicted using K neighboring pixels in M ​​rows above the top row of the transform block. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for M include, but are not limited to, values ​​greater than or equal to 2.

[0194] For example, refer to Figure 9 The transform block (900) in the middle uses two pixels from each of the top two rows, i.e., p 1,-1 p 1,-2 p 0,-1 and p 0,-2 To predict pixel p 1,0 The coefficient sign is used, and two pixels from each of the top two rows are used, i.e., p 2,-1 p 2,-2 p 3,-1 and p 3,-2 To predict pixel p 2,0 The coefficient sign. Alternatively, for example, p can be used. 1,-1 p 1,-2 p 3,-1 and p 3,-2 To predict pixel p 1,0 The coefficient sign. Alternatively, for example, p can be used. 2,-1 p 2,-2 p 1,-1 and p 1,-2 To predict pixel p 2,0 The coefficient sign.

[0195] In one embodiment, when a transform skip is applied in both directions, the coefficient sign of N pixels in the first column is predicted based on pixels in one or more rows above the top row of the transform block and / or pixels in one or more columns to the left of the first column of the transform block. In the example, K neighboring pixels in M ​​columns to the left of the first column in the transform block are used to predict the coefficient sign of the N pixels in the first column. Exemplary values ​​for K include, but are not limited to, values ​​greater than or equal to 1. Exemplary values ​​for M include, but are not limited to, values ​​greater than or equal to 2. For example, two pixels from each of the two left columns, i.e., p...-1,1 p -2,1 p -1,0 and p -2,0 To predict pixel p 0,1 The coefficient sign, and can use two pixels from each of the two columns on the left, i.e., p -1,2 p -2,2 p -1,3 and p -2,3 To predict pixel p 0,2 The coefficient sign.

[0196] In one embodiment, when skipping using a two-dimensional transformation, coefficient sign prediction is not applied at all.

[0197] In one embodiment, coefficient symbol prediction is not enabled for intra-coded blocks using FSC mode.

[0198] In one embodiment, coefficient symbol prediction is not enabled for inter-frame coded blocks using FSC mode.

[0199] In one embodiment, coefficient symbol prediction is enabled for inter-frame coded blocks that do not use FSC mode.

[0200] Some aspects include an apparatus comprising a memory storing instructions and a processor communicating with the memory, wherein, when the processor executes the instructions, the processor is configured to cause the apparatus to perform some or all of the steps described in this disclosure.

[0201] Some aspects include a non-transitory computer-readable storage medium storing computer-readable instructions, wherein, when executed by one or more processors, the computer-readable instructions are configured to cause one or more processors to perform some or all of the steps described in this disclosure.

[0202] Figure 10A A flowchart of an overview process (1000) according to an embodiment of the present disclosure is shown. The process (1000) can be used for the reconstruction of blocks, thereby generating predictive blocks for the blocks being reconstructed. In various embodiments, the process (1000) is executed by processing circuitry, such as processing circuitry in terminal devices (110), (120), (130), and (140), processing circuitry performing the functions of a video encoder (203), processing circuitry performing the functions of a video decoder (210), processing circuitry performing the functions of a video decoder (310), processing circuitry performing the functions of a video encoder (403), etc. In some embodiments, the process (1000) is implemented as software instructions, so that the processing circuitry executes the process (1000) when the software instructions are executed. The process begins at (S1001) and proceeds to (S1010).

[0203] At (S1010), method (1000) includes receiving an encoded video stream that includes a current block associated with a transform block.

[0204] At (S1020), method (1000) includes determining whether transformation skipping was used in the transformation block.

[0205] At (S1030), method (1000) includes performing a first transform coefficient sign prediction in response to determining that no transform skip was used in the transform block. The first transform coefficient sign prediction includes predicting a first sign of a first plurality of transform coefficients based on a first plurality of neighboring samples, and predicting a second sign of a second plurality of transform coefficients based on a second plurality of neighboring samples. The prediction of the first sign of the first plurality of transform coefficients is independent of the prediction of the second sign of the second plurality of transform coefficients. At (S1040), method (1000) includes reconstructing the current block based on the first plurality of transform coefficients and the second plurality of transform coefficients.

[0206] Figure 10B A flowchart of an overview process (1050) according to an embodiment of the present disclosure is shown. Process (1050) can be used for block reconstruction, thereby generating predictive blocks for the blocks being reconstructed. In various embodiments, process (1050) is executed by processing circuitry, such as processing circuitry in terminal devices (110), (120), (130), and (140), processing circuitry performing the functions of a video encoder (203), processing circuitry performing the functions of a video decoder (210), processing circuitry performing the functions of a video decoder (310), processing circuitry performing the functions of a video encoder (403), etc. In some embodiments, process (1050) is implemented as software instructions, so that when the processing circuitry executes the software instructions, the processing circuitry executes process (1050). The process begins at (S1051) and proceeds to (S1060).

[0207] At (S1060), method (1050) includes determining whether a transform skip was used in the transform block. For example, method (1050) may determine whether a one-dimensional horizontal transform skip, a one-dimensional vertical transform skip, a two-dimensional transform skip, or an FSC was used in the transform block.

[0208] At (S1070), method (1050) includes performing transform coefficient sign prediction based on determining whether a transform skip was used in the transform block. At (S1080), method (1050) includes determining transform coefficients based on determining whether a transform skip was used in the transform block. At (S1090), method (1050) includes reconstructing the current block based on the determined transform coefficients, wherein the transform coefficient sign prediction is a first transform coefficient sign prediction performed on the transform block in response to determining that no transform skip was used in the transform block.

[0209] For example, if no one-dimensional horizontal transformation skip, one-dimensional vertical transformation skip, two-dimensional transformation skip, or FSC is used in the transform block, then method (1050) can perform the first coefficient sign prediction as follows: First, a set of transform coefficients is identified such that the associated signs need to be predicted. Then, a set of hypotheses is created for the predicted sign values ​​of these coefficients. For example, for three coefficients, the number of hypotheses can be at most 8 (2^3). To predict sign values, there is a cost value associated with each hypothesis, and the hypothesis with the minimum cost is used to specify the predicted sign values ​​of the coefficients covered by the hypothesis. Figure 9 An example of pixel positions in the transform block (900) and its adjacent top two rows (902) and left two columns (904) is shown. The cost calculation for each hypothesis is as follows. After the normal reconstruction process (dequantization, inverse transform), a reconstructed block is generated associated with a given hypothesis (hypothesis reconstruction), and the boundary samples of the reconstructed block, i.e., p, are derived. 0,y and p x,0 For each reconstructed pixel p at the left boundary of the reconstructed block 0,y A simple linear prediction is performed using the two previously reconstructed adjacent pixels on the left to obtain its predicted pred. 0,y =(2p) -1,y -p -2,y The predicted and reconstructed pixel p 0,y The absolute difference between them is added to the cost of the hypothesis. A similar process is performed on the pixels in the top row of the reconstructed block, adding each predicted value... x,0 =(2p) x,-1 -p x,-2 ) and reconstructed pixel p x,0 The absolute differences are added together. Therefore, the cost of each symbolic prediction hypothesis is calculated as follows:

[0210]

[0211] In some optional aspects, transform coefficient sign prediction is a second transform coefficient sign prediction performed on the transform block in response to determining that transform skipping was used in the transform block, the second transform coefficient sign prediction being different from the first transform coefficient sign prediction. In some other optional aspects, the first transform coefficient sign prediction is disabled in response to transform skipping being used in the transform block.

[0212] For example, if a one-dimensional horizontal transform skip, a one-dimensional vertical transform skip, a two-dimensional transform skip, or FSC is used in the transform block, the method (1050) can skip the aforementioned first coefficient sign prediction for rows or columns of the transform block, or perform a second coefficient sign prediction different from the aforementioned first coefficient sign prediction. This is because when transform skip (identity transform) is applied in one or two dimensions of the block, or if the FSC mode is used to encode the block, the reconstructed neighboring pixels and pixels in the current block may be unrelated. Therefore, the method used for coefficient sign prediction can be modified to improve coding efficiency. For example, when a one-dimensional identity transform / transform skip is used in the transform block, the signs of N coefficients in different columns (in the case of horizontal transform skip) or different rows (in the case of vertical transform skip) are predicted differently compared to the sign prediction process when no identity transform / transform skip is applied. In one embodiment, sign prediction is disabled when a one-dimensional identity transform is applied.

[0213] In one embodiment, for example, in response to the use of a one-dimensional horizontal transform skip in the transform block, the signs of the coefficients in different columns of the transform block are predicted independently by a second transform coefficient sign prediction, and performing the second transform coefficient sign prediction includes predicting the signs of the column coefficients using only reconstructed samples of neighboring columns.

[0214] In one embodiment, for example, in response to the use of a one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in the first column of the transform block is predicted by second transform coefficient sign prediction based on: one or more reconstructed columns to the left of the transform block; and one or more reconstructed pixels above the first column of the transform block. For example, when transform skip is applied only in the horizontal direction, the signs of N coefficients in the first column are predicted using the reconstructed pixels in columns A (example values ​​of A include, but are not limited to, 2, 3, ...) to the left of the first column in the transform block and K (example values ​​of K include, but are not limited to, 1, 2, 3, ...) pixels in rows B (example values ​​of B include, but are not limited to, 2, 3, ...) at the top of the first column. For example, refer to Figure 9 When the transform skip is applied only in the horizontal direction, in order to predict the coefficient p in the first column of the transform block (900) 0,1 The symbol is used to represent the reconstructed pixels (pixels p) in the two columns to the left of the first column in the transform block. -1,0 to p -1,3 and p -2,0 to p -2,3 ) and directly in p 0,1 The two pixels p in the top two rows above 0,-1 and p 0,-2 .

[0215] In one embodiment, for example, in response to a one-dimensional horizontal transform skip used in a transform block, the sign of coefficients in a column other than the first column of the transform block is predicted by second transform coefficient sign prediction using one or more reconstructed pixels in one or more rows above the transform block. For example, when a transform skip is applied in the horizontal direction, the signs of N coefficients in each column other than the first column are predicted using reconstructed pixels in M ​​rows (example values ​​of M include, but are not limited to, 2, 3, ...) above the top row of the transform block.

[0216] In one embodiment, for example, in response to the use of a one-dimensional vertical transform skip in the transform block, the signs of coefficients in different rows of the transform block are predicted independently by a second transform coefficient sign prediction, and performing the second transform coefficient sign prediction includes predicting the coefficient signs of the rows using only reconstructed samples from neighboring rows.

[0217] In one embodiment, for example, in response to a one-dimensional vertical transform skip used in a transform block, the signs of the coefficients in the first row of the transform block are predicted by a second transform coefficient sign prediction based on one or more reconstructed rows above the transform block and one or more reconstructed pixels to the left of the transform block. For example, when a transform skip is applied only in the vertical direction, the signs of the N coefficients in the first row are predicted using the reconstructed pixels in rows A (example values ​​of A include, but are not limited to, 2, 3, ...) above the top row in the transform block and K (example values ​​of K include, but are not limited to, 1, 2, 3, ...) pixels in columns B (example values ​​of B include, but are not limited to, 2, 3, ...) to the left. For example, refer to Figure 9 In order to predict the coefficient p in the first row of the transform block (900) 1,0 The symbol uses the reconstructed pixels (pixels p) from the top two rows of the first row. 0,-1 to p 3,-1 and p 0,-2 to p 3,-2 ) and two adjacent pixels p in the left two columns -2,0 and p -1,0 .

[0218] In one embodiment, for example, in response to the use of a one-dimensional vertical transform skip in the transform block, the sign of the coefficients in a row of the transform block other than the first row is predicted by using one or more reconstructed pixels from one or more columns to the left of the transform block via second transform coefficient sign prediction. For example, when a transform skip is applied in the vertical direction, the signs of N coefficients in each row other than the first row are predicted using reconstructed pixels from M columns (example values ​​of M include, but are not limited to, 2, 3, ...) to the left of the first column in the transform block.

[0219] In one embodiment, for example, in response to the use of one-dimensional transform skipping in the transform block, transform coefficient sign prediction is disabled.

[0220] In one embodiment, for example, the first total number of symbols predicted by the second transform coefficient symbol when a one-dimensional transform skip is applied in the transform block is different from the second total number of symbols predicted by the first transform coefficient symbol when no transform skip is applied in any direction in the transform block.

[0221] In an embodiment, for example, in response to the use of 2D transform skip or forward skip coding (FSC) in a transform block, performing a second transform coefficient symbol prediction involves using one or more reconstructed pixels from one or more rows or columns adjacent to the transform block to predict coefficient symbols only for pixels in the first row or first column of the transform block. For example, when using 2D transform skip or if the FSC mode is used for the coding block, the symbols for only N pixels in the first row / column of the transform block are predicted using K (example values ​​of K, including but not limited to 2, 3, ...) adjacent pixels from the top A (example values ​​of A include but are not limited to 2, 3, ...) rows or left B (example values ​​of B include but are not limited to 2, 3, ...) columns adjacent to the transform block.

[0222] In one embodiment, for example, in response to the use of a two-dimensional transform skip in the transform block, the sign of the coefficients in the top-left pixel of the transform block is predicted by a second transform coefficient sign prediction based on: one or more reconstructed pixels above the top-left pixel in one or more rows above the transform block; and one or more reconstructed pixels to the left of the top-left pixel in one or more columns to the left of the transform block. For example, when transform skip is applied in both directions, the sign of the coefficients of the top-left pixel can be predicted using the M (example values ​​of M include, but are not limited to, 2, 3, ...) neighboring pixels in rows above the top row (K (example values ​​of K include, but are not limited to, 1, 2, 3, ...) and the L (example values ​​of L include, but are not limited to, 1, 2, 3, ...) neighboring pixels in columns N (example values ​​of N include, but are not limited to, 2, 3, ...) to the left of the first column. For example, refer to Figure 9 The transform block (900) in the middle uses two pixels from the top two rows respectively, i.e., p 0,-1 and p 0,-2 And two pixels from the left two columns, namely p -1,0 and p- 2,0 To predict pixel p 0,0 The coefficient sign.

[0223] In one embodiment, for example, in response to a two-dimensional transform skip used in a transform block, the sign of the coefficients in the pixels of the first row of the transform block is predicted by a second transform coefficient sign prediction based on: one or more first reconstructed pixels in one or more rows above the transform block and the pixels; and one or more second reconstructed pixels to the left or right of one or more first reconstructed pixels in one or more rows. For example, when a transform skip is applied in both directions, the sign of the coefficients of N pixels in the first row is predicted using M (example values ​​of M include, but are not limited to, 2, 3, ...) neighboring pixels in rows M (example values ​​of M include, but are not limited to, 2, 3, ...) above the top row of the transform block. For example, refer to Figure 9 The transform block (900) in the middle uses two pixels from each of the top two rows, i.e., p 1,-1 p 1,-2 p 0,-1 and p 0,-2 To predict pixel p 1,0 The coefficient sign is used, and two pixels from each of the top two rows are used, i.e., p 2,-1 p 2,-2 p 3,-1 and p 3,-2 To predict pixel p 2,0 The coefficient sign.

[0224] In one embodiment, for example, in response to a two-dimensional transform skip used in a transform block, the sign of the coefficients in the pixels of the first column of the transform block is predicted by a second transform coefficient sign prediction based on: one or more first reconstructed pixels in one or more columns to the left of the transform block and the pixels; and one or more second reconstructed pixels located above or below one or more first reconstructed pixels in one or more columns. For example, when a transform skip is applied in both directions, the sign of the coefficients of N pixels in the first column is predicted using M (example values ​​of M include, but are not limited to, 2, 3, ...) neighboring pixels in columns M (example values ​​of M include, but are not limited to, 2, 3, ...) to the left of the first column in the transform block.

[0225] In one embodiment, for example, in response to the use of two-dimensional transform skipping in the transform block, transform coefficient sign prediction is not enabled.

[0226] In one embodiment, for example, for an intra-coded block using the forward skip coding (FSC) mode, transform coefficient symbol prediction is not enabled.

[0227] In one embodiment, for example, for inter-frame coded blocks using the forward skip coding (FSC) mode, transform coefficient symbol prediction is not enabled.

[0228] In one embodiment, for example, coefficient symbol prediction is enabled for inter-coded blocks that do not use the forward skip coding (FSC) mode.

[0229] The process stops at (S1099).

[0230] The above-described techniques can be implemented as computer software that uses computer-readable instructions and is physically stored in one or more computer-readable media. For example, Figure 11 A computer system (1100) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0231] Computer software can be coded using any suitable machine code or computer language. Any suitable machine code or computer language can be assembled, compiled, linked, or similarly processed to create code containing instructions that can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), or through interpreted microcode.

[0232] The instructions can be executed on various types of computers or their components, including personal computers, tablets, servers, smartphones, gaming devices, and Internet of Things devices.

[0233] Figure 11 The components of the computer system (1100) shown are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement relating to any one or a combination of components shown in the exemplary embodiments of the computer system (1100).

[0234] The computer system (1100) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, movement of a data glove), audio input (e.g., speech, clapping), visual input (e.g., gestures), and olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images acquired from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video), etc.

[0235] The input human-machine interface device may include one or more of the following (only one of each is shown): keyboard (1101), mouse (1102), touchpad (1103), touch screen (1110), data glove (not shown), joystick (1105), microphone (1106), scanner (1107), camera (1108).

[0236] The computer system (1100) may also include certain human-machine interface output devices. Such human-machine interface output devices can, for example, stimulate the senses of one or more human users through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include haptic output devices (e.g., haptic feedback from a touchscreen (1110), a data glove (not shown), or a joystick (1105), but may also be haptic feedback devices that are not input devices), audio output devices (e.g., speakers (1109), headphones (not shown)), and visual output devices (e.g., screens including cathode ray tube (CRT) screens, liquid-crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens (1110), each screen may or may not have touchscreen input functionality, each screen may or may not have haptic feedback functionality - some of these screens are capable of outputting two-dimensional or more three-dimensional visual outputs via devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays and smoke boxes (not depicted), and printers (not depicted).

[0237] The computer system (1100) may also include human-accessible storage devices and their associated media: for example, optical media including CD / DVD ROM / RW (1120) with media such as CD / DVD (1121), finger drives (1122), removable hard disk drives or solid-state drives (1123), conventional magnetic media such as magnetic tapes and floppy disks (not shown), and devices based on application-specific ROM / ASIC / PLD such as security dongles (not shown). ASIC stands for Application-Specific Integrated Circuit. PLD stands for Programmable Logic Device.

[0238] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0239] The computer system (1100) may also include an interface to one or more communication networks. The network may be, for example, a wireless network, a wired network, or an optical network. The network may further be a local area network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a latency-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless local area networks (LANs), cellular networks including Global System for Mobile communication (GSM), 3G, 4G, 5G, Long-Term Evolution (LTE), etc., cable or wireless wide area digital television networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial television including Controller Area Network Bus (CANBus), etc. Some networks typically require external network interface adapters (e.g., the Universal Serial Bus (USB) port of a computer system (1100) to connect to certain general-purpose data ports or peripheral buses (1149); other network interfaces are typically integrated into the core of the computer system (1100) by connecting to a system bus (e.g., an Ethernet interface in a PC computer system or a cellular network interface in a smartphone computer system). The computer system (1100) can use any of these networks to communicate with other entities. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., a CANbus connected to a certain CANbus device), or bidirectional, such as connecting to other computer systems using a local area network (LAN) or wide area network (WAN) digital network. As mentioned above, certain protocols and protocol stacks can be used on each of those networks and network interfaces.

[0240] The aforementioned human-machine interface device, human-machine accessible storage device, and network interface can be attached to the kernel (1140) of the computer system (1100).

[0241] The kernel (1140) may include one or more Central Processing Units (CPUs) (1141), Graphics Processing Units (GPUs) (1142), dedicated programmable processing units in the form of Field Programmable Gate Areas (FPGAs) (1143), hardware accelerators (1144) for certain tasks, etc. These devices, as well as read-only memory (ROMs) (1145), random access memory (1146), and internal mass storage such as internal non-user-accessible hard disk drives (HDDs) and solid-state drives (SSDs) (1147), may be connected via a system bus (1148). In some computer systems, the system bus (1148) may be accessed in the form of one or more physical plugs to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly connected to the kernel's system bus (1148) or connected to the kernel's system bus (1148) via a peripheral bus (1149). The architecture of peripheral buses includes Peripheral Component Interconnect (PCI), USB, etc.

[0242] The CPU (1141), GPU (1142), FPGA (1143), and accelerator (1144) can execute certain instructions, which can be combined to form the aforementioned computer code. This computer code can also be stored in ROM (1145) or Random Access Memory (RAM) (1146). Transient data can also be stored in RAM (1146), while permanent data can be stored, for example, in internal mass storage (1147). Fast storage and retrieval of any storage device can be achieved using a cache, which can be closely associated with one or more CPUs (1141), GPUs (1142), mass storage (1147), ROM (1145), RAM (1146), etc.

[0243] Computer-readable media may have computer code thereon for performing various computer-implemented operations. The media and computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0244] By way of non-limiting example, a computer system having an architecture (1100), particularly a kernel (1140), can provide functionality by having one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, and some non-transitory memory of the kernel (1140), such as internal kernel mass storage (1147) or ROM (1145). Software implementing various embodiments of this disclosure can be stored in such means and executed by the kernel (1140). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the kernel (1140), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (1146) and modifying such data structures according to software-defined processes. Additionally or alternatively, the computer system may provide functionality through hard-wired or otherwise embodied logic in circuitry (e.g., the accelerator (1144)), which may replace or operate with the software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0245] Although several exemplary embodiments have been described in this disclosure, modifications, substitutions, and various equivalent alternatives that fall within the scope of this disclosure exist. Therefore, it should be understood that those skilled in the art will be able to design numerous systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and thus fall within its spirit and scope.

Claims

1. A method for video decoding in a video decoder, characterized in that, The method includes: Receive the encoded video stream, including the current block associated with the transform block; Determine whether transformation skipping was used in the transformation block; In response to determining that no transform skipping is used in the transform block, a first transform coefficient sign prediction is performed, the first transform coefficient sign prediction including: The first sign of the first multiple transform coefficients is predicted based on the first multiple neighboring samples, and The prediction of the second sign of the second plurality of transform coefficients is based on the prediction of the second plurality of transform coefficients from the prediction of the first plurality of transform coefficients, wherein the prediction of the first sign of the first plurality of transform coefficients is independent of the prediction of the second sign of the second plurality of transform coefficients; and Reconstruct the current block based on the first plurality of transformation coefficients and the second plurality of transformation coefficients; and In response to determining that the transform skip was used in the transform block, a second transform coefficient sign prediction is performed on the transform block, the second transform coefficient sign prediction being different from the first transform coefficient sign prediction; The determination of whether a transform skip was used in the transform block includes: determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used; in response to the use of the one-dimensional horizontal transform skip in the transform block, independently predicting the signs of coefficients in different columns of the transform block by the second transform coefficient sign prediction; and performing the second transform coefficient sign prediction on the transform block includes: predicting the signs of coefficients in the column using only reconstructed samples adjacent to the column.

2. The method according to claim 1, characterized in that, In response to the use of the transform skip in the transform block, the first transform coefficient sign prediction is disabled.

3. The method according to claim 1, characterized in that, Determining whether a transformation skip was used in the transformation block includes: determining whether one of a one-dimensional horizontal transformation skip and a one-dimensional vertical transformation skip was used; In response to the use of the one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in the first column of the transform block is predicted by the second transform coefficient sign prediction based on the following: One or more reconstructed columns to the left of the transform block; and One or more reconstructed pixels above the first column of the transform block.

4. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used, and In response to the use of the one-dimensional horizontal transform skip in the transform block, the sign of the coefficients in a column other than the first column of the transform block is predicted by the second transform coefficient sign prediction using one or more reconstructed pixels in one or more rows above the transform block.

5. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used. In response to the use of the one-dimensional vertical transform skip in the transform block, the signs of the coefficients in different rows of the transform block are independently predicted by the second transform coefficient sign prediction, and Performing a second transform coefficient sign prediction on the transform block includes predicting the coefficient sign of the row using only the reconstructed samples adjacent to the row.

6. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used. In response to the use of the one-dimensional vertical transform skip in the transform block, the sign of the coefficients in the first row of the transform block is predicted by the second transform coefficient sign prediction based on the following: One or more reconstructed rows above the transform block; and One or more reconstructed pixels to the left of the transform block.

7. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used, and In response to the use of the one-dimensional vertical transform skip in the transform block, the sign of the coefficients in a row other than the first row of the transform block is predicted using one or more reconstructed pixels in one or more columns to the left of the transform block, through the second transform coefficient sign prediction.

8. The method according to claim 1, characterized in that, In response to the use of one-dimensional transform skipping in the transform block, the first transform coefficient sign prediction is disabled.

9. The method according to claim 1, characterized in that, The first total number of symbols predicted by the second transform coefficient symbol prediction when a one-dimensional transform skip is applied in the transform block is different from the second total number of symbols predicted by the first transform coefficient symbol prediction when the transform skip is not applied in any direction in the transform block.

10. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether a two-dimensional transform skip was used, and In response to the use of the two-dimensional transform skip in the transform block, performing the second transform coefficient sign prediction includes: using one or more reconstructed pixels in one or more rows or columns adjacent to the transform block, predicting coefficient signs only for pixels in the first row or first column of the transform block.

11. The method according to claim 1, characterized in that, Determining whether transform skipping was used in the transform block includes: determining whether forward skip coding (FSC) was used, and In response to the use of the FSC in the transform block, performing the second transform coefficient sign prediction includes: using one or more reconstructed pixels in one or more rows or columns adjacent to the transform block, predicting coefficient signs only for pixels in the first row or first column of the transform block.

12. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether a two-dimensional transform skip was used; and In response to the use of the two-dimensional transform skip in the transform block, the sign of the coefficients in the top-left pixel of the transform block is predicted by the second transform coefficient sign prediction based on the following: One or more reconstructed pixels above the top-left pixel in one or more rows above the transform block; and One or more reconstructed pixels to the left of the top-left pixel in one or more columns to the left of the transform block.

13. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether a two-dimensional transform skip was used, and In response to the use of the two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels of the first row of the transform block is predicted by the second transform coefficient sign prediction based on the following: The transform block and one or more first reconstructed pixels in one or more rows above the pixel; and One or more second reconstructed pixels to the left or right of the one or more first reconstructed pixels in one or more rows.

14. The method according to claim 1, characterized in that, Determining whether a transform skip was used in the transform block includes: determining whether a two-dimensional transform skip was used, and In response to the use of the two-dimensional transform skip in the transform block, the sign of the coefficients in the pixels of the first column of the transform block is predicted by the second transform coefficient sign prediction based on the following: The transform block and one or more first reconstructed pixels in one or more columns to the left of the pixel; and One or more second reconstructed pixels above or below the one or more first reconstructed pixels in one or more columns.

15. The method according to claim 1, characterized in that, In response to the use of two-dimensional transform skipping in the transform block, the first transform coefficient sign prediction is not enabled.

16. The method according to claim 1, characterized in that, For intra-coded blocks that use the forward skip coding (FSC) mode, the first transform coefficient symbol prediction is not enabled; and / or Since the inter-frame coding block does not use the forward skip coding (FSC) mode, the first transform coefficient symbol prediction is enabled for the inter-frame coding block.

17. A video decoding apparatus, characterized in that, The device includes: The processing circuit is configured to perform the video decoding method according to any one of claims 1-16.

18. A video decoding apparatus, characterized in that, The device includes: The receiving module is configured to receive an encoded video stream including the current block associated with the transform block; The determination module is configured to determine whether transformation skipping is used in the transformation block; The prediction module is configured to perform a first transform coefficient sign prediction in response to determining that no transform skipping is used in the transform block, the first transform coefficient sign prediction including: The first sign of the first multiple transform coefficients is predicted based on the first multiple neighboring samples, and The prediction of the second sign of the second plurality of transform coefficients is based on the prediction of the second plurality of transform coefficients from the prediction of the first plurality of transform coefficients, wherein the prediction of the first sign of the first plurality of transform coefficients is independent of the prediction of the second sign of the second plurality of transform coefficients; and In response to determining that the transform skip was used in the transform block, a second transform coefficient sign prediction is performed on the transform block, the second transform coefficient sign prediction being different from the first transform coefficient sign prediction; wherein determining whether a transform skip was used in the transform block includes: determining whether one of a one-dimensional horizontal transform skip and a one-dimensional vertical transform skip was used; in response to the use of the one-dimensional horizontal transform skip in the transform block, the signs of coefficients in different columns of the transform block are independently predicted using the second transform coefficient sign prediction; and performing the second transform coefficient sign prediction on the transform block includes: predicting the coefficient signs of the column using only reconstructed samples adjacent to the column; and The reconstruction module is configured to reconstruct the current block based on the first plurality of transformation coefficients and the second plurality of transformation coefficients.

Images