Block-level symbol prediction adaptation

Abstract

A video decoding apparatus, comprising a processing circuit configured to: receive an encoded video bitstream, the encoded video bitstream comprising encoding information of one or more pictures; determining a specific number on the basis of the information of the current picture or the current block, the specific number defining a count of prediction symbols in symbol prediction of transform coefficients of the current block in the current picture; determining one or more transform coefficients for symbol prediction according to the specific number; calculating cost values associated with symbol combinations of the one or more transform coefficients, respectively; determining a particular symbol combination of the one or more transform coefficients based on a cost value associated with the symbol combination; and reconstructing the current block according to the specific symbol combination of the one or more transform coefficients.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 460,883, filed on April 20, 2023, entitled “BLOCK-LEVEL SIGN PREDICTION ADAPTATION,” which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure describes embodiments generally related to video coding. Background Art

[0003] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent that the work of the presently named inventors described in this background section and in various aspects of this specification was performed, it does not indicate that it qualifies as prior art at the time of filing, and it is never explicitly or implicitly admitted that it is prior art to the present disclosure.

[0004] Image / video compression helps to transmit image / video data with minimal quality loss across different devices, storage, and networks. In some examples, video codec technology can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current picture being reconstructed to make sample predictions. In another example, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can predict samples in the current picture based on previously reconstructed pictures through motion compensation. Motion compensation can be indicated by a motion vector (MV). Summary of the invention

[0005] Aspects of the present disclosure include video encoding / decoding methods and apparatus.

[0006] Some aspects of the present disclosure provide a method for processing visual media data. The method includes processing a code stream of visual media data according to a format rule. The format rule specifies: determining a quantity based on information of a current picture or a current block, the quantity defining a count of prediction symbols in a symbol prediction of a transform coefficient of a current block in the current picture. In addition, the format rule specifies: determining one or more transform coefficients for symbol prediction based on the quantity, the absolute values ​​of the one or more transform coefficients being in the code stream, and calculating cost values ​​associated with symbol combinations respectively for the one or more transform coefficients. The first cost value associated with the first symbol combination measures the discontinuity between a reconstructed sample of a current block reconstructed according to the first symbol combination and a reconstructed sample of one or more adjacent blocks. The format rule further specifies: determining a specific symbol combination of one or more transform coefficients, the specific symbol combination having a minimum cost value among the cost values, and reconstructing the current block according to the specific symbol combination of the one or more transform coefficients.

[0007] Some aspects of the present disclosure provide a video encoding method. The method includes: determining a specific number based on information of a current picture or a current block, the specific number defining a count of prediction symbols in symbol prediction of a transform coefficient of a current block in the current picture; determining one or more transform coefficients for symbol prediction based on the specific number; calculating cost values ​​associated with symbol combinations of one or more transform coefficients respectively; determining a specific symbol combination of one or more transform coefficients, the specific symbol combination having a minimum cost value among the cost values; and reconstructing the current block according to the specific symbol combination of one or more transform coefficients.

[0008] In some examples, to calculate the cost value, the method includes calculating a first cost value associated with a first symbol combination by measuring a discontinuity between reconstructed samples of a current block reconstructed according to the first symbol combination of one or more transform coefficients and reconstructed samples of one or more neighboring blocks.

[0009] In some examples, to determine the specific number, the method includes: determining the specific number, the specific number being the minimum of a first number indicating an upper limit of prediction symbols of a current transform block of a current block and a second number of non-zero transform coefficients in the current transform block of the current block, the first number being allocated based on a maximum number of prediction symbols of transform blocks in a current picture. The method also includes: signaling the maximum number of prediction symbols as a syntax element at a picture level in an encoded video bitstream, the encoded video bitstream including encoding information of the current picture, the syntax element indicating the maximum number of prediction symbols of the transform block in the current picture.

[0010] In some examples, the method includes deriving a second number of non-zero transform coefficients in a current transform block of the current block, and adjusting the first number based on the second number of non-zero transform coefficients in the current transform block.

[0011] In some examples, the method includes adjusting the first amount based on a sum of absolute values ​​of non-zero transform coefficients of the current transform block.

[0012] In some examples, the method includes adjusting the first amount based on at least one of whether a secondary transform is applied, whether a specific secondary transform kernel is applied, whether the current block is an intra-coded block, and whether the current block is an inter-coded block.

[0013] In some examples, the method includes adjusting the first quantity based on at least one of: availability of neighboring reconstructed samples, values ​​of quantization parameters of neighboring blocks, and a relative position of the current block in a current Coding Tree Unit (CTU).

[0014] Some aspects of the present disclosure provide a video decoding device. The device includes a processing circuit, which is configured to: receive an encoded video code stream, the encoded video code stream includes encoding information of one or more pictures; determine a specific number based on information of a current picture or a current block, the specific number defining the count of prediction symbols in the symbol prediction of the transform coefficient of the current block in the current picture; determine one or more transform coefficients for symbol prediction according to the specific number; calculate cost values ​​associated with the symbol combination of one or more transform coefficients respectively; determine a specific symbol combination of one or more transform coefficients based on the cost value associated with the symbol combination; and reconstruct the current block according to the specific symbol combination of one or more transform coefficients.

[0015] In some examples, the processing circuit is configured to calculate a first generation value associated with the first symbol combination by measuring a discontinuity between reconstructed samples of a current block and reconstructed samples of a neighboring block re-added according to the first symbol combination of one or more transform coefficients.

[0016] In some examples, the processing circuit is configured to determine a particular combination of the cost values ​​having a minimum cost value.

[0017] In some examples, the processing circuit is configured to: parse the encoded video code stream to obtain a maximum number of prediction symbols for a transform block in the current picture according to a syntax element of the current picture signaled at a picture level, the first number defining an upper limit of the prediction symbols for the current transform block of the current block, the upper limit being allocated according to the maximum number of prediction symbols for the transform block in the current picture. In addition, the processing circuit is configured to determine a second number of non-zero transform coefficients in the current transform block of the current block, and select a minimum value of the first number and the second number as the specific number.

[0018] In some examples, the processing circuit is configured to parse a control flag of a syntax element from an encoded video bitstream, the control flag being in at least one of a sequence parameter set (SPS) and a picture parameter set (PPS). When the control flag indicates the use of the syntax element, the processing circuit is configured to parse the encoded video bitstream to obtain a first quantity based on the syntax element signaled at the picture level.

[0019] In some examples, the processing circuit is configured to derive a second number of non-zero transform coefficients in a current transform block of the current block, and adjust the first number based on the second number of non-zero transform coefficients.

[0020] In some examples, the processing circuit is configured to: parse the absolute value of the transform coefficient of the current transform block from the encoded video code stream, and adjust the first number based on the absolute value of the transform coefficient.

[0021] In some examples, the processing circuit is configured to adjust the first amount based on a sum of absolute values ​​of the transform coefficients.

[0022] In some examples, the processing circuit is configured to adjust the first amount based on at least one of: whether a secondary transform is applied, whether a particular secondary transform kernel is applied, whether the current block is an intra-coded block, and whether the current block is an inter-coded block.

[0023] In some examples, the processing circuit is configured to adjust the first number based on the availability of adjacent reconstruction samples.

[0024] In some examples, the processing circuit is configured to adjust the first amount based on a value of a quantization parameter of a neighboring block.

[0025] In some examples, the processing circuit is configured to adjust the first amount based on a relative position of the current block in a coding tree unit (CTU).

[0026] According to another aspect of the present disclosure, a device is provided. The device includes a processing circuit. The processing circuit can be configured to perform any of the video decoding / encoding methods described.

[0027] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions, which, when executed by a computer, cause the computer to perform any of the described video decoding / encoding methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other features, properties and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0029] Figure 1 is a schematic diagram of an exemplary block diagram of a communication system (100).

[0030] Figure 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0031] Figure 3 is a schematic diagram of an exemplary block diagram of an encoder.

[0032] Figure 4 A diagram is shown to illustrate symbol prediction in some examples.

[0033] Figure 5 A flow chart outlining a decoding process according to some embodiments of the present disclosure is shown.

[0034] Figure 6 A flowchart outlining an encoding process according to some embodiments of the present disclosure is shown.

[0035] Figure 7 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0036] Figure 1 A block diagram of a video processing system (100) in some examples is shown. As an example of an application of the disclosed subject matter, the video processing system (100) is a video encoder and a video decoder in a streaming environment. The disclosed subject matter is equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0037] The video processing system (100) may include an acquisition subsystem (113), which may include a video source (101) such as a digital camera, which creates an uncompressed video picture stream (102). In one example, the video picture stream (102) includes samples captured by the digital camera. The video picture stream (102) is depicted as a thick line to emphasize the high data volume of the video picture stream compared to the encoded video data (104) (or the encoded video code stream), and the video picture stream (102) can be processed by an electronic device (120), which includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. The encoded video data (104) (or the encoded video code stream) depicted as a thin line to emphasize the lower data volume compared to the video picture stream (102) can be stored on a streaming server (105) for future use. One or more streaming client subsystems, such as Figure 1 A client subsystem (106) and a client subsystem (108) in a streaming server (105) may access a streaming server (105) to retrieve a copy (107) and a copy (109) of the encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and generates an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application may be used in the context of the VVC standard.

[0038] It should be noted that the electronic device (120) and the electronic device (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0039] Figure 2 An exemplary block diagram of a video decoder (210) is shown. The video decoder (210) may be disposed in an electronic device (230). The electronic device (230) may include a receiver (231) (eg, a receiving circuit). The video decoder (210) may be used to replace Figure 1A video decoder (110) of an embodiment.

[0040] The receiver (231) may receive one or more encoded video sequences (e.g., included in a bitstream) to be decoded by the video decoder (210). In one embodiment, the encoded video sequences are received one at a time, wherein the decoding of each encoded video sequence is independent of the decoding of the other encoded video sequences. The encoded video sequence may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (231) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be set outside the video decoder (210) (not shown). In other cases, a buffer memory (not shown) is set outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be configured inside the video decoder (210) to, for example, handle broadcast timing. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network, it may not be necessary to configure the buffer memory (215), or the buffer memory may be made smaller. Of course, in order to use on a service packet network such as the Internet, a buffer memory (215) may also be required. The buffer memory may be relatively large and may have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not shown) outside the video decoder (210).

[0041] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potential information for controlling a display device such as a display device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown in . The control information for the display device may be a parameter set fragment (not indicated) of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (220) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0042] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0043] Depending on the type of coded video picture or part of coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. For the sake of brevity, such subgroup control information flow between the parser (220) and the multiple units below is not described.

[0044] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0045] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) and control information from the parser (220), including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into the aggregator (255).

[0046] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of a current picture. Such predictive information may be provided by an intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates surrounding blocks of the same size and shape as the block being reconstructed using reconstructed information extracted from a current picture buffer (258). For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0047] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the block-related symbols (221), these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The acquisition of prediction samples by the motion compensated prediction unit (253) from the address in the reference picture memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbol (221), which includes, for example, X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0048] The output samples of the aggregator (255) may be used by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and available to the loop filter unit (256) as symbols (221) from the parser (220). The video compression may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop filtered sample values.

[0049] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0050] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0051] The video decoder (210) may perform decoding operations according to a predetermined video compression technology or standard such as ITU-T H.265. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used in the sense that the encoded video sequence follows the syntax of the video compression technology or standard and the profile recorded in the video compression technology or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technology or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management signaled in the encoded video sequence.

[0052] In an embodiment, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0053] Figure 3 An exemplary block diagram of a video encoder (303) is shown. The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (eg, a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an embodiment.

[0054] The video encoder (303) can be used to obtain the video source (301) (not Figure 3 In another example, the video source (301) is a part of the electronic device (320) in the embodiment, and receives the video samples. The video source can collect video images to be encoded by the video encoder (303). In another example, the video source (301) is a part of the electronic device (320).

[0055] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), which may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, where each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The following description focuses on samples.

[0056] According to an embodiment, the video encoder (303) may encode and compress pictures of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints as required. Implementing an appropriate encoding speed is a function of the controller (350). In some embodiments, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of brevity, couplings are not shown in the figure. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, group of pictures (group of pictures, GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0057] In some embodiments, the video encoder (303) operates in a coding loop. As a simple description, in embodiments, the coding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0058] The operation of the "local" decoder (333) may be combined with, for example, Figure 2 The "remote" decoder described in detail for the video decoder (210) is identical. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0059] In one embodiment, decoder techniques other than parsing / entropy decoding present in the decoder are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on decoder operation. The description of encoder techniques can be simplified because encoder techniques are mutually inverse to the decoder techniques described comprehensively. A more detailed description in some areas is provided below.

[0060] During operation, in some examples, the source encoder (330) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the difference between a pixel block of the input picture and a pixel block of a reference picture that may be selected as a prediction reference for the input picture.

[0061] The local video decoder (333) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When the video sequence is decoded at a remote video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture that will be obtained by the remote video decoder.

[0062] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (335) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (334).

[0063] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0064] The outputs of all the above functional units may be entropy encoded in an entropy encoder (345). The entropy encoder (345) applies lossless compression to the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0065] The transmitter (340) may buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0066] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0067] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0068] A predictive picture (P picture) may be a picture that may be encoded and decoded using intra prediction or inter prediction, which uses a motion vector and a reference index to predict sample values ​​of each block.

[0069] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstructing a single block.

[0070] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, a block of an I picture may be non-predictively coded, or the block may be predictively coded (spatial prediction or intra prediction) with reference to an already coded block of the same picture. A block of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. A block of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0071] The video encoder (303) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0072] In an embodiment, the transmitter (340) may transmit additional data when transmitting the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0073] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0074] In some embodiments, bidirectional prediction techniques may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. Specifically, the block may be predicted by a combination of the first reference block and the second reference block.

[0075] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0076] According to some embodiments disclosed in the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine the prediction type for the CU, such as an inter-prediction type or an intra-prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luminance prediction block (PB) and two chrominance PBs. In an embodiment, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luminance prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luminance values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0077] It should be noted that the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using any suitable technology. In an embodiment, the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using one or more integrated circuits. In another embodiment, the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using one or more processors that execute software instructions.

[0078] Aspects of the present disclosure provide techniques for block-level symbol prediction adaptation.

[0079] In some examples, symbol prediction of transform coefficients is used to improve codec efficiency. In some examples, before encoding the coefficients in a TU, the encoder can determine which symbols to predict and predict these symbols. The encoder performs hypothesis processing during RDO decision.

[0080] In the example, the encoder first dequantizes the TU and then selects n coefficients (n is a positive integer) to predict the symbol. In the example, using n coefficients, perform 2 n Secondary boundary reconstruction (corresponding to the positive and negative signs of n coefficients, respectively, 2 n The boundary reconstruction is performed once for each unique sign combination (each sign can be positive or negative) for n coefficients. In addition, a cost value associated with each hypothesis (the hypothesis corresponds to a sign combination) is calculated based on the concept of image continuity at block boundaries. Sign prediction can be performed by selecting the hypothesis that minimizes the cost value.

[0081] For example, when two symbols are selected for symbol prediction, there may be 4 possible combinations (4 hypotheses), namely (+, +), (+, -), (-, +), (-, -). For all four combinations, the cost function is calculated, and the combination with the minimum cost is selected as the symbol predictor combination. When more symbols are predicted and more combinations need to be tried, the same process is applied, and a trade-off can be made between complexity and compression efficiency.

[0082] Figure 4 A diagram is shown to illustrate sign prediction in some examples. In some examples, the reconstructed residuals for negative and positive sign combinations of applicable transform coefficients are calculated, and then a hypothesis is selected by minimizing a cost function. Figure 4In an example, some samples in the current block (such as the first row samples in the current block and / or the first column samples in the current block) may be reconstructed for the negative sign combination and the positive sign combination of the applicable transform coefficients. Then, for each of the negative sign combination and the positive sign combination, a cost value may be calculated based on the reconstructed samples for the sign combination and the reconstructed adjacent samples in the current block. It should be noted that in some examples, residual reconstruction is performed for each sign combination instead of sample reconstruction for each sign combination to reduce the amount of computation.

[0083] In some examples, the cost function is defined as a measure of discontinuity across block boundaries. Figure 4 In some examples, the residual of the boundary sample (410) is reconstructed according to each hypothesis of the sign of the transform coefficient. Then, the cost value of the hypothesis is calculated based on the cost function. The one hypothesis with the smallest cost value is selected as the predictor of the sign of the transform coefficient. In some examples, the cost function (cost) is defined as the sum of the absolute second-order derivatives of the upper row and left column of the current block in the residual domain, such as defined according to equation (1): Where R represents the reconstructed neighboring samples, P represents the predicted samples of the current block, r represents the sample residual of the hypothesis, w represents the width of the current block, and h represents the height of the current block. In some examples, some items do not change with the hypothesis and can be calculated once for the current block and stored in the buffer. For example, each block can only be calculated once (-R x,-1 +2R x,0 -P x,1 )(x∈0…w),(-R -1,y +2R 0,y -P 1,y )(y∈0…h).

[0084] In some examples, a portion of the transform coefficients is selected for symbol prediction. In an example, the transform coefficient with the largest K qIdx value in the upper left 4×4 region is selected (e.g., K is a positive integer), and the qIdx value is the transform coefficient level after compensating for the impact of multiple quantizers in dependent quantization. A larger qIdx value will produce a larger dequantized transform coefficient level. In an example, qIdx is derived according to equation (2): qIdx = (abs(level)<<1)-(state&1) Equation (2) Among them, "level" represents the transform coefficient level parsed from the bitstream, and "state" is a variable maintained by the encoder and decoder in dependent quantization.

[0085] In some examples, the symbol prediction area can be expanded to a maximum of 32×32. For example, the symbol of the upper left M×N block is predicted. The values ​​of M and N can be calculated according to equations (3) and (4): M=min(Tw,maxW) Equation (3) N = min(Th, maxH) Equation (4) Where Tw and Th are the width and height of the transform block, and maxW and maxH are the width and height of the maximum region. In some examples, the maximum region for symbol prediction is set to 32×32. In some examples, the maximum region for symbol prediction is not always set to 32×32. In examples, the encoder sets the maximum region (maxW, maxH) based on parameters such as configuration, sequence category, QP, etc., and signals the maximum region in the SPS.

[0086] In some related examples, the maximum number of predicted symbols remains unchanged. Symbol prediction is also applied to low-frequency non-separable transform (LFNST) blocks. For example, for LFNST blocks, symbol prediction is allowed for a maximum predetermined number of coefficients (e.g., 4, 8, etc.) in the upper left 4×4 region.

[0087] In a related example, the maximum number of prediction symbols (denoted as NumSignPred) is specified in the sequence parameter set. In another related example, the default configuration sets NumSignPred to 8, and the minimum value between 8 and the number of non-zero coefficients in the region for symbol prediction is applied as the number of prediction symbols in the transform block. In another related example, a smaller value of NumSignPred is used in a higher rate test configuration to reduce the overall encoding runtime.

[0088] According to one aspect of the present disclosure, the code rate is usually unevenly distributed among different pictures or different blocks, and the general sequence level setting of NumSignPred may not be optimal for better achieving the coding gain and runtime trade-off.

[0089] It should be noted that in the present disclosure, the term "transform" may refer to a primary transform (e.g., multiple transform selection (MTS) or non-separable primary transform (NSPT)) or a secondary transform (e.g., non-separable secondary transform (NSST) or LFNST).

[0090] Some aspects of the present disclosure provide techniques for determining the number of prediction symbols using frame-level and / or block-level coding information, so that the number of prediction symbols is adapted at the frame-level and / or block-level. For example, the encoder / decoder can determine a specific number based on information of the current picture or the current block, and the specific number defines the count of prediction symbols in the symbol prediction of the transform coefficients of the current block in the current picture. And, one or more transform coefficients used for symbol prediction are based on the specific number. Calculate cost values ​​associated with symbol combinations of one or more transform coefficients, respectively. Determine a specific symbol combination of one or more transform coefficients based on the cost value associated with the symbol combination, and reconstruct the current block based on the specific symbol combination of one or more transform coefficients.

[0091] In some embodiments, a picture-level syntax (e.g., pic_num_pred_signs) is signaled to indicate the maximum number of prediction symbols for each transform block in the current picture. In an example, the actual number of prediction symbols for a transform block is determined as the minimum between pic_num_pred_signs and the number of non-zero transform coefficients in the transform block.

[0092] In some examples, picture level syntax is used when a control flag of this syntax element in a sequence parameter set (SPS) or picture parameter set (PPS), etc., is true.

[0093] In some examples, the number of non-zero coefficients (N) is first derived for each transform block, and then the maximum number of prediction symbols is further adjusted for the transform block based on the value of N. In one example, the maximum number of prediction symbols is reduced for larger values ​​of N. In another example, the maximum number of prediction symbols is increased for larger values ​​of N.

[0094] In some examples, the absolute values ​​of the transform coefficients are first parsed for each transform block, and then the maximum number of prediction symbols is further adjusted for the transform block based on the absolute values ​​of the non-zero coefficients. In one example, for larger values ​​of the sum of the absolute values ​​of the coefficients, the maximum number of prediction symbols for the transform block is reduced. In another example, for larger values ​​of the sum of the absolute values ​​of the coefficients, the maximum number of prediction symbols for the transform block is increased.

[0095] In some examples, based on the coding mode information, the maximum number of prediction symbols is further adjusted for the transform block. In one example, the coding mode information is: whether to apply a secondary transform or a specific secondary transform kernel / index. In another example, the coding mode information is: whether the coding block is an intra-frame coding block or an inter-frame coding block.

[0096] In some examples, the maximum number of prediction symbols is further adjusted for the transform block based on the availability of neighboring reconstructed samples.

[0097] In an example, when the above, left, or top-left neighboring reconstructed samples are not available (eg, at a picture boundary), the maximum number of prediction symbols for the current transform block is adjusted by different amounts (eg, decreased or increased).

[0098] In some examples, the maximum number of prediction symbols is further adjusted for the transform block based on the applied QP value of the neighboring reconstructed samples. In an example, the maximum number of prediction symbols is reduced when the neighboring block is encoded with a higher QP value than the current block.

[0099] In some examples, the maximum number of prediction symbols is further adjusted for the transform block based on the relative position of the current coding block in the current CTU. In an example, when the current block is located at an upper CTU boundary or a left CTU boundary, the maximum number of prediction symbols is reduced.

[0100] Figure 5 A flow chart outlining a process (500) according to an embodiment of the present disclosure is shown. The process (500) may be used in a video decoder. In various embodiments, the process (500) is performed by a processing circuit, such as a processing circuit that performs the functions of a video decoder (110), a processing circuit that performs the functions of a video decoder (210), etc. In some embodiments, the process (500) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (500). The process starts at (S501) and proceeds to (S510).

[0101] At (S510), an encoded video stream including encoding information of one or more pictures is received.

[0102] At (S520), a specific number defining a count of prediction symbols in symbol prediction of a transform coefficient of a current block in the current picture is determined based on information of the current picture or the current block.

[0103] At (S530), one or more transform coefficients for symbol prediction are determined according to a specific number.

[0104] At (S540), cost values ​​respectively associated with sign combinations of one or more transform coefficients are calculated.

[0105] At (S550), a particular sign combination of one or more transform coefficients is determined based on a cost value associated with the sign combination.

[0106] At (S560), the current block is reconstructed according to a specific sign combination of one or more transform coefficients. In an example, a residual block of the current block is reconstructed based on the specific sign combination of one or more transform coefficients, and then the current block is reconstructed (e.g., the current block is reconstructed based on the prediction of the current block and the residual block of the current block).

[0107] In some examples, a first generation value associated with a first symbol combination is calculated by measuring a discontinuity between reconstructed samples of a current block and reconstructed samples of a neighboring block reconstructed according to the first symbol combination of one or more transform coefficients.

[0108] In some examples, the particular combination is determined to be the combination having the smallest cost value among the cost values.

[0109] In some examples, the encoded video bitstream is parsed to obtain a maximum number of prediction symbols for a transform block in the current picture according to a syntax element of the current picture signaled at a picture level. A first number defines an upper limit of the prediction symbols for the current transform block of the current block, and the first number is allocated according to the maximum number of prediction symbols for the transform block in the current picture. A second number of non-zero transform coefficients in the current transform block of the current block is determined. Then, a minimum value of the first number and the second number is selected as the specific number.

[0110] In some examples, a control flag of a syntax element is parsed from a coded video bitstream, the control flag being in at least one of a sequence parameter set (SPS) and a picture parameter set (PPS). When the control flag indicates the use of the syntax element, the coded video bitstream is parsed to obtain a maximum number of syntax elements signaled at a picture level.

[0111] In some examples, a second number of non-zero transform coefficients in a current transform block of the current block is determined, and the first number is adjusted based on the second number of non-zero transform coefficients. In one example, when the second number is greater than a threshold, the first number is increased. In another example, when the second number is greater than the threshold, the first number is decreased.

[0112] In some examples, the absolute value of the transform coefficient of the current transform block is parsed from the encoded video code stream. The first quantity is adjusted based on the absolute value of the transform coefficient. In one example, the first quantity is adjusted based on the sum of the absolute values ​​of the transform coefficients. In one example, when the sum of the absolute values ​​of the transform coefficients is greater than a threshold, the first quantity is increased. In another example, when the sum of the absolute values ​​of the transform coefficients is greater than a threshold, the first quantity is decreased.

[0113] In some examples, the first number is adjusted based on coding mode information of the current block (e.g., whether a secondary transform is applied, whether a specific secondary transform kernel is applied, whether the current block is an intra-coded block, and whether the current block is an inter-coded block).

[0114] In some examples, the first number is adjusted based on the availability of adjacent reconstruction samples. In one example, the first number is reduced when the upper adjacent sample is not available. In another example, the first number is reduced when the left adjacent sample is not available.

[0115] In some examples, the first number is adjusted based on a value of a quantization parameter of a neighboring block. In an example, the first number is reduced when the neighboring block is encoded with a higher QP value than the current block.

[0116] In some examples, the first number is adjusted based on the relative position of the current block in a coding tree unit (CTU). In one example, the first number is reduced when the current block is located at an upper CTU boundary. In another example, the first number is reduced when the current block is located at a left CTU boundary.

[0117] Then, the process proceeds to (S599) and ends.

[0118] The process (500) may be adjusted appropriately. Steps in the process (500) may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0119] Figure 6 A flow chart outlining a process (600) according to an embodiment of the present disclosure is shown. The process (600) may be used in a video encoder. In various embodiments, the process (600) is performed by a processing circuit, such as a processing circuit that performs the functions of the video encoder (103), a processing circuit that performs the functions of the video encoder (303), etc. In some embodiments, the process (600) is implemented in software instructions, so when the processing circuit executes the software instructions, the processing circuit performs the process (600). The process starts at (S601) and proceeds to (S610).

[0120] At (S610), a specific number defining a count of prediction symbols in symbol prediction of a transform coefficient of a current block in the current picture is determined based on information of a current picture or a current block.

[0121] At (S620), one or more transform coefficients for symbol prediction are determined based on the specific quantity.

[0122] At (S630), cost values ​​respectively associated with sign combinations of one or more transform coefficients are calculated.

[0123] At (S640), a specific symbol combination of the one or more transform coefficients is determined, the specific symbol combination having a minimum cost value among the cost values.

[0124] At (S650), the current block is reconstructed according to a specific sign combination of one or more transform coefficients. In an example, a residual block of the current block is reconstructed based on the specific sign combination of one or more transform coefficients, and then the current block is reconstructed (e.g., the current block is reconstructed based on the prediction of the current block and the residual block of the current block).

[0125] In some examples, a first generation value associated with a first symbol combination is calculated by measuring discontinuity between reconstructed samples of a current block and reconstructed samples of one or more neighboring blocks reconstructed according to the first symbol combination of one or more transform coefficients.

[0126] In some examples, the specific number is a minimum value of a first number indicating an upper limit of prediction symbols of a current transform block of the current block and a second number of non-zero transform coefficients in the current transform block of the current block. The first number is allocated based on a maximum number of prediction symbols of transform blocks in the current picture. The maximum number of prediction symbols is signaled as a syntax element at a picture level in an encoded video bitstream, the encoded video bitstream including encoding information of the current picture, the syntax element indicating the maximum number of prediction symbols of the transform blocks in the current picture.

[0127] In some examples, a second number of non-zero transform coefficients in a current transform block of the current block is derived. The first number is adjusted based on the second number of non-zero transform coefficients in the current transform block.

[0128] In some examples, the first amount is adjusted based on a sum of absolute values ​​of non-zero transform coefficients of the current transform block.

[0129] In some examples, the first number is adjusted based on at least one of: whether a secondary transform is applied, whether a particular secondary transform kernel is applied, whether the current block is an intra-coded block, and whether the current block is an inter-coded block.

[0130] In one example, the first number is adjusted based on the availability of neighboring reconstructed samples. In another example, the first number is adjusted based on the value of a quantization parameter of a neighboring block. In another example, the first number is adjusted based on the relative position of the current block in a current coding tree unit (CTU).

[0131] Then, the process proceeds to (S699) and ends.

[0132] The process (600) may be adjusted appropriately. Steps in the process (600) may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0133] According to one aspect of the present disclosure, in some examples, a code stream of visual media data is processed according to a format rule. The format rule specifies: determining a specific number based on information of a current picture or a current block, the specific number defining a count of prediction symbols in a symbol prediction of a transform coefficient of a current block in the current picture; determining one or more transform coefficients for symbol prediction based on the specific number; including the absolute value of the one or more transform coefficients in the code stream; calculating cost values ​​associated with symbol combinations respectively for one or more transform coefficients, a first cost value associated with a first symbol combination measuring a discontinuity between a reconstructed sample of the current block reconstructed according to the first symbol combination and a reconstructed sample of one or more adjacent blocks; determining a specific symbol combination of one or more transform coefficients, the specific symbol combination having a minimum cost value among the cost values; reconstructing the current block according to the specific symbol combination of one or more transform coefficients.

[0134] The above techniques may be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. Figure 7 A computer system (700) suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0135] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked, or similar mechanisms to create code comprising instructions that may be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or through interpretation, microcode, etc.

[0136] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, IoT devices, etc.

[0137] Figure 7 The components shown for the computer system (700) are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Nor should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the exemplary embodiment of the computer system (700).

[0138] The computer system (700) may include certain human-machine interface input devices. Such human-machine interface input devices may be responsive to input from one or more human users, for example, through: tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), video (e.g., two-dimensional video, three-dimensional video including stereoscopic video), etc.

[0139] The human-machine interface input device may include one or more of the following (only one of each is shown): keyboard (701), mouse (702), touch pad (703), touch screen (710), data gloves (not shown), joystick (705), microphone (706), scanner (707), camera (708).

[0140] The computer system (700) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate one or more human user senses, for example, through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (710), a data glove (not shown), or a joystick (705), but may also be a tactile feedback device that is not an input device), an audio output device (e.g., a speaker (709), a headset (not shown)), a visual output device (e.g., a screen (710) including a CRT screen, an LCD screen, a plasma screen, an OLED screen, each with or without a touch screen input function, each with or without a tactile feedback function, some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through devices such as stereoscopic image output, virtual reality glasses (not shown), a holographic display and a smoke box (not shown), and a printer (not shown)).

[0141] The computer system (700) may also include human-accessible storage devices and their associated media: for example, optical media including CD / DVD ROM / RW (720) with CD / DVD etc. media (721), thumb drives (722), removable hard drives or solid-state drives (723), traditional magnetic media such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0142] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transient signals.

[0143] The computer system (700) may also include an interface (754) to one or more communication networks (755). The network may be, for example, a wireless network, a wired network, an optical network. The network may further be a local network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a delay-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial television including CANBus, etc. Some networks typically require an external network interface adapter connected to some universal data port or peripheral bus (749) (e.g., a USB port of the computer system (700)). Other network interfaces are typically integrated into the kernel of the computer system (700) by connecting to a system bus as described below (e.g., connected to an Ethernet interface in a PC computer system or connected to a cellular network interface in a smartphone computer system). The computer system (700) can use any of these networks to communicate with other entities. Such communications may be one-way receive only (e.g., broadcast television), one-way send only (e.g., a CANbus connected to certain CANbus devices), or bidirectional, for example, connecting to other computer systems using a LAN or WAN digital network. Certain protocols and protocol stacks may be used on each of those networks and network interfaces as described above.

[0144] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the kernel ( 740 ) of the computer system ( 700 ).

[0145] The core (740) may include one or more central processing units (CPUs) (741), graphics processing units (GPUs) (742), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (743), hardware accelerators (744) for certain tasks, graphics adapters (750), etc. These devices, as well as read-only memory (ROM) (745), random access memory (746), internal mass storage (747) such as internal non-user accessible hard drives, SSDs, etc., may be connected via a system bus (748). In some computer systems, the system bus (748) may be accessed in the form of one or more physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (748) or to the core's system bus via a peripheral bus (749). In one example, a screen (710) may be connected to a graphics adapter (750). The architecture of the peripheral bus includes PCI, USB, etc.

[0146] The CPU (741), GPU (742), FPGA (743) and accelerator (744) can execute certain instructions, which can be combined to form the above-mentioned computer code. The computer code can be stored in ROM (745) or RAM (746). Transition data can also be stored in RAM (746), while permanent data can be stored, for example, in internal mass storage (747). Fast storage and retrieval to any storage device can be performed by using a cache, which can be closely associated with the following: one or more CPUs (741), GPUs (742), mass storage (747), ROM (745), RAM (746), etc.

[0147] The computer readable medium may have thereon computer codes for performing various computer-implemented operations. The medium and computer codes may be those specially designed and constructed for the purposes of the present disclosure, or the medium and computer codes may be of a type well known and available to those skilled in the art of computer software.

[0148] As a non-limiting example, a computer system having an architecture (700), particularly a kernel (740), provides functionality due to one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software contained in one or more tangible computer-readable media. Such computer-readable media may be media associated with user-accessible mass storage as described above, as well as certain non-temporary memories of the kernel (740), such as kernel internal mass storage (747) or ROM (745). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the kernel (740). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software may cause the kernel (740), particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (746) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system provides functionality due to logic hardwired or otherwise embodied in circuitry (e.g., accelerator (744)), which circuitry may replace software or operate in conjunction with software to perform specific processes or specific portions of specific processes described herein. Where appropriate, references to portions of software may include logic and vice versa. Where appropriate, references to portions of computer-readable media may include circuitry (e.g., integrated circuits (ICs)) storing software for execution, circuitry embodying logic for execution, or both. The present disclosure includes any suitable combination of hardware and software.

[0149] The use of "at least one of" or "one of" in this disclosure is intended to include any one of the elements or combinations of the elements. For example, reference to at least one of A, B, or C, at least one of A, B, and C, at least one of A, B, and / or C, and at least one of A to C is intended to include only A, only B, only C, or any combination thereof. Reference to one of A or B and one of A and B is intended to include A or B or (A and B). Where appropriate, the use of "one of" does not exclude any combination of the elements, such as when the elements are not mutually exclusive.

[0150] Although the present disclosure has described a number of exemplary embodiments, there are modifications, permutations, and various replacement equivalents that fall within the scope of the present disclosure. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore fall within the spirit and scope of the present disclosure.

Claims

1. A method for processing visual media data, the method comprising: Processing a bit stream of visual media data according to a format rule, wherein the bit stream of the visual media data includes: a current picture, and the number of prediction symbols, wherein: The format rules specify: The number of prediction symbols, the number of prediction symbols being based on information of a current picture or a current block and defining a count of prediction symbols in a sign prediction of a transform coefficient of the current block in the current picture; one or more transform coefficients for said symbol prediction based on said quantity; the absolute value of the one or more transform coefficients in the bitstream; cost values ​​respectively associated with symbol combinations calculated for the one or more transform coefficients, a first cost value associated with a first symbol combination measuring a discontinuity between reconstructed samples of a current block and reconstructed samples of one or more neighboring blocks reconstructed according to the first symbol combination; a specific sign combination of the one or more transform coefficients, the specific sign combination having a minimum cost value among the cost values; and The current block is reconstructed according to the specific sign combination of the one or more transform coefficients.

2. A video encoding method, comprising: Determining a number of prediction symbols based on information of a current block, the number of prediction symbols defining a count of prediction symbols in a sign prediction of transform coefficients of the current block in a current picture; determining one or more transform coefficients for the symbol prediction based on the number of the predicted symbols for the current block; calculating cost values ​​respectively associated with sign combinations of the one or more transform coefficients; determining a specific symbol combination of the one or more transform coefficients, the specific symbol combination having a minimum cost value among the cost values; and The current block is reconstructed according to the specific sign combination of the one or more transform coefficients.

3. The method according to claim 2, wherein: The calculating of the cost value further comprises: A first generation value associated with the first symbol combination is calculated by measuring discontinuity between reconstructed samples of the current block and reconstructed samples of one or more neighboring blocks reconstructed according to the first symbol combination of the one or more transform coefficients.

4. The method according to any one of claims 2 to 4, wherein: The determining of the quantity further comprises: determining a number that is a minimum value of a first number indicating an upper limit of a prediction symbol of a current transform block of the current block and a second number of non-zero transform coefficients in the current transform block of the current block, the first number being allocated based on a maximum number of prediction symbols of transform blocks in the current picture; and The maximum number of prediction symbols is signaled as a syntax element at a picture level in an encoded video bitstream, wherein the encoded video bitstream includes encoding information of the current picture, and the syntax element indicates the maximum number of prediction symbols for the transform block in the current picture.

5. The method according to claim 4, further comprising: deriving a second number of the non-zero transform coefficients in the current transform block of the current block; as well as The first number is adjusted based on a second number of the non-zero transform coefficients in the current transform block.

6. The method according to claim 4, further comprising: The first number is adjusted based on a sum of absolute values ​​of non-zero transform coefficients of the current transform block.

7. The method according to claim 4, further comprising: The first amount is adjusted based on at least one of: whether to apply the quadratic transformation; whether to apply a specific quadratic transform kernel; Whether the current block is an intra-coded block; and Whether the current block is an inter-frame coded block.

8. The method according to claim 4, further comprising: The first amount is adjusted based on at least one of: Availability of adjacent reconstruction samples; the values ​​of the quantization parameters of the neighboring blocks; as well as The relative position of the current block in the current coding tree unit (CTU).

9. A video decoding device, comprising a processing circuit, wherein the processing circuit is configured to: Receiving an encoded video code stream, wherein the encoded video code stream includes encoding information of one or more pictures; Determining a number of prediction symbols based on information of a current block, the number of prediction symbols defining a count of prediction symbols in a sign prediction of transform coefficients of the current block in a current picture; determining one or more transform coefficients for the symbol prediction based on the number of the predicted symbols; calculating cost values ​​respectively associated with sign combinations of the one or more transform coefficients; determining a particular sign combination of the one or more transform coefficients based on the cost value associated with the sign combination; and The current block is reconstructed according to the specific sign combination of the one or more transform coefficients.

10. The device according to claim 9, wherein: The processing circuit is configured to calculate a first generation value associated with the first symbol combination by measuring discontinuity between reconstructed samples of the current block and reconstructed samples of a neighboring block reconstructed according to the first symbol combination of the one or more transform coefficients.

11. The device according to any one of claims 9 to 10, wherein: The processing circuit is configured to determine the particular combination having a minimum cost value among the cost values.

12. The device according to any one of claims 9 to 11, wherein: The processing circuit is configured to: Parsing the encoded video bitstream to obtain a maximum number of prediction symbols for a transform block in the current picture according to a syntax element of the current picture signaled at a picture level, wherein a first number defines an upper limit of prediction symbols for a current transform block of the current block, the upper limit being allocated according to the maximum number of prediction symbols for the transform block in the current picture; determining a second number of non-zero transform coefficients in the current transform block of the current block; as well as A minimum value between the first number and the second number is selected as the number.

13. The device according to claim 12, wherein: The processing circuit is configured to: Parsing a control flag of the syntax element from the encoded video bitstream, where the control flag is in at least one of a sequence parameter set (SPS) and a picture parameter set (PPS); as well as When the control flag indicates to use the syntax element, the encoded video code stream is parsed to obtain the first number according to the syntax element signaled at the picture level.

14. The device according to claim 12, wherein: The processing circuit is configured to: deriving a second number of the non-zero transform coefficients in the current transform block of the current block; and The first number is adjusted based on the second number of the non-zero transform coefficients.

15. The device according to claim 12, wherein: The processing circuit is configured to: Parsing the absolute value of the transform coefficient of the current transform block from the encoded video code stream; and The first amount is adjusted based on the absolute value of the transform coefficient.

Images