Video decoding method, apparatus, and program, and video encoding method

By employing a Rice parameter coding tool for residual coding in transform skip mode, the method optimizes video encoding and decoding processes, addressing redundancy and compression challenges, thereby enhancing efficiency and reducing bandwidth and storage needs.

JP7592969B2Active Publication Date: 2024-12-03TENCENT AMERICA LLC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023534916
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2022-04-29
Publication Date
2024-12-03
Estimated Expiration
2042-04-29

AI Technical Summary

Technical Problem

Existing video encoding and decoding technologies face challenges in efficiently reducing redundancy and optimizing compression techniques, particularly in intra-prediction and motion compensation, leading to suboptimal bandwidth and storage requirements.

Method used

The implementation of a Rice parameter coding tool for residual coding in transform skip mode, controlled by syntax elements within the bitstream, to enable/disable specific coding tools based on picture-level constraints, thereby optimizing compression efficiency.

Benefits of technology

This approach enhances compression efficiency by adaptively managing coding tools, reducing redundancy, and improving bandwidth and storage requirements without compromising video quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007592969000001
    Figure 0007592969000001
  • Figure 0007592969000002
    Figure 0007592969000002
  • Figure 0007592969000003
    Figure 0007592969000003
Patent Text Reader

Abstract

Aspects of the disclosure provide methods and apparatus for video data processing. In some examples, the apparatus for video data processing includes a processing circuit. For example, the processing circuit determines a first syntax element for coding control in a first range of coded video data in a bitstream. The first syntax element is associated with a coding tool for slice-based Rice coding for residual coding in transform skip mode. The coding tool provides Rice parameters at the slice level for residual coding in transform skip mode. In response to the first syntax element being a first value indicating disabling of the coding tool in the first range, the processing circuit decodes the first range of coded video data, including one or more second ranges of coded video data, without invoking the coding tool.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] [Incorporated by reference] This patent application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 250,180, filed September 29, 2021, and entitled “TECHNIQUES FOR CONSTRAINT FLAG SIGNALING FOR RANGE EXTENSION WITH TRANSFORM RESIDUAL RICE CODING,” which claims the benefit of priority to U.S. Provisional Patent Application No. 17 / 710,794, filed May 31, 2022, and entitled “TECHNIQUES FOR CONSTRAINT FLAG SIGNALING FOR RANGE EXTENSION WITH RICE CODING,” the disclosures of which are incorporated herein by reference in their entirety.

[0002] [Technical field] This disclosure describes embodiments generally relating to video coding. [Background technology]

[0003] The background description provided herein is intended to generally present the context of the disclosure. The work of the currently named inventors is not admitted expressly or impliedly as prior art to the present disclosure, to the extent that that work is described in this Background section, and any aspects of the description that may not otherwise qualify as prior art at the time of filing.

[0004] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luminance samples and associated chrominance samples. The sequence of pictures can have a fixed or variable picture rate (commonly known as frame rate) of, for example, 60 pictures per second, i.e., 60 Hz. Uncompressed video has certain bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luminance sample resolution at a frame rate of 60 Hz) at 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 Gbytes of storage space.

[0005] One goal of video encoding and decoding can be the reduction of redundancy in the input video signal through compression. Compression can help reduce the above bandwidth and / or storage space requirements by more than two orders of magnitude in some cases. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be the same as the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for the intended application. In the case of video, lossy compression is widely used. The amount of tolerable distortion depends on the application, e.g., a user of a particular consumer streaming application may tolerate higher distortion than a user of a television distribution application. The achievable compression ratio can reflect that a higher tolerable / acceptable distortion may result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transformation, quantization, and entropy coding.

[0007] Video codec techniques may include a technique known as intra-coding, in which sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. If a whole block of samples is coded in intra mode, the picture may be an intra picture. Intra pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and thus may be used as the first picture in a coded bitstream and video session or as still images. Samples of an intra block may undergo a transform, and the transform coefficients may be quantized before entropy coding. Intra prediction may be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value after the transform and the smaller the AC coefficients, the fewer bits are needed for a given quantization step size to represent the block after entropy coding.

[0008] Conventional intra-coding, for example as known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to do so from surrounding sample data and / or metadata obtained during the encoding / decoding of spatially adjacent and preceding blocks of data in the decoding order. Such techniques are hereafter referred to as "intra-prediction" techniques. It should be noted that at least in some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from the reference picture.

[0009] There may be many different forms of intra-prediction. If more than one such technique is available for a given video coding technique, the technique in use may be coded in an intra-prediction mode. In certain cases, a mode may have sub-modes and / or parameters, which may be coded independently or may be included in a mode codeword. Since which codeword to use for a given mode / sub-mode / parameter combination may affect the coding efficiency gain through intra-prediction, entropy coding techniques may be used to convert the codeword into a bitstream.

[0010] A specific mode of intra prediction was introduced by H.264, refined in H.265, and further refined in newer coding techniques such as Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). A predictor block may be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of the neighboring samples are copied into the predictor block according to the direction. The reference of the direction in use may be coded into the bitstream or may itself be predicted.

[0011] Referring to FIG. 1A, at the bottom right, a subset of 9 known predictor directions from the 33 possible predictor directions of H.265 (corresponding to the 33 angle modes of the 35 intra modes) is represented. The point where the arrows converge (101) corresponds to the sample being predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples to the upper right and at an angle of 45 degrees from the horizontal. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the lower left of sample (101) and at an angle of 22.5 degrees from the horizontal.

[0012] Still referring to FIG. 1A, at the top left is depicted a square block (104) of 4×4 samples (indicated by the thick dashed line). The square block (104) contains 16 samples, with each sample labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the block is 4×4 samples in size, S44 is at the bottom right. Also shown are reference samples that follow a similar numbering scheme. The reference samples are labeled with "R" and their Y position (e.g., row index) and X position (column index) relative to the block (104). In both H.264 and H.265, the predicted samples are adjacent to the block being reconstructed, so negative values ​​do not need to be used.

[0013] Intra-picture prediction can work by copying reference sample values ​​from neighboring samples as necessary depending on the signaled prediction direction. For example, suppose the coded video bitstream includes signaling for this block indicating a prediction direction consistent with arrow (102), i.e., the sample is predicted from one or more prediction samples in the upper right corner at an angle of 45 degrees from the horizontal. In that case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then sample S44 is predicted from reference sample R08.

[0014] In certain cases, values ​​of multiple reference samples may be combined, for example through interpolation, to calculate a reference sample, particularly when the orientation is not equally divisible by 45 degrees.

[0015] The number of possible directions has increased as video coding techniques have developed. In H.264 (2003), nine different directions could be represented. That increased to 33 in H.265 (2013), and JEM / VVC / BMS can support up to 65 directions at the time of this disclosure. Experiments have been performed to identify the most likely directions, and certain techniques in entropy coding are used to represent those likely directions with a small number of bits, while accepting some penalty for less likely directions. Furthermore, the direction itself can sometimes be predicted from neighboring directions used in neighboring, already decoded blocks.

[0016] FIG. 1B shows a schematic diagram (180) representing 65 intra prediction directions according to JEM to illustrate the growing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits in a coded video bitstream representing directions can vary from one video coding technique to another, and can range, for example, from a simple direct mapping of prediction directions to intra-prediction modes to codewords, to complex adaptation schemes including most-probable modes, and similar techniques. In all cases, however, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Since the goal of video compression is to reduce redundancy, these less likely directions will be represented by more bits than more likely directions in a video coding technique that works well.

[0018] Motion compensation can be a lossy compression technique and can refer to a technique used for the prediction of a newly reconstructed picture or picture portion after blocks of sample data from a previously reconstructed picture or portion thereof (reference picture) are spatially shifted in a direction indicated by a motion vector (hereafter MV). In some cases, the reference picture can be the same as the picture currently being reconstructed. The MV can have two dimensions X and Y, or three dimensions, the third dimension being an indication of the reference picture in use (the latter can indirectly be a temporal dimension).

[0019] In some video compression techniques, the MV applicable to a particular area of ​​sample data can be predicted from other MVs, e.g., from those related to other areas of sample data that are spatially adjacent to the area being reconstructed and that precede that MV in the decoding order. Doing so can significantly reduce the amount of data required to code the MV, thereby removing redundancy and increasing compression. For example, when coding an input video signal obtained from a camera (known as natural video), MV prediction can work because there is a statistical possibility that areas larger than the area to which a single MV is applicable move in a similar direction, and therefore, in some cases, can be predicted using similar motion vectors derived from the MVs of neighboring areas. As a result, the MV determined for a given area is similar or the same as the MV predicted from the surrounding MVs, and after entropy coding, can be represented with fewer bits than would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from an original signal (i.e., a sample stream). In other cases, the MV prediction itself can be lossy, e.g., due to rounding errors when calculating a predictor from several surrounding MVs.

[0020] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H265, “High Efficiency Video Coding”, December 2016). Among the many MV prediction mechanisms proposed by H.265, a technique hereafter called “spatial merge” is described in this specification.

[0021] Referring to Figure 2, a current block (201) has samples that are found by the encoder during the motion search process to be predictable from a previous block of the same size that is spatially shifted. Instead of coding its MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example, from the most recent reference picture (in decoding order), using the MV associated with any one of five surrounding samples denoted as A0, A1 and B0, B1, B2 (202 to 206, respectively). In H.265, MV prediction can use predictors from the same reference picture that neighboring blocks use. Summary of the Invention

[0022] Aspects of the disclosure provide methods and apparatus for video data processing. In some examples, the apparatus for video data processing includes a processing circuit. For example, the processing circuit determines a first syntax element for coding control in a first range of coded video data in a bitstream. The first syntax element is associated with a coding tool of slice-based Rice coding for residual coding in a transform skip mode. The coding tool, in some examples, provides Rice parameters at a slice level for residual coding in a transform skip mode. In response to the first syntax element being a first value indicating disabling of the coding tool in the first range, the processing circuit decodes the first range of coded video data including one or more second ranges of coded video data without invoking the coding tool.

[0023] In some embodiments, the first syntax element is included in general constraint information for coding control of pictures in an output layer set. In some examples, a first value of the first syntax element indicates disabling a coding tool in each Coded Layer Video Sequence (CLVS) in the output layer set.

[0024] In some embodiments, in response to the first syntax element being the first value, the processing circuit constrains a second syntax element for coding control of a coding layer video sequence (CLVS) in the bitstream to have a value indicating not to invoke a coding tool to decode the CLVS. In some examples, the value of the second syntax element indicates the absence of a slice header flag associated with the coding tool in a slice header of a slice in the picture of the CLVS.

[0025] In some embodiments, in response to the first syntax element being a second value and transform skip mode being enabled in a coding layer video sequence (CLVS), the processing circuit decodes a value of a second syntax element in a sequence parameter set (SPS) referenced by the CLVS. The second syntax element indicates the presence / absence of a slice-level syntax element indicating a selection of a Rice parameter. In response to the value of the second syntax element indicating the presence of a slice-level syntax element, in one example, the processing circuit decodes a third syntax element from a slice header of the slice, the third syntax element indicating a selection of a Rice parameter for residual coding of the slice in transform skip mode.

[0026] In some embodiments, in response to the second syntax element not being present in the SPS of the CLVS, the processing circuit infers a value of the second syntax element that indicates non-use of a coding tool in the CLVS.

[0027] In some examples, the processing circuit decodes a first syntax element from the syntax structure in response to a syntax element in the syntax structure for the general constraint information indicating additional bits for the general constraint information in the syntax structure.

[0028] Aspects of the disclosure also provide a non-transitory computer-readable storage medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform a method for video decoding.

[0029] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0030] [Figure 1A] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes. [Figure 1B] FIG. 2 is an illustration of an exemplary intra-prediction direction. [Diagram 2] FIG. 2 is a schematic diagram of a current block and its surrounding spatial merging candidates in one example. [Diagram 3] FIG. 1 is a schematic block diagram of a communication system (300) according to an embodiment. [Figure 4] FIG. 1 is a schematic block diagram of a communication system (400) according to an embodiment. [Diagram 5] FIG. 2 is a schematic block diagram of a decoder according to an embodiment; [Figure 6] FIG. 2 is a schematic block diagram of an encoder according to an embodiment. [Figure 7] 4 shows a block diagram of an encoder according to another embodiment; [Figure 8] 4 shows a block diagram of a decoder according to another embodiment; [Figure 9] 1 illustrates an example of signaling of adaptive resolution change (ARC) parameters according to an embodiment of the present disclosure. [Figure 10]1 shows an example of a table (1000) for mapping of upsampling or downsampling coefficients, codewords, and Ext-Golomb codes. [Figure 11] 1 illustrates several examples of ARC parameter signaling according to embodiments of the present disclosure. [Figure 12] 1 illustrates an example syntax structure of a set of PTL syntax elements in several examples. [Figure 13] 13 shows an example syntax structure of general constraint information in some examples. [Figure 14A] 1 illustrates an example of PTL information including a PTL syntax structure and a general constraint information syntax structure according to some embodiments of the present disclosure. [Figure 14B] 1 illustrates an example of PTL information including a PTL syntax structure and a general constraint information syntax structure according to some embodiments of the present disclosure. [Figure 15A] 1 illustrates an example of a general constraint information syntax structure according to an embodiment of the present disclosure. [Figure 15B] 1 illustrates an example of a general constraint information syntax structure according to an embodiment of the present disclosure. [Figure 16] 1 illustrates a syntax structure of general constraint information according to some embodiments of the present disclosure. [Figure 17] 1 illustrates an example syntax structure for a sequence parameter set (SPS) range extension according to some embodiments of the present disclosure. [Figure 18] 1 shows a flowchart illustrating a process according to an embodiment of the present disclosure. [Figure 19] 1 shows a flowchart illustrating a process according to an embodiment of the present disclosure. [Figure 20] FIG. 1 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] FIG. 3 illustrates a simplified block diagram of a communication system (300) according to an embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) that are interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) perform a one-way transmission of data. For example, the terminal device (310) may code video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data to recover the video pictures, and display the video pictures according to the recovered video data. One-way data transmission may be common, such as in media serving applications.

[0032] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) performing bidirectional transmission of coded video data that may occur, for example, during a video conference. For the bidirectional transmission of data, in the example, each of the terminal devices (330) and (340) may code video data (e.g., a stream of video pictures captured by that terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) may also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures on an accessible display device in accordance with the recovered video data.

[0033] In the example of FIG. 3, the terminal devices (310), (320), (330), and (340) may be represented as a server, a personal computer, and a smartphone, although the principles of the present disclosure may not be so limited. The embodiments of the present disclosure find use with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (350) represents any number of networks that convey coded video data between the terminal devices (310), (320), (330), and (340), including, for example, wireline and / or wireless communication networks. The communication network (350) may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of the network (350) may be irrelevant to the operation of the present disclosure, unless otherwise described hereinafter.

[0034] 4 illustrates the placement of a video encoder and a video decoder in a streaming environment as an example application of the disclosed subject matter. The disclosed subject matter can be similarly applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0035] The streaming system may include a capture subsystem (413) that may include, for example, a video source (401), e.g., a digital camera, that generates a stream of uncompressed video pictures (402). In an example, the stream of video pictures (402) includes samples taken by the digital camera. The stream of video pictures (402) is represented by a bold line to emphasize its high data volume compared to the encoded video data (404) (or coded video bitstream), which may be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or coded video bitstream (404)), is represented by a thin line to emphasize its lower data volume compared to the stream of video pictures (402), which may be stored in a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) of FIG. 4, can access the streaming server (405) to retrieve copies (407) and (409) of the encoded video data (404). The client subsystem (406) can include a video decoder (410), for example in an electronic device (430). The video decoder (410) decodes an incoming copy (407) of the encoded video data and generates an outgoing stream (411) of video pictures that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. An example of such a standard is ITU-T Recommendation H.265.In an example, the developing video coding standard is commonly known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in conjunction with VVC.

[0036] The electronics 420 and 430 may include other components (not shown). For example, the electronics 420 may include a video decoder (not shown), and the electronics 430 may also include a video encoder (not shown).

[0037] 5 shows a block diagram of a video decoder (510) according to an embodiment of the present disclosure. The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., a receiving circuit). The video decoder (510) may be used in place of the video decoder (410) of the example of FIG. 4.

[0038] The receiver (531) may receive one or more coded video sequences to be decoded by the video decoder (510), one coded video sequence at a time, in the same or other embodiments, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device storing the coded video data. The receiver (531) may receive the coded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to their respective use entities (not shown). The receiver (531) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / parser (520) (hereafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In others, it can be external to the video decoder (510) (not shown). In still others, there can be a buffer memory (not shown) external to the video decoder (510), e.g., to combat network jitter, plus another buffer memory (515) within the video decoder (510), e.g., to manipulate playback timing. When the receiver (531) is receiving data from a storage / forwarding device of sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory (515) may not be needed or can be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be needed and can be relatively large, advantageously of adaptive size, and at least partially implemented in an operating system or similar element (not shown) external to the video decoder (510).

[0039] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (510) and information for controlling a rendering device such as a render device (512) (e.g., a display screen) that is not an essential part of the electronic device (530) but may be coupled to the electronic device (530) as shown in FIG. 5. The control information for the rendering device may take the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, context-dependent or non-context-dependent arithmetic coding, etc. The parser (520) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information from the coded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0040] The parser (520) may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory (515) to generate symbols (521).

[0041] The reconstruction of the symbols (521) can have many different units depending on the type of coded video picture or portion thereof (e.g., inter and intra pictures, inter and intra blocks) and other factors. Which units are included and how may be controlled by subgroup control information parsed by the parser (520) from the coded video sequence. The flow of such subgroup control information between the parser (520) and the following units is not shown for clarity.

[0042] Beyond the functional blocks already mentioned, the video decoder (510) may be conceptually subdivided into a number of functional units, which are described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is adequate.

[0043] The first unit is a scalar / inverse transform unit (551), which receives the quantized transform coefficients as symbols (521) from the parser (520) along with control information including which transform to use, block size, quantization factor, quantization scaling matrix, etc. The scalar / inverse transform unit (551) can output blocks containing sample values ​​that can be input to an aggregator (555).

[0044] In some cases, the output samples of the scaler / inverse transformer (551) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from a current picture buffer (558). The current picture buffer (558) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (555) adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transformer unit (551).

[0045] In other cases, the output samples of the scalar / inverse transform unit (551) may relate to an inter-coded and potentially motion-compensated block. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols (521) related to the block, the samples may be added by the aggregator (555) to the output of the scalar / inverse transform unit (551) (in this case referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensated prediction unit (553) fetches prediction samples may be controlled by motion vectors available to the motion compensated prediction unit (553), for example in the form of symbols (521), which may have X, Y, and reference picture components. Motion compensation can also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are used, as well as motion vector prediction mechanisms.

[0046] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). Video compression techniques may include in-loop filter techniques that are controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the parser (520), but may also be responsive to meta-information obtained during the decoding of previous portions (in decoding order) of the coded picture or coded video sequence, or even to previously constructed loop filtered sample values.

[0047] The output of the loop filter unit (556) can be a sample stream that can be output to a render device (512) and further stored in a reference picture memory (557) for use in future inter-picture prediction.

[0048] Once a particular coded picture is fully reconstructed, it may be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) may become part of the reference picture memory (557), and any unused current picture buffer may be reallocated before beginning reconstruction of a subsequent coded picture.

[0049] The video decoder (510) may perform decoding operations according to a given video compression technique in a standard, such as ITU-T Recommendation H.265. The coded video sequence may conform to a syntax prescribed by the video compression technique or standard in use, in the sense that the coded video sequence conforms to both the syntax of the video compression technique or standard and a profile documented in the video compression technique or standard. In particular, a profile may select a particular tool from all tools available in the video compression technique or standard as the only tool available for use under that profile. Also, for compliance, the complexity of the coded video sequence must be within the bounds defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples / second), maximum reference picture size, etc. The limits set by the levels may in some cases be further restricted through Hypothetical Reference Decoder (HRD) specifications and metadata for HRD buffer management signaled in the coded video sequence.

[0050] In an embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0051] 6 shows a block diagram of a video encoder (603) according to an embodiment of the present disclosure. The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., a transmission circuit). The video encoder (603) may be used in place of the example video encoder (403) of FIG. 4.

[0052] The video encoder (603) may receive video samples from a video source (601) (which is not part of the electronic device (620) in the example of FIG. 6) that may capture video images to be coded by the video encoder (603). In other examples, the video source (601) is part of the electronic device (620).

[0053] The video source (601) may provide a source video sequence to be coded by the video encoder (603) in the form of a digital video sample stream that can be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCB, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media serving system, the video source (601) may be a storage device that stores pre-prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed in sequence, impart motion. The picture itself may be organized as a spatial array of pixels, each of which may have one or more samples depending on the sampling structure, color space, etc., in use. Those skilled in the art can easily understand the relationship between pixels and samples. The remainder of this specification focuses on samples.

[0054] According to an embodiment, the video encoder (603) may code and compress pictures of a source video sequence into a coded video sequence (643) in real-time or under any other time constraints required by the application. Imposing an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. Couplings are not shown for clarity. Parameters set by the controller (650) may include parameters related to rate control (picture skip, quantizer, lambda value for rate-distortion optimization techniques, etc.), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured with other appropriate functions related to the video encoder (603) optimized for a particular system design.

[0055] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As an oversimplified description, in an example, the coding loop can include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder (633) reconstructs the symbols to generate sample data in a similar manner that a (remote) decoder would also generate (given that any compression between the symbols and the coded video stream is lossless in the video compression techniques contemplated in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces bit-perfect results independent of the location of the decoder (local or remote), the content in the reference picture memory (634) is also bit-perfect between the local and remote encoders. That is, the prediction part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronicity (and the resulting drift when synchronicity cannot be maintained, e.g., due to channel errors) is also used in several related technologies.

[0056] The operation of the "local" decoder (633) can be the same as a "remote" decoder, such as the video decoder (510), already described in detail above in conjunction with Figure 5. Referring also momentarily to Figure 5, however, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (645) and parser (520) can be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), may not be fully implemented in the local decoder (633).

[0057] At this point, it can be observed that any decoder technique, other than parsing / entropy decoding, present in the decoder must necessarily be present in roughly the same functional form in the corresponding encoder. For this reason, the disclosed subject matter focuses on the operation of the decoder. Descriptions of the encoder techniques may be omitted, since they are the inverse of the decoder techniques described generically. Only to certain extents are more detailed descriptions required, which are given below.

[0058] In operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as "reference pictures." In this manner, the coding engine (632) codes differences between pixel blocks of the reference pictures and pixel blocks of the input picture that may be selected as predictive references for the input picture.

[0059] The local video decoder (633) may decode the coded video data of pictures that may be designated as reference pictures based on the symbols generated by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data may be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence may usually be a copy of the source video sequence with some errors. The local video decoder (633) may reproduce the decoding process that may be performed by the video decoder on the reference pictures, causing the reconstructed reference pictures to be stored in the reference picture cache (634). In this way, the video encoder (603) may locally store copies of reconstructed reference pictures that have common content with the reconstructed reference pictures that would be obtained by the far-end video decoder (without transmission errors).

[0060] The predictor (635) may perform a prediction search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may look for certain metadata, such as reference picture motion vectors, block shapes, or sample data (as candidate reference pixel blocks) from the reference picture memory (634) that may be suitable prediction references for the new picture. The predictor (635) may operate on a sample block-by-pixel block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (635), the input picture may have prediction references derived from multiple reference pictures stored in the reference picture memory (634).

[0061] The controller (650) may manage the coding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0062] The output of all the above functional units may undergo entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0063] The transmitter (640) may buffer the coded video sequence produced by the entropy coder (645) to prepare it for transmission over a communication channel (660), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (640) may merge the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0064] A controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types:

[0065] An Intra Picture (I-Picture) may be a picture that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow various types of Intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of such variations of I-pictures and their respective applications and characteristics.

[0066] A Predictive Picture (P-picture) may be a picture that can be encoded and decoded by intra- or inter-prediction using at most one motion vector and reference index to predict the sample values ​​of each block.

[0067] A Bi-directionally Predictive Picture (B-picture) may be a picture that can be coded and decoded by intra- or inter-prediction using at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple-predictive picture(s) can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0068] A source picture may generally be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to each picture of the block. For example, blocks of I pictures may be non-predictively coded or they may be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of P pictures may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of B pictures may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0069] The video encoder (603) may perform coding operations according to a given video coding technique or standard, such as ITU-T Recommendation H.265. During its operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax defined by the video coding technique or standard being used.

[0070] In an embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other types of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0071] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation in a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In an example, a particular picture being coded / decoded, called the current picture, is partitioned into blocks. If a block in the current picture is similar to a reference block in a reference picture that was coded earlier in the video and is still buffered, then that block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in a reference picture, and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0072] In some embodiments, bi-prediction techniques may be used in inter-picture prediction. According to bi-prediction techniques, two reference pictures are used, e.g., a first reference picture and a second reference picture, both preceding in decoding order (but may be past and future, respectively, in display order) the current picture in the video. A block in the current picture may be coded with a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. The block is predictable by a combination of the first and second reference blocks.

[0073] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0074] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively quad-tree partitioned into one or more coding units (CUs). For example, a CTU of 64×64 pixels may be partitioned into one CU of 64×64 pixels, or four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In an example, each CU is analyzed to determine a prediction type for that CU, such as an inter prediction type or an intra prediction type. A CU is divided into one or more Prediction Units (PUs) according to temporal and / or spatial predictability. In general, each PU includes one luma prediction block (PB) and two chroma PBs. In an embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0075] 7 shows a diagram of a video encoder (703) according to another embodiment of the disclosure. The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture included in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. In an example, the video encoder (703) is used in place of the video encoder (403) of the example of FIG. 4.

[0076] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8x8 samples. The video encoder (703) determines whether the processing block is best coded in intra mode, inter mode, or bi-prediction mode, for example using rate-distortion optimization. If the processing block is to be coded in intra mode, the video encoder (703) may use intra prediction techniques to encode the processing block into a coded picture, and if the processing block is to be coded in inter mode or bi-prediction mode, the video encoder (703) may use inter prediction or bi-prediction techniques, respectively, to encode the processing block into a coded picture. In certain video coding techniques, the merge mode can be an inter-picture prediction sub-mode in which motion vectors are derived from one or more motion vector predictors without benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In an example, the video encoder (703) includes other components, such as a mode decision module (not shown) that determines the mode of the processing block.

[0077] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), coupled as shown in FIG.

[0078] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture), generate inter-prediction information (e.g., a description of redundant information according to an inter-coding technique, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that has been decoded based on the coded video information.

[0079] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), in some cases compare the block to previously coded blocks in the same picture, and generate transformed quantized coefficients and, in some cases, intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques). In an example, the intra encoder (722) also calculates an intra prediction result (e.g., a prediction block) based on the intra prediction information and a reference block in the same picture.

[0080] The generic controller (721) is configured to determine generic control data and control other components of the video encoder (703) based on the generic control data. In an example, the generic controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, if the mode is an intra mode, the generic controller (721) controls the switch (726) to select the intra mode result for use by the residual calculator (723), and controls the entropy encoder (725) to select intra prediction information and include the intra prediction information in the bitstream. If the mode is an inter mode, the generic controller (721) controls the switch (726) to select the inter prediction result for use by the residual calculator (723), and controls the entropy encoder (725) to select inter prediction information and include the inter prediction information in the bitstream.

[0081] The residual calculator (723) is configured to calculate a difference (residual data) between the received block and a prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) is configured to operate on the residual data to encode the residual data and generate transform coefficients. In an example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be used by the intra-encoder (722) and the inter-encoder (730) as appropriate. For example, the inter-encoder (730) can generate decoded blocks based on the decoded residual data and the inter-prediction information, and the intra-encoder (722) can generate decoded blocks based on the decoded residual data and the intra-prediction information. The decoded blocks are appropriately processed to generate decoded pictures, which may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0082] The entropy encoder (725) is configured to format a bitstream to include the encoded block. The entropy encoder (725) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In an example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. It should be noted that when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode in accordance with the disclosed subject matter, the residual information is not present.

[0083] 8 shows a diagram of a video decoder (810) according to another embodiment of the disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In an example, the video decoder (810) is used in place of the video decoder (410) of the example of FIG. 4.

[0084] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872), coupled as shown in FIG.

[0085] The entropy decoder (871) may be configured to reconstruct from the coded picture certain symbols representing syntax elements from which the coded picture is constructed. Such symbols may include, for example, prediction information (e.g., intra- or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra- or bi-prediction mode in merge or other submodes), the specific samples or metadata used for prediction by the intra decoder (872) or the inter decoder (880), respectively, residual information in the form of, for example, quantized transform coefficients, etc. In an example, if the prediction mode is an inter or bi-prediction mode, the inter prediction information is provided to the inter decoder (880), and if the prediction type is an intra prediction type, the intra prediction information is provided to the intra decoder (872). The residual information may undergo inverse quantization and is provided to the residual decoder (873).

[0086] The inter decoder (880) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0087] The intra decoder (872) is configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0088] The residual decoder (873) is configured to perform inverse quantization to retrieve inverse quantized transform coefficients, and process the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information (to include quantization parameters (QPs)), which may be provided by the entropy decoder (871) (datapath not shown since this is only low volume control information).

[0089] The reconstruction module (874) is configured to combine the residual output by the residual decoder (873) and the prediction result (possibly output by an inter- or intra-prediction module) in the spatial domain to form a reconstructed block. The reconstructed block may be part of a reconstructed picture, which in turn may be part of a reconstructed video. Note that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0090] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technology. In some embodiments, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0091] Aspects of the present disclosure provide coding tools and feature control techniques that include constraint flags in a coded video stream.

[0092] According to aspects of the present disclosure, the picture size in the bitstream can remain the same or may change. In some related examples, video encoders and decoders can operate on a given picture size that remains constant defined for a Coded Video Sequence (CVS), Group Of Pictures (GOP), or similar multi-picture time frame. In one example, such as MPEG-2, system designs are known to change the horizontal resolution (and thus the picture size) depending on factors such as scene activity, but only for I-pictures, and therefore the picture size is typically defined and remains constant for a GOP. Resampling of reference pictures for use of different resolutions in a CVS is known, for example from ITU-T Recommendation H.263 Annex P. However, the picture size in a CVS does not change, and only the reference pictures are resampled, which may result in only a portion of the picture canvas being used (e.g., in the case of downsampling) or only a portion of the scene being captured (in the case of upsampling). In some instances, such as H.263 Annex Q, resampling of individual macroblocks by a factor of 2 in each dimension (e.g., up or down) is permitted, but the picture size remains the same. When the size of the macroblocks can be fixed, for example in H.263, the size of the macroblocks does not need to be signaled.

[0093] In some related examples, the picture size at the predicted picture can vary. Examples such as VP9 allow reference picture resampling and resolution change of the whole picture. Some examples (e.g., Hendry, et. al, “On adaptive resolution change (ARC) for VVC”, Joint Video Team document JVET-M0135-v1, January 9, 2019; incorporated herein by reference in its entirety) allow resampling of the whole reference picture to a different resolution (e.g., higher or lower resolution). The different candidate resolutions can be coded in the Sequence Parameter Set (SPS) and referenced by per-picture syntax elements in the Picture Parameter Set (PPS).

[0094] In accordance with aspects of the present disclosure, a source video can be compressed by layered coding, which can encode a picture into a bitstream that includes one or more layers with different qualities, such as different resolutions. The bitstream can have syntax elements that specify which layers (or sets of layers) can be output at the decoder side. The set of layers that are output can be defined as an output layer set. For example, in a video codec that supports multiple layers and scalability, one or more output layer sets can be signaled in a video parameter set (VPS). Syntax elements that specify the profile tier level (PTL) of the entire bitstream or one or more output layer sets can be signaled in a VPS, a Decoder Parameter Set (DPS), which in some examples may be referred to as Decode Capability Information (DCI), an SPS, a PPS, an SEI message, etc. In the PTL information, there can be general constraint information that can specify constraints on coding tools or functions. It is desirable to efficiently represent and signal constraint information for various coding tools and functions.

[0095] In some examples, the term "subpicture" may be used to refer to a rectangular arrangement of samples, blocks, macroblocks, coding units, or similar entities that may be semantically grouped together and coded independently at a changed resolution, for example. One or more subpictures may form a picture. One or more coded subpictures may form a coded picture. One or more subpictures may be assembled into a picture, and one or more subpictures may be extracted from a picture. In some examples, one or more coded subpictures may be assembled into a coded picture in the compressed domain without transcoding to the sample level. In some examples, one or more coded subpictures may be extracted from a coded picture in the compressed domain.

[0096] In some examples, a mechanism that allows changing the resolution of a picture or subpicture in a CVS, for example by reference picture resampling, may be referred to as an Adaptive Resolution Change (ARC). The control information used to perform the adaptive resolution change may be referred to as ARC parameters. The ARC parameters may include filter parameters, scaling factors, resolution of output and / or reference pictures, various control flags, and / or others.

[0097] In some examples, ARC encoding / decoding is picture-based, so a set of control information (ARC parameters) is used for encoding / decoding signal- and semantically independent coded video pictures. In some examples, ARC encoding / decoding is sub-picture-based, so multiple sub-pictures within a picture can be encoded / decoded with independent ARC parameters. It should be noted that ARC parameters can be signaled by various techniques.

[0098] 9 illustrates examples (e.g., options) of signaling techniques for ARC parameters according to some embodiments of the present disclosure. Coding efficiency, complexity, and architecture can vary from example to example. A video coding standard or technology may select one or more examples or other variations for signaling ARC parameters. The examples may not be mutually exclusive and may be interchanged based on application needs, standard technology, encoder choice, and / or the like.

[0099] In accordance with aspects of the disclosure, the ARC parameters may be provided as classes of ARC parameters in various ways. In some examples, the classes of ARC parameters include separate or combined upsample and / or downsample coefficients in the X and Y dimensions. In one example, one or more short syntax elements may be coded that can point to tables that include the upsample and / or downsample coefficients.

[0100] In some examples, the class of ARC parameters includes upsample and / or downsample coefficients, and a time dimension is added to indicate a constant rate of zooming in and / or out for a given number of pictures. In one example, one or more short syntax elements may be coded that can point to a table that includes the upsample and / or downsample coefficients to which the time dimension is added.

[0101] In some examples, the classes of ARC parameters include the resolution of the X or Y dimensions of the input picture, output picture, reference picture, coded picture, at a sample, block, macroblock, CU, or other suitable granularity, either in combination or separately. In some examples, there is more than one resolution used in video coding (e.g., one resolution for the input picture and another resolution for the reference picture), and a set of values ​​(corresponding to one of the resolutions) can be inferred from another set of values ​​(corresponding to the other one of the resolutions). The determination of the values ​​can be gated, for example, based on the use of a flag. The use of flags for gating is described in more detail below.

[0102] In some examples, the ARC parameter class includes warping coordinates similar to those used in H.263 Annex P, with appropriate granularity as described above. H.263 Annex P defines an efficient method for coding the warping coordinates. Other efficient methods can be devised. For example, the variable length reversible Huffman style coding of the warping coordinates of Annex P can be replaced with an appropriate length binary coding, where the binary codewords can be determined by multiplying the maximum image size by a factor to offset values ​​to allow warping outside the bounds of the maximum image size.

[0103] In some examples, the class of ARC parameters includes upsample and / or downsample filter parameters. In one example, there is only a single filter for upsampling and / or downsampling. In other examples, multiple filters may be used. In some examples, the filter parameters may be signaled to make the filter design more flexible. The filter parameters may be selected by using an index among a list of possible filter designs. The filters may be fully specified (e.g., by specifying a list of filter coefficients using an appropriate entropy coding technique), the filters may be implicitly selected by the upsample and / or downsample ratios signaled according to any of the above mechanisms, and / or may be otherwise specified / selected.

[0104] In the following description, a finite set of upsample and / or downsample coefficients (the same coefficients used in both X and Y dimensions) is used to describe the signaling of ARC parameters by codewords. In some examples, the codewords may be variable length coded, for example, using Ext-Golomb codes for certain syntax elements in video coding standards (e.g., H.264 and H.265).

[0105] FIG. 10 shows an example of a table (1000) for mapping of upsampled or downsampled coefficients, codewords, and Ext-Golomb codes.

[0106] It should be noted that other similar tables may be derived according to the application and capabilities of up-scaling and down-scaling mechanisms available in video compression techniques or standards. In some examples, Table 1 may be appropriately extended to additional values. It should be noted that values ​​may be represented by entropy coding mechanisms other than Ext-Golomb codes, for example, by using binary coding. In one example, entropy coding mechanisms other than Ext-Golomb codes may have certain advantages when the resampling factor is important outside the video processing engines (e.g., encoders and decoders), for example, to a Media-Aware Network Element(s), MANE. In some examples, a short Ext-Golomb code (e.g., only a single bit shown in Table 1) may be selected when no resolution change is required (e.g., the original / target resolution is 1 in Table 1). This may have coding efficiency advantages over, for example, using binary codes in the most general case.

[0107] In accordance with aspects of the present disclosure, a mapping table such as Table 1 is configurable. For example, the number of entries in Table 1 and the corresponding semantics are fully or partially configurable. In some examples, the basic outline of the mapping table is conveyed in a high-level parameter set such as an SPS or a DPS. Alternatively, or additionally, in some examples, one or more tables similar to Table 1 may be defined in a video coding technology or standard, and one of the tables may be selected, for example, by the SPS or the DPS.

[0108] ARC information, such as upsampled or downsampled coefficients, may be included in a video coding technique or standard syntax, coded as described above. It should be noted that one or more codewords may be used to control other classes of ARC information, such as upsampled or downsampled coefficients. In some instances, very large amounts of data are required for filters or other data structures.

[0109] 9, in an example (910) such as H.263 Annex P, the ARC information (912) can take the form of four warping coordinates and is included in a picture header such as an H.263 PLUSPTYPE (913) header extension. Example (910) may be applied when i) picture headers are in use and ii) frequent changes of ARC information are expected. However, overhead can be high when using H.263-style signaling, such as that shown in example (910), and scaling factors may not be applicable between picture boundaries due to the potentially temporal nature of picture headers.

[0110] Referring to FIG. 9, in an example (920) such as JVCET-M135-v1, ARC reference information (925) (e.g., index) is placed in a PPS (924) and can point to a table (i.e., target resolution table) (926) that contains the target resolutions (e.g., resolutions 1-3). In one example, the table (926) is located in an SPS (927). Placing the target resolution in the table (926) in the SPS (927) can be justified by using the SPS as an interoperability negotiation point during capability exchange. The resolution can vary among a limited set of values ​​(e.g., resolutions 1-3) in the table (926) on a picture-by-picture basis by reference (e.g., ARC reference information (925)) in the appropriate PPS (924).

[0111] 9 also illustrates additional techniques, such as examples 930, 940, and 950, that may be used to convey ARC information in a video bitstream. The techniques may be used individually or in any suitable combination within the same video coding technology or standard.

[0112] 9, in an example (930), the ARC information (939), such as a resampling factor (or zoom factor), may be present in a header, such as a slice header, a GOB header, a tile header, a tile group header, etc. For example, a tile group header (938) is depicted in FIG. 9. The technique depicted by the example (930) can be used when the ARC information (939) can be decoded into a small number of bits, such as a fixed-length codeword of a few bits or a single variable-length ue(v).

[0113] In accordance with aspects of the disclosure, having the ARC information (939) directly in the header (e.g., the tile group header (938) of FIG. 9, a slice header, or a tile header) can have an additional advantage in that the ARC information (939) can be applied to, for example, a sub-picture represented by a corresponding tile group (or slice, tile) rather than to the entire picture. Furthermore, in one example, even if a video compression technology or standard only contemplates adaptive resolution changes for the entire picture (e.g., a tile group or slice, tile), the ARC information (939) can be applied to, for example, a sub-picture represented by a corresponding tile group (or slice, tile). (as opposed to group-based adaptive resolution changes), example (930) may have certain advantages over example (910) from an error resilience standpoint.

[0114] Referring to FIG. 9, in an example (940), the ARC information (942) may reside in a parameter set (941), such as a PPS, a header parameter set, a tile parameter set, an adaptation parameter set (APS), etc. For example, in FIG. 9, an APS (941) is depicted. In some examples, the scope of the parameter set (941) may not be greater than a picture, and may be, for example, a picture, a tile group, etc. The use of the ARC information (e.g., ARC information (942)) may be implicit by the activation of the associated parameter set (e.g., APS (941)). For example, if a video coding technology or standard only contemplates picture-based ARC, then a PPC or equivalent may be appropriate.

[0115] Referring to Figure 9, in an example (950), the ARC reference information (953) may be present in a tile group header (954) or a similar data structure (e.g., a picture header, a slice header, a tile header, or a GOP header), as described above. In Figure 9, a tile group header (954) is shown as an example. The ARC reference information (953) can reference a subset (955) of the ARC information available in a parameter set (956) that has a scope beyond a single picture, e.g., an SPS, a DPS, etc. In Figure 9, an SPS (956) is shown as an example.

[0116] 11 illustrates some examples of ARC parameter signaling according to some embodiments of the present disclosure. FIG. 11 illustrates an example syntax diagram used in a video coding standard. In one example, the notation of the syntax diagram closely follows C-style programming. Bolded lines may indicate syntax elements present in the bitstream, and non-bolded lines may indicate control flow or variable settings.

[0117] Referring to FIG. 11, the tile group header (1101) includes a syntax structure for a header applicable to a portion (e.g., a rectangular portion) of a picture. In one example, the tile group header (1101) can conditionally include a variable-length, Ext-Golomb coded syntax element dec_pic_size_idx (1102) (shown in bold). The presence of a syntax element (e.g., dec_pic_size_idx (1102)) in the tile group header (1101) can be gated based on, for example, an adaptive resolution represented by a flag (e.g., adaptive_pic_resolution_change_flag) (1103). The value of the flag (e.g., adaptive_pic_resolution_change_flag) (1103) is not shown in bold, and thus the flag is present in the bitstream at the point where the flag appears in the syntax diagram. Whether adaptive resolution is in use for a picture or part of a picture may be signaled in a high level syntax structure (eg, SPS (1110) in FIG. 11) in or outside the bitstream.

[0118] Referring to Figure 11, an excerpt of an SPS (1110) is shown. The SPS (1110) includes a first syntax element (1111) that is a flag (1111) (e.g., adaptive_pic_resolution_change_flag). When the flag (1111) is true, the flag (1111) can indicate the use of adaptive resolution, which may require specific control information. In one example, the specific control information is conditionally present based on the value of the flag (1111), as indicated by the if() statements in the SPS (1110) and the tile group header (1101).

[0119] As shown in the example of Figure 11, when adaptive resolution is in use, the output resolution (or output picture resolution) (1113) in units of samples is coded. In one example, the output resolution (1113) is coded based on a width resolution (e.g., output_pic_width_in_luma_samples) and a height resolution (e.g., output_pic_height_in_luma_samples). A video coding technology or standard may define specific limitations on the value of the output resolution (1113). For example, a level definition may limit the number of total output samples (e.g., the product of output_pic_width_in_luma_samples and output_pic_height_in_luma_samples). In some examples, a video coding technology or standard, or an external technology or standard (e.g., a system standard), may restrict the numbering range of the width and / or height resolution (e.g., the width and / or height resolution is divisible by a power of 2), the aspect ratio of the width resolution to the height resolution (e.g., the ratio of the width resolution to the height resolution is 4:3 or 16:9), etc. In one example, the above restrictions may be introduced to facilitate hardware implementation.

[0120] In certain applications, an encoder can instruct a decoder to use a particular reference picture size rather than implicitly assuming that size is the output picture size. For example, a syntax element (e.g., reference_pic_size_present_flag) (1114) gates the conditional presence of a reference picture dimension (1115). The reference picture dimension (1115), in one example, can include both a width (e.g., reference_pic_width_in_luma_samples) and a height (e.g., reference_pic_hight_in_luma_samples).

[0121] Also depicted in Figure 11 are tables of applicable decoded picture widths and heights. In one example, the number of entries in the table may be represented by a table indication (e.g., syntax element num_dec_pic_size_in_luma_samples_minus1) (1116). "minus1" may refer to the interpretation of the value of the syntax element (1116). For example, if the coded value is zero, there is one table entry. If the coded value is five, there are six table entries. For each entry in the table, the decoded picture width and height are included as syntax elements (1117).

[0122] The table entries represented by the syntax element (1117) can be indexed using the syntax element dec_pic_size_idx (1102) in the tile group header (1101), thus allowing different decoding sizes and zoom factors for each tile group.

[0123] In accordance with aspects of the present disclosure, certain video coding techniques or standards (e.g., VP9) may enable spatial scalability by implementing a certain form of reference picture resampling in conjunction with temporal scalability. In an embodiment, the reference pictures are upsampled using ARC-style techniques to a higher resolution to form the base of the spatial enhancement layer. The upsampled pictures may be refined at higher resolutions, for example to add detail, using regular prediction mechanisms (e.g., motion compensated prediction for inter prediction from the reference picture).

[0124] In some examples, values ​​in a Network Abstraction Layer (NAL) unit header, e.g., a Time ID field, are used to indicate spatial layer information as well as temporal layer information. Using values ​​in a NAL unit header to indicate both temporal and spatial layer information can enable the use of existing Selected Forwarding Units (SFUs) for scalable environments without modification. For example, existing SFUs can be created and optimized for forwarding of selected temporal layers based on the Time ID value in the NAL unit header. The existing SFUs can then be used for spatial scalability (e.g., spatial layer selection) without modification in some examples. In some examples, a mapping can be provided between coded pictures and temporal layers indicated by the Time ID field in the Time ID value of the NAL unit header.

[0125] According to aspects of the present disclosure, some characteristics of a coded bitstream may be specified using profile, tier, and level combination (PTL) information, which includes profile, tier, level, and general constraint information. In some examples, a profile defines a subset of bitstream characteristics, such as color reproduction, resolution, additional video compression, etc. A video codec may define various profiles, such as a baseline profile (e.g., a simple profile with a low compression ratio), a high profile (a complex profile with a high compression ratio), a main profile (e.g., a profile with an intermediate compression ratio between the baseline profile and the high profile may be the default profile setting), etc.

[0126] Additionally, tiers and levels may be used to specify certain constraints that define the bitstream in terms of maximum bitrate, maximum luma sample rate, maximum luma picture size, minimum compression ratio, maximum number of slices allowed, maximum number of tiles allowed, etc. Lower tiers are more restrictive than higher tiers and lower levels are more restrictive than higher levels. In one example, the standard may define two tiers: Main and High. The Main tier is a lower tier than the High tier. The tiers are made to handle applications that differ in terms of their maximum bitrate. The Main tier is designed for most applications while the High tier, in one example, is designed for very demanding applications. A level is a set of constraints for a bitstream where the standard may define multiple levels. In one example, for levels below level 4, only the Main tier is allowed. In some examples, a decoder conforming to a given tier / level is required to be able to decode all bitstreams that are encoded for that tier / level and all lower tiers / levels.

[0127] The general constraint information may include constraint information regarding video source type, coding tools and features. For example, the constraint flags may indicate whether inter-coding tools, intra-coding tools, DBF, entropy coding, transform, partitioning (e.g., tiles, slices), buffer management, random access (e.g., IDR), parameter sets (e.g., SPS, PPS), and / or the like are present or used in the coded video bitstream. The constraint information may be signaled in parameter sets (e.g., SPS, VPS, or DCI). The constraint flags may be signaled in high-level syntax structures (e.g., SPS, VPS, DCI).

[0128] In accordance with some embodiments of the present disclosure, the PTL information may relate to a scope (e.g., a portion of the coded video data within a bitstream). In some examples, the PTL information may be defined for, for example, the entire bitstream, the CVS of the bitstream, each Output Layer Set (OLS) of the bitstream, and / or the like, and may be signaled in High-Level Syntax (HLS) structures such as VPS, DPS, DCI, SPS, PPS, APS, GOP, sequence, header, SEI message, etc.

[0129] In some examples, the high level syntax (HLS) is defined with respect to the block level. Block level coding tools can be used to decode pixels or samples within a picture to reconstruct the picture. Block level coding tools can include any suitable coding tools used to reconstruct coding blocks, such as coding tools for inter prediction (i.e., inter coding tools), coding tools for intra prediction (i.e., intra coding tools), adaptive loop filters (ALF), deblocking filters (DBF), entropy coding, transforms, etc.

[0130] High Level Syntax (HLS) can specify information regarding tool and buffer control capabilities, system interfaces, picture level control, etc. For example, HLS can specify partitions (e.g., tiles, slices, subpictures), buffer management, random access (e.g., IDR, Clean Random Access (CRA)), parameter sets (e.g., VPS, SPS, PPS, APS), Reference Picture Resampling (PRP), scalability, and / or the like. High Level Syntax can be above the block level.

[0131] The control information may have an appropriate level, such as SPS-level tool control information, PPS-level tool control information, sequence-level control information, bitstream-level control information, and / or the like. In some examples, the PTL information may be part of the control information and may be signaled as a constraint flag in the HLS structure to indicate tool control or constraints in the range corresponding to the HLS structure. For example, a constraint flag for the PTL information may be provided in one of the sequence-level control information and the bitstream-level control information. In one example, if a particular tool is disabled by a constraint flag in the HLS structure, the tool is not used, for example, for coding a block in the range corresponding to the HLS.

[0132] 12 and 13 show examples of PTL information according to some embodiments of the present disclosure. Fig. 12 shows an example syntax structure of a set of PTL syntax elements (1200), and Fig. 12 shows an example syntax structure of general constraint information (1300).

[0133] In FIG. 12, the set of PTL syntax elements may include general_profile_idc, general_tier_flag, general_level_idc, num_sub_profiles, general_sub_profile_idc, sublayer_level_present_flag, ptl_alignment_zero_bit, and sublayer_level_idc.

[0134] In Figure 13, the general constraint information may include multiple constraint flags. In one example, a constraint flag (e.g., intra_only_constraint_flag) equal to 1 indicates that the parameter sh_slice_type should be I (i.e., a slice that is an intra slice). The parameter sh_slice_type is a parameter in the slice header that specifies the coding type of the slice between types I, P, and B. A constraint flag (e.g., intra_only_constraint_flag) (1305) equal to 0 does not impose a constraint (e.g., sh_slice_type must be I) on all coded pictures within the scope of the PTL information, and other information (e.g., profile_idc) may allow non-intra slices. In another example, a constraint flag (e.g., no_alf_contraint_flag) (1306) equal to 1 can indicate that sps_alf_enabled_flag is equal to 0 for all CVSs within the PTL information, and thus adaptive loop filtering is not used even if adaptive loop filtering is allowed, for example, based on the profile_idc. A constraint flag (e.g., no-alf_contraint_flag) (1306) equal to 0 does not impose the above constraint.

[0135] In another example, a constraint flag (e.g., no_lossless_coding_tool_contraint_flag) (1301) may be signaled in the generic constraint information as shown in FIG. 13. A constraint flag (e.g., no_lossless_coding_tool_contraint_flag) (1301) equal to 1 may indicate that coding tools related to lossless coding are not available within the scope of the PTL information that includes the constraint flag (1301). A constraint flag (e.g., no_lossless_coding_tool_contraint_flag) (1301) equal to 0 does not impose the above constraint.

[0136] In another example, a constraint flag (e.g., no_lossy_coding_tool_constraint_flag) (1302) may be signaled in the generic constraint information as shown in FIG. 13. A constraint flag (e.g., no_lossy_coding_tool_constraint_flag) (1302) equal to 1 may indicate that coding tools related to lossy coding are not available within the scope of the PTL information that includes the constraint flag (1302). A constraint flag (e.g., no_lossy_coding_tool_constraint_flag) (1302) equal to 0 does not impose the above constraint.

[0137] In an embodiment, the constraint flag (e.g., no_lossless_coding_tool_contraint_flag) (1301) may not be equal to 1 when the constraint flag (e.g., no_lossy_coding_tool_constraint_flag) (1302) is equal to 1. Alternatively, the constraint flag (e.g., no_lossy_coding_tool_constraint_flag) (1302) may not be equal to 1 when the constraint flag (e.g., no_lossless_coding_tool_contraint_flag) (1301) is equal to 1.

[0138] The constraint flags in the generic constraint information may be sorted in a particular order. The order may be set, for example, based on the likelihood that each mechanism and / or tool will not be used in the scope of the PTL. The order may be referred to as a priority order. The order may be given in the generic constraint information syntax structure from high priority to low priority, where a high priority indicates a high likelihood of non-use of the tool (or mechanism) and a low priority indicates a low likelihood of non-use of the tool (or mechanism). Additional factors influencing the order may include that a tool is likely to be used only for a particular use case (e.g., sub-picture, scalability, and / or interlace support), the impact of the tool on encoder / decoder / implementation complexity, etc.

[0139] 14A-14B illustrate example PTL information including an example syntax structure (1410) of a PTL syntax structure (also referred to as a PTL bracket) and an example syntax structure (1420) of a generic constraint information syntax structure (also referred to as a generic constraint information bracket) according to some embodiments of the present disclosure. In some examples, a syntax element (e.g., num_available_contraint_flag) indicating the number of constraint flags may be signaled. In one example, the syntax element indicating the number of constraint flags may be signaled in a PTL syntax structure, such as shown by (1401) in the example syntax (1410) illustrated in FIG. 14A, which may be outside of the example syntax (1420) of the generic constraint information bracket. Alternatively, the syntax element indicating the number of constraint flags may be signaled at the beginning of the generic constraint information bracket, such as at the beginning of the syntax element (1420). The first N constraint flags may be present in a generic constraint information syntax structure if the syntax element (e.g., num_available_contraint_flag) is present and the number of syntax elements (e.g., num_available_contraint_flag) is equal to N. Additionally, other constraint flags may not be present or may be inferred to be equal to specific values. N may be a non-negative integer.

[0140] In an embodiment, the value N (e.g., num_available_contraint_flag) ranges from 0 to the maximum number of constraint flags (e.g., parameter MaxNumConstraintFlags). The maximum number of constraint flags can be any positive integer. The value of the maximum number of constraint flags (e.g., MaxNumConstraintFlags) may be predefined to be 16, 32, 64, 128, etc. When the value N (e.g., num_available_contraint_flag) is equal to 0, no constraint flags are present in the generic constraint information syntax structure. The coding of the value N (e.g., num_available_contraint_flag) may be selected such that the value N and the corresponding entropy coding representation of the constraint flags can be summed to a number divisible by 8 to ensure byte alignment.

[0141] In some examples, the constraint flags may be classified into one or more constraint information groups. Each constraint information group may include one or more constraint flags and may have a corresponding gate flag. The gate flag of the corresponding constraint information group may indicate whether the constraint flag in the corresponding constraint information group may be present. In one example, the gate flag may be referred to as a constraint group presence flag. In general, the gate flag is associated with the corresponding constraint information group and associated with the constraint flag in the corresponding constraint information group. In an embodiment, the gate flag gates whether the constraint flag in the corresponding constraint information group is present (or notified) in the constraint information. For example, if the gate flag of the corresponding constraint information group is equal to 1, the constraint flag corresponding to the constraint information group may be present, for example, in the general constraint information. If the gate flag of the corresponding constraint information group is equal to 0, the constraint flag corresponding to the constraint information group may not be present, for example, in the general constraint information. In one example, if all gate flags are equal to 0, the constraint flag is not present.

[0142] Constraint flags can have different scopes. For example, the scope of a constraint flag in a DCI can be the coded video bitstream. The scope of a constraint flag in a VPS can be a CLVS with multiple layers. The scope of a constraint flag in an SPS can be a single CLVS.

[0143] 15A to 15B show an example of a general constraint information syntax structure 1500 according to an embodiment of the present disclosure. The general constraint information syntax structure 1500 includes a flag that indicates the general constraint information. Specifically, the general constraint information syntax structure (1500) includes one or more gate flags, such as a gate flag (e.g., general_frame_structure_constraint_group_flag) (1501), a gate flag (e.g., high_level_functionality_constraint_group_flag) (1502), a gate flag (e.g., scalability_constraint_group_flag) (1503), a gate flag (e.g., partitioning_constraint_group_flag) (1504), a gate flag (e.g., intra_coding_tool_constraint_group_flag) (1505), a gate flag (e.g., inter_coding_tool_constraint_group_flag) (1506), a gate flag (e.g., transform_constraint_group_flag) (1507), a gate flag (e.g., inloop_filtering_constraint_group_flag) (1508), etc., of FIG. 15A. One or more gate flags (eg, gate flags (1501)-(1508)) may be present at the beginning of the general constraint information syntax structure (1500) as shown in FIG. 15A.

[0144] A gate flag (e.g., general_frame_structure_constraint_group_flag) (1501) is associated with a constraint information group (1510) and is associated with the constraint flags (1511)-(1514) that are in the constraint information group (1510). A gate flag (e.g., general_frame_structure_constraint_group_flag) (1501) equal to 1 can specify that the constraint flags (1511)-(1514) in the constraint information group (1510) may be present.

[0145] The constraint information group (1510) (or constraint flags (1511)-(1514)) may relate to an input source and a frame packing (e.g., packed or projected frame). Referring to FIG. 15A, the constraint flags (1511)-(1514) correspond to the general_non_packed_constraint_flag (1511), the general_frame_only_constraint_flag (1512), the general_non_projected_constraint_flag (1513), and the general_one_picture_only_constraint_flag (1514). Otherwise, a gate flag (e.g., the general_frame_structure_constraint_group_flag) (1501) equal to 0 may specify that the constraint flags (1511)-(1514) in the constraint information group (1510) may not be present in the general constraint information syntax structure (1500).

[0146] Further, in some examples, a gate flag (e.g., high_level_functionality_contraint_group_flag) (1502) equal to 1 may specify that a constraint flag for a high level functionality (e.g., reference picture resampling) in the constraint information group (1520) may be present as shown by Figure 15B. Otherwise, a gate flag (e.g., high_level_functionality_contraint_group_flag) (1502) equal to 0 may specify that a constraint flag in the constraint information group (1520) may not be present in the generic constraint information syntax structure (1500).

[0147] 15A, a gate flag (e.g., scalability_constraint_group_flag) (1503) equal to 1 can specify that a constraint flag for scalability (e.g., inter-layer prediction) may be present. Otherwise, a constraint flag for scalability may not be present in the generic constraint information syntax structure (1500).

[0148] A gate flag (e.g., partitioning_constraint_group_flag) (1504) equal to 1 can specify that constraint flags for higher level partitioning (e.g., subpicture or tile) may be present. Otherwise, constraint flags for higher level partitioning may not be present in the generic constraint information syntax structure (1500).

[0149] A gate flag (e.g., intra_coding_tool_constraint_group_flag) (1505) equal to 1 can specify that constraint flags for intra-coding (e.g., intra-prediction) may be present. Otherwise, constraint flags for intra-coding may not be present in the generic constraint information syntax structure (1500).

[0150] A gate flag (e.g., inter_coding_tool_constraint_group_flag) (1506) equal to 1 may specify that constraint flags for inter-coding (e.g., motion compensation for inter-picture prediction) may be present. Otherwise, constraint flags for inter-coding may not be present in the generic constraint information syntax structure (1500).

[0151] A gate flag (e.g., transform_constraint_group_flag) (1507) equal to 1 can specify that constraint flags for transform coding (e.g., multiple transform matrices) may be present. Otherwise, constraint flags for transform coding may not be present in the general constraint information syntax structure (1500).

[0152] In an embodiment, when all gate flags (eg, gate flags (1501) to (1508) in FIG. 15A) are equal to 0, the constraint flag is not present in the general constraint information syntax structure (eg, general constraint information syntax structure (1500)).

[0153] In accordance with aspects of the present disclosure, a syntax can be designed such that control information including gate flags (e.g., gate flags (1501)-(1508)), associated constraint flags (e.g., constraint flags (1511)-(1512) and constraint flags in the constraint information group (1520)), additional control information, and / or the like, can be byte aligned, e.g., the number of flags is divisible by 8 to maintain byte alignment. In one example, the number of gate flags and constraint flags in the constraint information (e.g., the generic constraint information syntax structure (1500)) is divisible by 8. A byte alignment mechanism can be used to achieve byte alignment of the control information. Referring to FIG. 15B, a syntax (e.g., a while loop) (1530) can be used for byte alignment.

[0154] In some embodiments, offset information, such as an offset (e.g., using the syntax element constraint_info_offset[]), and length information, such as a length (e.g., syntax element constraint_info_length[]), are present in the constraint information (e.g., at the beginning of the generic constraint information syntax structure) to help indicate the constraint flags in each constraint information group associated with the gating flags in the constraint information. In an embodiment, one or more of the at least one constraint information group are present in the coded video bitstream. In the case of a constraint information group, an offset and a length may be present in the constraint information for the constraint information group. The offset may indicate an offset to the first constraint flag in the constraint information group, and the length may indicate the number of constraint flags in the constraint information group. In some examples, the number of constraint information groups may be explicitly indicated, for example, by the syntax element num_constraint_info_set. The value of num_constraint_info_set may be an integer equal to or greater than 0. When the value of num_constraint_info_set is 0, constraint_info_offset[], constraint_info_length[], and constraint flags are not present in the generic constraint information syntax structure.

[0155] In an embodiment, the constraint information offset (e.g., syntax element constraint_info_offset[i]) and the constraint information length (e.g., syntax element constraint_info_length[i]) can help indicate the constraint flags for the constraint information group i (i is a positive integer) in the constraint information (e.g., the generic constraint information syntax structure). In one example, when the value of the constraint information offset (e.g., syntax element constraint_info_offset[i]) is equal to 5 and the value of the constraint information length (e.g., syntax element constraint_info_length[i]) is equal to 3, the fifth, sixth and seventh constraint flags are associated with the constraint information group i and are present in the constraint information (e.g., the generic constraint information syntax structure).

[0156] In one example, run-length coding may be used to code the constraint flags defined in a predefined order (or a given sequence).

[0157] In an embodiment, run-coding may be used, where the constraint flags are defined in a predefined order (or a given sequence). Instead of coding the constraint flags directly, a suitably coded list of "skip" values ​​can indicate a constraint flag equal to zero, with the following constraint flag being implied to be equal to 1. The above run-coding may be particularly useful when (i) the number of constraint flags is large and (ii) only a small fraction of the constraint flags are equal to 1.

[0158] In an embodiment, one or more of the at least one constraint information group are present in the coded video bitstream. The constraint flags in the one or more of the at least one constraint information group are signaled according to a predetermined order. Thus, the constraint flags may be run coded (e.g., run encoded or run decoded). Furthermore, prediction information for a subset of the coding blocks may be determined based on the constraint flags.

[0159] In an embodiment, at least one constraint flag in the constraint information group of the gate flag includes multiple constraint flags notified according to a predetermined order, and thus the multiple constraint flags can be run-coded (e.g., run-encoded or run-decoded).

[0160] In an embodiment, the full list of constraint flags may be specified in a video coding standard (e.g., the VVC specification), an external table, etc. In one example, only available constraint flags of the constraint flags are present in the coded video stream, as indicated, for example, by one or more of the following: the number of available constraint flags (e.g., num_available_contraint_flags), a gating flag (or constraint group present flag), constraint information offset information, and constraint information length information, etc.

[0161] In one example, a full list of constraint flags is defined and available to the encoder and the decoder. The full list of constraint flags may be stored in the decoder. The full list of constraint flags may include 100 constraint flags. Ten of the 100 constraint flags are present in the constraint information for the CLVS and are therefore available to a subset of coding blocks in the CLVS. The ten of the 100 constraint flags are referred to as the ten available constraint flags. In one example, the number of available constraint flags (e.g., ten) is signaled. In one example, the ten available constraint flags are in two constraint information groups and are gated by a first gate flag and a second gate flag. Thus, the first gate flag and the second gate flag may be signaled to indicate the ten available constraint flags.

[0162] In one example, a first constraint information offset (e.g., syntax element constraint_info_offset[0]) and a first constraint information length (e.g., syntax element constraint_info_length[0]) are signaled. A second constraint information offset (e.g., syntax element constraint_info_offset[1]) and a second constraint information length (e.g., syntax element constraint_info_length[1]) are signaled. For example, the syntax element constraint_info_offset[0] is 15, the syntax element constraint_info_length[0] is 3, the syntax element constraint_info_offset[1] is 82, and the syntax element constraint_info_length[1] is 7, so that the 15th to 17th constraint flags and the 82nd to 88th constraint flags in the full list (e.g., 100 constraint flags) are available or present in the constraint information.

[0163] In an embodiment, any of the various techniques (or methods, embodiments, examples) for efficient coding of constraint flags may be combined using appropriate control information. The combination may be any suitable combination of two or more of such techniques. Alternatively, one of the various techniques (or methods, embodiments, examples) may be used independently. The constraint information may be organized into groups. In certain groups, run coding may be used, while other groups may use simple binary coding.

[0164] The value of the maximum number of constraint flags (eg, MaxNumConstraintFlags) may be predefined to be 16, 32, 64, 128, and so on.

[0165] The value of the maximum number of constraint flags (e.g., MaxNumConstraintFlags) may be determined by profile information such as general_profile_idc or general_sub_profile_idc, or codec version information, so that the range of the number of constraint flags (e.g., num_available_contraint_flags (1401)) may be limited by the profile information or version information. For example, the value of the number of constraint flags (e.g., num_available_contraint_flags (1401)) in the main profile (where MaxNumConstraintFlags=64) may range from 0 to 64, while the value of the number of constraint flags (e.g., num_available_contraint_flags (1401)) in the advanced profile (where MaxNumConstraintFlags=128) may range from 0 to 128.

[0166] In an embodiment, the value of the number of constraint flags (e.g., num_available_contraint_flags) can be inferred to be equal to a value predefined by profile information such as general_profile_idc or general_sub_profile_idc, or codec version information, so that the value of num_available_contraint_flags can be determined without explicit signaling.

[0167] In some embodiments, reserved byte information may be present in the generic constraint information syntax structure. For example, as shown in FIG. 13, flags gci_num_reserved_bytes (1303) and gci_reserved_bytes[] (1304) may be present in the generic constraint information syntax structure for the extension of the generic constraint information syntax structure. The flag gci_num_reserved_bytes may specify the number of reserved constraint bytes. In one example, the reserved constraint bytes are for additional flags (e.g., additional constraint flags). The flag gci_reserved_bytes[] may have any suitable value.

[0168] In an embodiment, the value of gci_reserved_bytes may be limited or determined by profile information such as general_profile_idc or general_sub_profile_idc, or codec version information. According to the Basic Profile (or Main Profile), the value of the flag gci_reserved_bytes can be 0. According to the Extended Profile (or Advanced Profile), the value of gci_num_reserved_bytes is greater than 0.

[0169] In some embodiments, a field sequence flag may be signaled in the coded video bitstream. The field sequence flag may indicate whether a picture in an output layer set is coded with field coding. In some examples, the field sequence flag may be signaled in the SPS using the syntax element sps_field_seq_flag. In embodiments, the flag sps_field_seq_flag is present in the SPS. The flag sps_field_seq_flag equal to 1 may indicate that the CLVS carries a picture representing a field. The flag sps_field_seq_flag equal to 0 may indicate that the CLVS carries a picture representing a frame.

[0170] In the general constraint information syntax structure of FIG. 13, a flag general_frame_only_constraint_flag may be present. The flag general_frame_only_constraint_flag equal to 1 may specify that the scope for the output layer set (e.g., OlsInScope) carries pictures that represent frames. The flag general_frame_only_constraint_flag equal to 0 specifies that the scope for the output layer set (e.g., OlsInScope) carries pictures that may or may not represent frames. In an embodiment, the flag general_frame_only_constraint_flag indicates whether the pictures in the output layer set are coded with field coding. The output layer set may include a subset of coding blocks. The flag sps_field_seq_flag may be false based on the flag general_frame_only_constraint_flag (e.g., is 1) indicating that the subset of pictures is not coded with field coding. The subset of pictures may be in one layer of the output layer set.

[0171] When the flag general_frame_only_constraint_flag is equal to 1, the value of the flag sps_field_seq_flag may be equal to 0.

[0172] In an embodiment, the flag pps_mixed_nalu_types_in_pic_flag may be present in the PPS. The flag pps_mixed_nalu_types_in_pic_flag equal to 1 may specify that each picture that references the PPS has more than one VCL NAL unit and that the VCL NAL units do not have the same value of nal_unit_type. The flag pps_mixed_nalu_types_in_pic_flag equal to 0 may specify that each picture that references the PPS has one or more VCL NAL units and that the VCL NAL units of each picture that references the PPS have the same value of nal_unit_type. In the generic constraint information syntax structure of FIG. 13, the flag no_mixed_nalu_types_in_pic_constraint_flag may be present. The flag no_mixed_nalu_types_in_pic_const equal to 1 may specify that the value of pps_mixed_nalu_types_in_pic_flag must be equal to 0. The flag no_mixed_nalu_types_in_pic_const equal to 0 does not impose such a constraint.

[0173] In an embodiment, the flag general_one_picture_only_constraint_flag may be present in the general constraint information syntax structure as shown in Figure 13. general_one_picture_only_constraint_flag equal to 1 may specify that only one coded picture is present in the bitstream. The flag general_one_picture_only_constraint equal to 0 does not impose such a constraint.

[0174] In an embodiment, the flag single_layer_constraint_flag may be present in the general constraint information syntax structure as shown in Figure 13. The flag single_layer_constraint_flag equal to 1 may specify that sps_video_parameter_set_id must be equal to 0. The flag single_layer_constraint_flag equal to 0 does not impose such a constraint. When the flag general_one_picture_only_constraint_flag is equal to 1, the value of the flag single_layer_constraint_flag may be equal to 1.

[0175] In an embodiment, the flag all_layers_independent_constraint_flag may be present in the generic constraint information syntax structure as shown in Figure 13. The flag all_layers_independent_constraint_flag equal to 1 may specify that the flag vps_all_independent_layers_flag may be equal to 1. The flag all_layers_independent_constraint_flag equal to 0 does not impose such a constraint. When the flag single_layer_constraint_flag is equal to 1, the value of the flag all_layers_independent_constraint_flag may be equal to 1.

[0176] In an embodiment, the flag no_res_change_in_clvs_constraint_flag may be present in the generic constraint information syntax structure as shown in Figure 13. The flag no_res_change_in_clvs_constraint_flag equal to 1 may specify that the flag sps_res_change_in_clvs_allowed_flag may be equal to 0. The flag no_res_change_in_clvs_constraint_flag equal to 0 does not impose such a constraint. When the flag no_ref_pic_resampling_constraint_flag is equal to 1, the value of the flag no_res_change_in_clvs_constraint_flag may be equal to 1.

[0177] In an embodiment, the flag no_mixed_nalu_types_in_pic_constraint_flag may be present in the generic constraint information syntax structure of Figure 12. The flag no_mixed_nalu_types_in_pic_constraint_flag equal to 1 specifies that the value of the flag pps_mixed_nalu_types_in_pic_flag may be equal to 0. The flag no_mixed_nalu_types_in_pic_constraint_flag equal to 0 does not impose such a constraint. When the flag one_subpic_per_pic_constraint_flag is equal to 1, the flag no_mixed_nalu_types_in_pic_constraint_flag may be equal to 1.

[0178] In an embodiment, the flag no_trail_constraint_flag may be present in the general constraint information syntax structure of Figure 13. The flag no_trail_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to TRAIL_NUT may not be present in OlsInScope (OlsInScope is the output layer set that includes all layers of the entire bitstream that references the DPS). The flag no_trail_constraint_flag equal to 0 does not impose such constraint. When the flag general_one_picture_only_constraint_flag is equal to 1, the flag no_trail_constraint_flag may be equal to 1.

[0179] In an embodiment, the flag no_stsa_constraint_flag may be present in the general constraint information syntax structure of Figure 13. The flag no_stsa_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to STSA_NUT may not be present in OlsInScope. The flag no_stsa_constraint_flag equal to 0 does not impose such constraint. When the flag general_one_picture_only_constraint_flag is equal to 1, the flag no_stsa_constraint_flag may be equal to 1.

[0180] In an embodiment, the flag no_trail_constraint_flag may be present in the general constraint information syntax structure of Figure 13. The flag no_trail_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to TRAIL_NUT may not be present in OlsInScope. The flag no_trail_constraint_flag equal to 0 does not impose such constraint. When the flag general_one_picture_only_constraint_flag is equal to 1, the flag no_trail_constraint_flag may be equal to 1.

[0181] In an embodiment, the flag no_stsa_constraint_flag may be present in the general constraint information syntax structure of Figure 13. The flag no_stsa_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to STSA_NUT may not be present in OlsInScope. The flag no_stsa_constraint_flag equal to 0 does not impose such constraint. When the flag general_one_picture_only_constraint_flag is equal to 1, the flag no_stsa_constraint_flag may be equal to 1.

[0182] In an embodiment, the flag no_idr_constraint_flag may be present in the generic constraint information syntax structure as shown in Figure 13. no_idr_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to IDR_W_RADL or IDR_N_LP may not be present in OlsInScope. The flag no_idr_constraint_flag equal to 0 does not impose such a constraint.

[0183] In an embodiment, the flag no_cra_constrraint_flag may be present in the generic constraint information syntax structure as shown in Figure 13. The flag no_cra_constrraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to CRA_NUT may not be present in OlsInScope. The flag no_cra_constrraint_flag equal to 0 does not impose such constraint.

[0184] In an embodiment, the flag no_rasl_constraint_flag may be present in the generic constraint information syntax structure of FIG. 13 (the flag no_rasl_constraint_flag is not shown). The flag no_rasl_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to RASL_NUT may not be present in OlsInScope. The flag no_rasl_constraint_flag equal to 0 does not impose such constraint. When the flag no_cra_constraint_flag is equal to 1, the flag no_rasl_constraint_flag may be equal to 1.

[0185] In an embodiment, the flag no_radl_constraint_flag may be present in the generic constraint information syntax structure as shown in Figure 13. The flag no_radl_constraint_flag equal to 1 may specify that NAL units with nuh_unit_type equal to RADL_NUT may not be present in OlsInScope. The flag no_radl_constraint_flag equal to 0 does not impose such constraint. When the flag no_idr_constraint_flag is equal to 1 and the flag no_cra_constraint_flag is equal to 1, the value of the flag no_rasl_constraint_flag may be equal to 1.

[0186] Some aspects of the present disclosure provide techniques for constraint flag signaling for range extension, such as range extension with residual Rice coding in transform skip mode, Rice coding with regular residual coding, and so on.

[0187] In transform skip mode, the transform and inverse transform processes are skipped. For example, in a video encoder, pixel domain residual data may be quantized and entropy coded without being transformed to the transform domain. In a video decoder, the coded data is entropy decoded and dequantized back to pixel domain residual data without an inverse transform operation.

[0188] According to an aspect of the present disclosure, residual coding may be performed based on Rice coding. Rice coding uses adjustable parameters to split an input value into two parts and then code the two parts using different coding techniques. The adjustable parameters are called Rice parameters. The Rice parameters may affect the coding efficiency of Rice coding. Techniques may be used to determine the Rice parameters for a particular application to improve the coding efficiency for that application. Some techniques for determining the Rice parameters may be included in the scope extension of the video standard.

[0189] In accordance with aspects of the present disclosure, some standards may be originally developed for a specific application. To make the standard applicable to other applications, range extensions are developed with tools to support other applications. For example, HEVC is originally targeted to applications with 4:2:0 chroma format at 8-10 bits / sample. To make the HEVC standard applicable to other formats and bit depths in addition to the specific chroma format and specific bit depth, range extensions may be developed to support applications using other chroma formats and / or higher bit depths.

[0190] To limit the feature set to those required for a particular group of applications, video coding standards may define profiles that can include predefined decoder feature sets that are supported for interoperability with encoders that use those features. In addition to profiles, some standards (e.g., VVC, HEVC, etc.) also define levels and tiers. Levels impose limitations on the bitstream in terms of spatial resolution, pixel rate, bitrate values ​​and variables that may correspond to decoder processing load and memory capabilities. Levels may be expressed in terms of sinusoidal, maximum sample rate, maximum picture size, maximum bitrate, minimum compression ratio, coded picture buffer capacity, etc. Higher values ​​of levels may impose higher complexity limitations. Tiers change the bitrate value and variance limitations for each level. For example, the Main tier targets most applications, while the High tier is designed to accommodate more demanding video contribution applications, including much higher bitrate values ​​than video distribution applications. Each of the profiles, tiers, and levels impacts the implementation and decoding complexity, and the combination of the three specifies the interoperability point of the bitstream and the decoder.

[0191] In some examples, a decoder conforming to a particular tier and level is required to be able to decode all bitstreams conforming to the same or lower tier at that level or any level below, and a decoder conforming to a particular profile can support all features of that profile. In some examples, an encoder is not required to use any particular set of features supported in a profile, but is required to generate bitstreams, i.e., bitstreams that obey certain constraints that allow them to be decoded by conforming decoders.

[0192] In addition to the PTL information, the PTL syntax structure may also contain a Generic Constraint Information (GCI) syntax structure that contains a list of non-flag syntax elements and constraint flags that indicate specific constraint characteristics of the bitstream.

[0193] In one example, HEVC natively includes three profiles called Main Profile, Main 10 Profile, and Main Still Picture Profile. The three profiles have some limitations, such as only supporting 4:2:0 chroma sampling. The Main Profile and Main Still Picture Profile only support 8 bits / sample video precision, while the Main 10 Profile supports up to 10 bits / sample. In the Main Still Picture Profile, the entire bitstream contains only one coded picture.

[0194] In some examples, HEVC with range extension can support additional profiles. In one example, the following profiles are collectively referred to as range extension profiles: Monochrome profile, Monochrome 10 profile, Monochrome 12 profile, Monochrome 16 profile, Main 12 profile, Main 4:2:2 10 profile, Main 4:2:2 12 profile, Main 4:4:4 profile, Main 4:4:4 10 profile, Main 4:4:4 12 profile, Main Intra profile, Main 10 Intra profile, Main 12 Intra profile, Main 4:2:2 10 Intra profile, Main 4:2:2 12 Intra profile, Main 4:4:4 Intra profile, Main 4:4:4 10 Intra profile, Main 4:4:4 12 Intra profile, Main 4:4:4 16 Intra profile, Main 4:4:4 Still Image profile, and Main 4:4:4 16 Still Image profile.

[0195] Some of the range extension profiles can support higher bit depths and may be referred to as profiles for operation range extensions with high bit depth. In some examples, profiles for operation range extensions with high bit depth include profiles supporting more than 10 bits / sample, such as the Main 12 profile, the Main 12 4:4:4 profile, the Main 16 4:4:4 profile, the Main 12 Intra profile, the Main 12 4:4:4 Intra profile, the Main 16 4:4:4 Intra profile, the Main 12 still image profile, the Main 12 4:4:4 still image profile, the Main 16 4:4:4 still image profile, etc.

[0196] Specifically, the Main 12 profile supports 4:0:0 and 4:2:0 chroma sampling, and both intra and inter prediction modes, allowing bit depths from 8 to 12 bits per sample. In some examples, a decoder conforming to the Main 12 profile can decode bitstreams created with the following profiles: Monochrome, Monochrome 12, Main, Main 10, and Main 12.

[0197] The Main 12 4:4:4 profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 chroma sampling and both intra and inter prediction modes, allowing a bit depth of 8 to 12 bits per sample. In some examples, a decoder conforming to the Main 12 4:4:4 profile can decode bitstreams created with the following profiles: Monochrome, Main, Main 10, Main 12, Main 10 4:2:2, Main 12 4:2:2, Main 4:4:4, Main 10 4:4:4, Main 12 4:4:4, and Monochrome 12.

[0198] The Main16 4:4:4 profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 chroma sampling, and both intra and inter prediction modes, allowing bit depths from 8 to 16 bits per sample.

[0199] The Main 12 Intra profile supports 4:0:0 and 4:2:0 chroma sampling, as well as intra prediction modes, allowing bit depths from 8 to 12 bits per sample.

[0200] The Main12 4:4:4 Intra profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 as well as intra prediction modes, allowing bit depths from 8 to 12 bits per sample.

[0201] The Main16 4:4:4 Intra profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 as well as intra prediction modes, allowing bit depths from 8 to 16 bits per sample.

[0202] The Main 12 still profile supports 4:0:0 and 4:2:0 chroma sampling, allowing bit depths from 8 to 12 bits per sample. In the Main 12 still profile, the entire bitstream contains only one coded picture.

[0203] The Main 12 4:4:4 Still Image profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 chroma sampling, allowing bit depths from 8 to 12 bits per sample. In the Main 12 4:4:4 Still Image profile, the entire bitstream contains only one coded picture.

[0204] The Main 16 4:4:4 Still Image profile supports 4:0:0, 4:2:0, 4:2:2 and 4:4:4 chroma sampling, allowing bit depths from 8 to 16 bits per sample. In the Main 16 4:4:4 Still Image profile, the entire bitstream contains only one coded picture.

[0205] In accordance with some aspects of the disclosure, the coding tool control may be performed at various scopes (e.g., portions of coded video data being coded utilizing persistence of instances of syntax elements for coding tool control), such as scope of a bitstream, a coded layer video sequence (CLVS), a picture, a slice of a picture, and the like. In some examples, the coding tool control may be provided in a generic constraint information (GCI) syntax structure that generally includes constraint information for a bitstream carrying an output layer set to a decoder. In some examples, the coding tool control may be provided in a sequence parameter set (SPS) associated with a CLVS, where the SPS generally includes information for the CLVS. In some examples, the coding tool control may be provided in a picture header of a picture, where the picture header generally includes information for the picture. In some examples, the coding tool control may be provided in a slice header of a slice, where the slice header generally includes information for the slice.

[0206] According to aspects of the present disclosure, control information for coding tools in range extensions can be provided in various ranges. In some examples, using a wider range of syntax elements can improve coding efficiency. For example, a GCI syntax element value greater than 0 indicates that the bitstream is constrained in a particular way, typically indicating that a particular coding tool is not used in the bitstream. Furthermore, a GCI syntax element value equal to the value 0 signals that the associated constraint may not apply, whereby the associated coding tool is allowed (but not required) to be used in the bitstream (if supported by the profile in which its use is indicated).

[0207] In accordance with other aspects of the present disclosure, when a coding tool is not used in coding video data in a bitstream, e.g., the PTL information and / or the general constraint information indicate non-use of the coding tool, a video decoder without support for the coding tool may determine that the video decoder can decode the bitstream based on the signaling in the PTL information and / or the general constraint information, and the functionality of the video decoder may be extended.

[0208] In some embodiments, an encoder may generate a bitstream that conforms to a video standard with a range extension, but does not use one or more features supported in the range extension. In some examples, a decoder that conforms to the video standard but does not support one or more features in the range extension, knowing the non-use of one or more features in the range extension, may determine that the decoder is able to decode the bitstream and may accept the bitstream for decoding rather than rejecting it.

[0209] FIG. 16 illustrates a syntax structure (1600) for generic constraint information according to some embodiments of the present disclosure. In some examples, the syntax structure (1600) includes constraints that are applied to a bitstream, such as a bitstream that includes an output layer set to a decoder. In the example of FIG. 16, a syntax element represented by gci_num_additional_bits in the syntax structure (1600) is used to specify the number of additional generic constraint information (GCI) bits in the generic constraint information syntax structure (1600) other than the alignment zero bit syntax element (if present). In some standards, the value of gci_num_additional_bits is required to be equal to 0 or 1. In some standards, a decoder may allow a value of gci_num_additional_bits greater than 1 to appear in the syntax structure.

[0210] In the example of FIG. 16, the syntax structure (1600) is: general_no_extended_precision_constraint_flag, genral_no_ts_residual_coding_rice_present_in_sh_constraint_flag, general_no_rrc_rice_extension_constraint_flag, general_no_persistent_rice_adaptation_constraint_flag, and general_no_reverse_last_sig_coeff_constraint_flag The five additional GCI bits (syntax elements) 1601-1605 each provide coding control information for a coding tool within the bitstream of the output layer set, in some examples.

[0211] FIG. 17 illustrates an example of a syntax structure (1700) for a sequence parameter set (SPS) range extension according to some embodiments of the present disclosure. The syntax structure (1700) may be added in the SPS for CLVS to provide coding tool control of the range extension for CLVS. The syntax structure (1700) may include: sps_extended_precision_flag, sps_ts_residual_coding_rice_present_in_sh_flag, sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag, and sps_reverse_last_sig_coeff_enabed_flag The five syntax elements (1701) to (1705) provide coding control information for a coding tool in the CLVS range, in some examples.

[0212] Specifically, in an embodiment, GCI bits (1601) and syntax elements (1701) are used to provide control over the use of extended precision, such as control of coding tools for extended dynamic range of binarization of transform coefficients in scaling and transformation processes, and some syntax elements, such as abs_remainder[] and dec_abs_level[], in different ranges.

[0213] A syntax element (1701) equal to 1 specifies that the extended dynamic range is used for the transform coefficients in the scaling and transformation process, as well as for the binarization of some syntax elements, such as abs_remainder[] and dec_abs_level[]. The syntax element abs_remainder[scan position n] is the absolute value of the remainder of the transform coefficient level coded by the Golomb-Rice code at the scan position n. If abs_remainder[] is not present, it is inferred to be equal to 0. The syntax element dec_abs_level[scan position n] may correspond to an intermediate value coded by the Golomb-Rice code at the scan position n, and is used to determine the level of the transform coefficient at the scan position n. A syntax element (1701) equal to 0 specifies that the extended dynamic range is not used in the scaling and transformation process, e.g., for the binarization of the syntax elements abs_remainder[] and dec_abs_level[].

[0214] In one example, a variable represented by Log2TransformRange is used to determine a dynamic range for transform coefficients in the scaling and transformation process and for binarization of a particular syntax element. For example, the variable Log2TransformRange can be the number of bits for representing transform coefficients in the scaling and transformation process and for binarization of a particular syntax element. The dynamic range can be the difference between the maximum number and the minimum number, expressed using the number of bits. In one example, the variable Log2TransformRange is expressed by the formula (1): Log2TransformRange= sps_extended_precision_flag? Max(15,Min(20,BitDepth+6)):15 Formula (1) The precision is derived according to the syntax element (1701) sps_extended_precision_flag, such as by using

[0215] The dynamic range for the transform coefficients in the scaling and transform process and for the binarization of certain syntax elements may be determined based on the variable Log2TransformRange. In some examples, when the flag sps_extended_precision_flag has a value of 0, the extended dynamic range feature (e.g., the extended dynamic range coding tool) is not used, and the dynamic range of the transform coefficients is based on a fixed number of bits, such as 15 bits. When the flag sps_extended_precision_flag has a value of 1, the extended dynamic range feature is enabled, and the number of bits for representing the transform coefficients in the scaling and transform process can be one of 15 bits, 16 bits, 17 bits, 18 bits, 19 bits, and 20 bits based on the bit depth BitDepth in the example of Equation (1). The dynamic range of the transform coefficients may be determined based on the number of bits.

[0216] According to aspects of the disclosure, a syntax element (e.g., represented by sps_bitdepth_minus8) may be used to signal the bit depth (e.g., represented by BitDepth) of the samples of the luma and chroma arrays, and the luma and chroma quantization parameter range offsets (e.g., represented by QpBdOffset). In one example, the bit depth BitDepth may be calculated according to equation (2), and the QP range offset QpBdOffset may be calculated according to equation (3): BitDepth=8+sps_bitdepth_minus8 (2) QpBdOffset=6×sps_bitdepth_minus8 (3)

[0217] In some examples, a GCI bit (1601) equal to 1 specifies that the syntax element instance (1701) for all pixels within the output layer set scope (OlsInScope) may be equal to 0. A GCI bit (1601) equal to 0 imposes no such constraint. Thus, a GCI bit (1601) equal to 1 can specify non-use of extended dynamic range coding tools in coding the bitstream.

[0218] In some embodiments, the GCI bits (1602) and syntax elements (1702) are used to provide control of coding tools for slice-based Rice coding for residual coding in transform skip mode, such as slice-based Rice parameter selection for residual coding in transform skip mode, in different ranges.

[0219] In accordance with aspects of the present disclosure, slice-based Rice parameter selection for transform skip residual coding may be included in the scope extension of the video standard. In some examples, one control flag (e.g., represented by the syntax element sps_ts_residual_coding_rice_present_in_sh_flag (1702)) is signaled in the sequence parameter set (SPS) when transform skip mode is enabled (e.g., syntax element sps_transform_skip_enabled_flag is true) to indicate that signaling of Rice parameters for transform skip slices is enabled or disabled, as shown in FIG. 17.

[0220] If the control flag is signaled as enabled (e.g., equal to “1”), one syntax element (e.g., represented by sh_ts_residual_coding_rice_idx_minus1) may further be signaled for each transform skip slice, e.g., in a slice header, to indicate the Rice parameter selection for that transform skip slice. If the control flag is signaled as disabled (e.g., equal to “0”), no further syntax element may be signaled at the slice level (e.g., slice header) to indicate the Rice parameter selection for the transform skip slice, and a default Rice parameter may be used for all transform skip slices in the coded video data that references the SPS, in one example.

[0221] For example, a syntax element (1702) equal to 1 in the SPS specifies that the slice header flag represented by sh_ts_residual_coding_rice_idx_minus1 may be present in the slice header (e.g., slice_header()) syntax structure of the slice that references sps_ts_residual_coding_rice_present_in_sj_flag. A syntax element (1702) equal to 0 in the SPS specifies that the slice header flag sh_ts_residual_coding_rice_idx_minus1 is not present in the slice_header() syntax structure of the slice that references the SPS. If not present, the value of sps_ts_residual_coding_rice_present_in_sh_flag is inferred to be equal to 0 in some instances.

[0222] In some examples, a syntax element may be included in the generic constraint information to control the use of a coding tool of slice-based Rice coding for residual coding in transform skip mode in the scope of an output layer set. For example, a syntax element (1602) equal to 1 specifies that the syntax element for all pictures in the scope (OlsInScope) of the output layer set may be equal to 0. A syntax element (1602) equal to 0 does not impose such a constraint. Thus, in some examples, a GCI bit (1602) equal to 1 in a bitstream may specify non-use of slice-based Rice parameter selection for transform skip residual coding coding bitstream.

[0223] In some embodiments, the GCI bits (1603) and syntax elements (1703) are used to provide control of one or more coding tools of Rice parameter derivation for binarization of some syntax elements, such as abs_remainder[] and dec_abs_level[] in Regular Residual Coding (RRC), in different scopes. In some examples, Regular Residual Coding (RRC) refers to some techniques for coding blocks obtained by transform and quantization. In some examples, RRC may be modified for blocks obtained by quantization only. In some examples, Transform Skip Residual Coding refers to some techniques that are dedicated to coding tools obtained by bypassing transform (also called transform skip).

[0224] In some examples, the video coding standard may include one or more coding tools of Rice parameter derivation for binarization of some syntax elements, such as abs_remainder[ ] and dec_abs_level[ ], and a scope extension of the video coding standard may include one or more alternative coding tools of Rice parameter derivation for binarization of some syntax elements, such as abs_remainder[ ] and dec_abs_level[ ].

[0225] In some examples, video standards use a local template-based technique for Rice parameter derivation. For example, a template including one or more (e.g., five in one example) adjacent coefficient levels is used for Rice parameter derivation. For example, a sum of absolute coefficient values ​​in the template may be calculated, and then the Rice parameter is determined based on the sum. In one example, a lookup table may be used to determine the Rice parameter based on the sum.

[0226] It should be noted that the Rice parameters may be determined by other suitable coding tools. In one example, a formula may be used to determine the Rice parameters based on sums. In another example, context modeling may be used to determine the Rice parameters based on statistics of adjacent coefficient levels. In some examples, range extensions of a video standard may specify one or more alternative coding tools for Rice parameter derivation.

[0227] In some examples, range extensions of the video standard may include modifications to the RRC for use in other scenarios. In one example, the range extensions may include different context modeling tools and residual signal rotation tools for residual coding in transform skip mode.

[0228] In some examples, a syntax element (1703) in an SPS equal to 1 specifies that an alternative Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] (e.g., an alternative coding tool for Rice parameter derivation in range extension) is used to code a CLVS that references the SPS. A syntax element (1703) equal to 0 specifies that an alternative Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] is not used to code a CLVS that references the SPS. If not present, the value of the syntax element (1703) is inferred to be equal to 0.

[0229] In some examples, a syntax element (1603) equal to 1 specifies that the syntax element (1703) may be equal to 0 for all pictures within the scope (OlsInScope) of the output layer set. A syntax element (1603) equal to 0 imposes no such constraint. Thus, in some examples, a GCI bit (1603) equal to 1 can specify non-use of an alternative Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] (e.g., an alternative coding tool for Rice parameter derivation specified in a defined range extension) for coding the bitstream.

[0230] In some embodiments, GCI bits (1604) and syntax elements (1704) are used to provide control of the statistically-based Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] in different ranges.

[0231] According to aspects of the present disclosure, the Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] may be initialized at the beginning of each transform unit (TU) with accumulated statistics from the previous TU. In some examples, the statistics-based Rice parameter derivation may be included in the scope extension of the video standard.

[0232] In some examples, a control flag, e.g., a syntax element (1704) represented by sps_persistent_rice_adaptation_enabled_flag in the SPS, is used to control the statistics-based Rice parameter derivation. For example, a syntax element (1704) equal to 1 in the SPS specifies that the Rice parameter derivation for binarization of abs_remainder[] and dec_abs_level[] is initialized at the beginning from each TU using the statistics accumulated from the previous TU. A syntax element (1704) equal to 0 specifies that the state of the previous TU is not used in the Rice parameter derivation of the current TU. If not present, the value of the syntax element (1704) is inferred to be equal to 0.

[0233] Further, in an embodiment, a syntax element (1704) equal to 1 specifies that the syntax element (1704) may be equal to 0 for all pictures within the scope (OlsInScope) of the output layer set. A syntax element (1704) equal to 0 imposes no such constraint. Thus, in some examples, a GCI bit (1604) equal to 1 can specify no use of statistics-based Rice parameter derivation for coding the bitstream.

[0234] In some embodiments, the GCI bits (1605) and syntax elements (1705) are used to code the location of the last valid coefficient during entropy coding of transform coefficients in different ranges. In one example, the location of the last valid coefficient may be coded by different coding tools. For example, a video standard may specify a first coding tool that can determine the location of the last valid coefficient by using two coordinates of that location represented by LastSignificantCoeffX and LastSignificantCoeffY, and a range extension of the video standard may specify an alternative coding tool, such as a second coding tool that can determine the location of the last valid coefficient by coding the relative coordinate of the last valid coefficient with reference to the lower right corner of a zero-out transform block in one example.

[0235] In some examples, a syntax element (1705) equal to 1 in the SPS specifies that the slice header flag (slice range) represented by sh_reverse_last_sig_coeff_flag is present in the slice header syntax structure (e.g., slice_header() in some examples) that references the SPS. A syntax element (1705) equal to 0 in the SPS specifies that the slice header flag sh_reverse_last_sig_coeff_flag is not present in the slice header syntax structure that references the SPS, and the slice header flag sh_reverse_last_sig_coeff_flag may be inferred to be zero. If not present, the value of the syntax element (1705) is inferred to be equal to 0.

[0236] In some examples, the slice header flag sh_reverse_last_sig_coeff_flag of a slice is used to determine the location derivation of the last valid coefficient in the transform coefficients of the scaling and transform process during coding of the slice. In one example, if sh_reverse_last_sig_coeff_flag is equal to 1, the location of the last valid coefficient is coded by an alternative coding tool in the range extension of the video standard, such as a second coding tool, otherwise the current coordinate of the location of the last valid coefficient is coded by the first coding tool.

[0237] In some examples, a GCI bit (1605) equal to 1 specifies that the syntax element (1705) may be equal to 0 for all pictures within the scope (OlsInScope) of the output layer set. A GCI bit (1605) equal to 0 imposes no such constraint. Thus, a GCI bit (1605) equal to 1 can specify non-use of the second coding tool in deriving the position of the last valid coefficient for the scope of the bitstream.

[0238] FIG. 18 shows a flow chart illustrating a process (1800) according to an embodiment of the present disclosure. The process (1800) may be used in a video decoder. In various embodiments, the process (1800) is performed by a processing circuit, such as the processing circuit of the terminal devices (310), (320), (330), and (340), a processing circuit performing the functions of the video decoder (410), a processing circuit performing the functions of the video decoder (510), etc. In some embodiments, the process (1800) is implemented with software instructions, and thus, the processing circuit performs the process (1800) when the processing circuit executes the software instructions. The process starts at (S1801) and proceeds to (S1810).

[0239] At (S1810), a first syntax element (e.g., general_no_ts_residual_coding_rice_present_in_sh_constraint_flag) for coding control in a first range of coded video data in the bitstream is determined. The first syntax element is associated with a coding tool for slice-based Rice coding for residual coding in transform skip mode. In some examples, the coding tool provides Rice parameters at the slice level for residual coding in transform skip mode.

[0240] In one example, the first syntax element is decoded from the syntax structure for the general constraint information in response to a syntax element (eg, gci_num_additional_bits) in the syntax structure indicating additional bits for the general constraint information in the syntax structure.

[0241] In some examples, the coding tools are defined in a range extension of the standard. In one example, a video decoder may support the standard but may not support the coding tools in the range extension of the standard. In another example, the video decoder supports the range extension of the standard.

[0242] At (S1820), if the value of the first syntax element is a first value, the process proceeds to (S1830), otherwise the process proceeds to (S1840), where the first value indicates non-use (disabling) of the coding tool in coding a first range of coded video data in a bitstream that includes one or more second ranges of coded video data (e.g., one or more CLVSs in an output layer set).

[0243] In some examples, the first syntax element is in the general constraint information for controlling coding of pictures in an output layer set output at a decoder. In one example, the first value of the first syntax element indicates non-use of a coding tool in coding each coded layer video sequence (CLVS) in the output layer set.

[0244] At (S1830), in response to the first syntax element being a first value, a first range of coded video data in the bitstream is decoded without invoking a coding tool.

[0245] In some examples, a second syntax element for coding control of a coded layer video sequence (CLVS) in a bitstream (e.g., sps_ts_residual_coding_rice_present_in_sh_flag) is constrained to have a value indicating not to invoke a coding tool to decode the CLVS.

[0246] At (S1840), in response to the first syntax element being a second value, a value of a second syntax element (e.g., sps_ts_residual_coding_rice_present_in_sh_flag) for coding control of a second range of coded video data (e.g., a coded layer video sequence (CLVS)) in the bitstream is determined for decoding of the coded video data in the second range. The second range of coded video data is then decoded accordingly.

[0247] In some examples, the value of the second syntax element is determined in response to the first syntax element being a second value and the transform skip mode being enabled in the coding layer video sequence (CLVS). In one example, the second syntax element is decoded from a sequence parameter set (SPS) referenced by the CLVS. The value of the second syntax element indicates the presence / absence of a slice level syntax element indicating a selection of a Rice parameter. In response to the value of the second syntax element indicating the presence of a slice level syntax element, in one example, a third syntax element is decoded from a slice header of the slice. The third syntax element indicates a selection of a Rice parameter for residual coding of the slice in transform skip mode.

[0248] In one example, the second syntax element is not present in a sequence parameter set (SPS) for CLVS, and the value of the second syntax element is inferred to indicate non-use of a coding tool in CLVS or the absence of a slice-level syntax element indicating selection of Rice parameters for residual coding of the slice in transform skip mode.

[0249] The process 1800 may be adapted as appropriate. Steps of the process 1800 may be modified and / or omitted. Additional steps may be added. Any suitable order of performance may be used.

[0250] FIG. 19 shows a flow chart illustrating a process (1900) according to an embodiment of the present disclosure. The process (1900) may be used in a video encoder. In various embodiments, the process (1900) is performed by a processing circuit, such as the processing circuit of the terminal devices (310), (320), (330), and (340), the processing circuit performing the functions of the video encoder (403), the processing circuit performing the functions of the video encoder (603), the processing circuit performing the functions of the video encoder (703), etc. In some embodiments, the process (1900) is implemented with software instructions, and thus, the processing circuit performs the process (1900) when the processing circuit executes the software instructions. The process starts at (S1901) and proceeds to (S1910).

[0251] At (S1910), the processing circuitry determines whether a coding tool for slice-based Rice coding for residual coding in transform skip mode is used during encoding of a first range of coded video data in the bitstream (e.g., an output layer set). The first range of coded video data includes one or more second ranges of coded video data (e.g., CLVS). In some examples, the coding tool provides Rice parameters at a slice level for residual coding in transform skip mode.

[0252] In some examples, the processing circuitry may determine whether the coding tool is used based on a second syntax element for coding control of a coded layer video sequence (CLVS) in the bitstream (e.g., sps_ts_residual_coding_rice_present_in_sh_flag).

[0253] At (S1920), if the coding tool is not used for the first range of coded video data, the process proceeds to (S1930), otherwise the process proceeds to (S1940).

[0254] At (S1930), a first syntax element (e.g., general_no_persistent_rice_adaptation_constraint_flag) having a first value is coded in the bitstream. The first syntax element is for coding control in a first range of coded video data in the bitstream. The first syntax element is associated with a coding tool of slice-based Rice coding for residual coding in transform skip mode. The first value indicates non-use of the coding tool in coding of the first range of coded video data.

[0255] In one example, the first syntax element is encoded in a syntax structure for general constraint information, and a syntax element (eg, gci_num_additional_bits) in the syntax structure is adjusted to indicate additional bits for the general constraint information in the syntax structure.

[0256] At (S1940), the first syntax element having the second value is encoded in the bitstream. In some examples, the first syntax element is not encoded in the bitstream, e.g., if the second value is a default value for the first syntax element, and thus the first syntax element can be inferred if not present, in which case (S1940) may be skipped.

[0257] The process 1900 may be adapted as appropriate. Steps of the process 1900 may be modified and / or omitted. Additional steps may be added. Any suitable order of performance may be used.

[0258] The techniques described above (e.g., techniques for signaling constraint flags, adaptive resolution parameters, and / or the like) may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 20 illustrates a computer system (2000) suitable for implementing certain embodiments of the disclosed subject matter.

[0259] Computer software can be coded in any suitable machine code or computer language that can be subject to mechanisms such as assembly, compilation, linking, etc. to generate code including instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., directly, or through interpretation, microcode execution, etc.

[0260] The instructions may be executable by various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming consoles, Internet of Things devices, and the like.

[0261] The components illustrated in Figure 20 for the computer system (2000) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of the computer system (2000).

[0262] The computer system (2000) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more users through, for example, tactile input (e.g., keystrokes, swipes, dataglove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0263] The input human interface devices may include one or more of a keyboard (2001), a mouse (2002), a trackpad (2003), a touch screen (2010), a data glove (not shown), a joystick (2005), a microphone (2006), a scanner (2007), and a camera (2008) (only one of each is shown).

[0264] The computer system (2000) may also include certain human interface output devices that may stimulate one or more of the user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (2010), data gloves (not shown), or joystick (2005), although there may also be haptic feedback devices that do not function as input devices), audio output devices (e.g., speakers (2009), headphones (not shown)), visual output devices (e.g., screens (2010) including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capability, each with or without haptic feedback capability, some of which are capable of outputting two-dimensional visual output or output in more than three dimensions by means of stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0265] The computer system (2000) may also include human accessible storage devices and their associated media, such as CD / DVD ROM / RW (2020) with CD / DVD or similar media (2021), thumb drives (2022), removable hard disks or solid state drives (2023), legacy magnetic media, such as tapes and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices, such as security dongles (not shown), and the like.

[0266] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0267] The computer system (2000) may also include an interface (2054) to one or more communication networks (2055). The networks may be, for example, wireless, wireline, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wireline or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and factory networks including CANBus, etc. Certain networks generally require an external network interface adapter attached to a particular general purpose data port or peripheral bus (2049) (e.g., a USB port of the computer system (2000)). Others are generally built into the core of the computer system (2000) by attachment to a system bus as described below (e.g., an Ethernet network to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (2000) can communicate with other entities. Such communications can be one-way receive-only (e.g., broadcast TV) or one-way transmit-only (e.g., the CANBus to a specific CANBus device), or can be bidirectional to other computer systems, for example, using local or wide area digital networks. Specific protocols or protocol stacks can be used with each of the networks and network interfaces as described above.

[0268] The above human interface devices, human accessible storage devices, and network interfaces may be attached to a core (2040) of the computer system (2000).

[0269] The cores (2040) may include one or more central processing units (CPUs) (2041), graphics processing units (GPUs) (2042), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (2043), hardware accelerators (2044) for specific tasks, graphics adapters (2050), etc. These devices may be connected through a system bus (2048), along with read only memory (ROM) (2045), random access memory (RAM) (2046), internal mass storage devices such as internal non-user accessible hard drives, SSDs, etc. (2047). In some computer systems, the system bus (2048) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripherals may be attached directly to the core's system bus (2048) or through a peripheral bus (2049). In one example, a screen (2010) may be connected to a graphics adapter (2050). Architectures for peripheral buses include PCI, USB, etc.

[0270] The CPU (2041), GPU (2042), FPGA (2043), and accelerator (2044) can execute certain instructions that can be combined to constitute the above computer code. The computer code can be stored in ROM (2045) or RAM (2046). Temporary data can also be stored in RAM (2046), while persistent data can be stored, for example, in an internal mass storage device (2047). Rapid storage and retrieval from any of the memory devices is made possible by the use of cache memory. Cache memory can be closely associated with one or more of the CPU (2041), GPU (2042), mass storage device (2047), ROM (2045), RAM (2046), etc.

[0271] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the kind well known and available to those of ordinary skill in the computer software arts.

[0272] By way of example, and not by way of limitation, a computer system having the architecture (2000), and in particular the core (2040), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with the user-accessible mass storage devices introduced above, in addition to specific storage of the core (2040) that is non-transitory in nature, such as the core's internal mass storage device (2047) or ROM (2045). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (2040). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (2040), and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to execute certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (2046) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerators (2044)) that can operate in place of or in conjunction with software to execute certain processes or certain portions of certain processes described herein. References to software can encompass logic, where appropriate, and vice versa. References to computer-readable media can encompass circuitry (e.g., integrated circuits (ICs)) storing software for execution, circuitry embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0273] Appendix A: Acronyms JEM: Joint Exploration Model VVC:Versatile Video Coding BMS:Benchmark Set MV:Motion Vector HEVC:High Efficiency Video Coding SEI:Supplementary Enhancement Information VUI:Video Usability Information GOP:Group of Picture(s) TU:Transform Unit(s) PU:Prediction Unit(s) CTU:Coding Tree Unit(s) CTB:Coding Tree Block(s) PB:Prediction Block(s) HRD:Hypothetical Reference Decoder SNR:Signal Noise Ratio CPU:Central Processing Unit(s) GPU:Graphics Processing Unit(s) CRT:Cathode Ray Tube LCD:Liquid-Crystal Display OLED:Organic Light-Emitting Diode CD:Compact Disc DVD:Digital Video Disc ROM:Read-Only Memory RAM:Random Access Memory ASIC:Application-Specific Integrated Circuit PLD:Programmable Logic Device LAN:Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI:Peripheral Component Interconnect FPGA:Field Programmable Gate Area(s) SSD: Solid-State Drive IC: Integrated Circuit CU: Coding Unit

[0274] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that are within the scope of this disclosure. Thus, it will be apparent to those skilled in the art that numerous systems and methods will be contemplated that embody the principles of the present disclosure and are therefore within its spirit and scope, even if not explicitly shown or described herein.

Claims

1. 1. A method of video decoding performed by a decoder, comprising: obtaining a video bitstream; determining that the video bitstream includes a first syntax element corresponding to a General Constraint Information (GCI) syntax structure for coding control of pictures in an output layer set for the decoder, the first syntax element indicating a number of additional bits in the GCI syntax structure; and if the first syntax element indicates that there is at least one additional bit, decoding from the GCI syntax structure a second syntax element for coding control in a first range of coded video data in the video bitstream, the second syntax element being associated with a coding tool for slice-based Rice coding for residual coding in a transform skip mode, the coding tool providing Rice parameters at a slice level for the residual coding in the transform skip mode. decoding the first range of coded video data, including one or more second ranges of coded video data, without invoking the coding tool if the second syntax element is a first value indicating disabling of the coding tool in the first range; The method according to claim 1,

2. The second syntax element is represented by general_no_ts_residual_coding_rice_present_in_sh_constraint_flag in the GCI syntax structure; The method of claim 1.

3. the first value of the second syntax element indicates disabling the coding tool for each coding layer video sequence (CLVS) in the output layer set. The method of claim 2.

4. and constraining a third syntax element for coding control of a coding layer video sequence (CLVS) in the video bitstream to have a value indicating not to invoke the coding tool to decode the CLVS when the second syntax element is the first value. The method of claim 2.

5. the value of the third syntax element indicates the absence of a slice header flag associated with the coding tool in a slice header of a slice in the picture of the CLVS. The method according to claim 4.

6. When the second syntax element is a second value and the transform skip mode is enabled in a coding layer video sequence (CLVS), determining a value of a third syntax element in a sequence parameter set (SPS) referenced by the CLVS; The third syntax element indicates the presence / absence of a slice level syntax element indicating a selection of a Rice parameter. The method of claim 2.

7. if the value of the third syntax element indicates the presence of the slice level syntax element, decoding a fourth syntax element from a slice header of a slice; The fourth syntax element indicates a selection of a Rice parameter for the residual coding of the slice in the transform skip mode. The method according to claim 6.

8. The step of determining the value of the third syntax element comprises: and if the third syntax element is not present in the SPS of the CLVS, inferring the value of the third syntax element indicating non-use of the coding tool at the CLVS. The method according to claim 6.

9. 1. An apparatus for video decoding, comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to act as directed by the program code; having Apparatus, wherein the program code, when executed by the at least one processor, causes the at least one processor to perform a method according to any one of claims 1 to 8.

10. A program which, when executed by at least one processor, causes the at least one processor to carry out a method according to any one of claims 1 to 8.

11. 1. A method of video encoding performed by an encoder, comprising: determining whether a coding tool for slice-based Rice coding for residual coding in transform skip mode is used during encoding of a first range of coded video data in a video bitstream, the first range of coded video data including one or more second ranges of the coded video data, the coding tool providing Rice parameters at a slice level for the residual coding in transform skip mode; if the coding tool is not used for the first range of the coded video data, encoding into the video bitstream a first syntax element corresponding to a General Constraint Information (GCI) syntax structure for coding control of pictures in an output layer set for a decoder, the first syntax element being set to indicate the presence of at least one additional bit in the GCI syntax structure; encoding in the video bitstream a second syntax element having a first value in the GCI syntax structure, the second syntax element being for coding control in a first range of the coded video data in the video bitstream, the second syntax element being related to the coding tool of the slice-based Rice coding for the residual coding in the transform skip mode, the first value indicating non-use of the coding tool in the coding of the first range of the coded video data; The method further comprises:

Citation Information

Patent Citations

  • Image decoding device and image coding device

    JP2023006203A

  • Residual and coefficients coding for video coding

    WO2022251545A1