Video data encoding and decoding

Abstract

A device includes: a video data decoder circuit configured to decode an input video data stream, the video data decoder being responsive to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format; a detector circuit configured to detect constraint data associated with the input video data stream, the constraint data defining a difference between decoding properties applicable to the input video data stream and decoding properties defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding properties by a zero value of the constraint data; and a control circuit configured to control the video data decoder to decode the input video data stream to generate a decoded video data stream having decoding properties defined by the encoding profile provided by the parameter data and modified by the difference value defined by the constraint data.

Description

Technical Field

[0001] The present disclosure relates to video data encoding and decoding. Background Art

[0002] The "background technology" description provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent described in this background section, the work of the presently designated inventors and aspects of the description that may not qualify as prior art at the time of filing are neither explicitly nor implicitly admitted as prior art against the present disclosure.

[0003] There are several systems such as video or image data encoding and decoding systems that involve transforming the video data into a frequency domain representation, quantizing the frequency domain coefficients, and then applying some form of entropy coding to the quantized coefficients. This can achieve compression of the video data. A corresponding decoding or decompression technique is applied to recover a reconstructed version of the original video data.

[0004] The encoding and decoding functions are defined by so-called profiles, levels and tiers. Among other things, a profile may define various properties of the encoding and decoding functions, such as (possibly among others) the chroma subsampling format and the bit depth (or maximum bit depth) of the video or image data. Various other functions for encoding and decoding operations (such as specific filtering operations) may also be defined by profiles. So-called constraints may be encoded in the bitstream (otherwise referred to as a data stream or video data stream) to define variations of the properties defined by a particular profile. Summary of the invention

[0005] The present disclosure solves or alleviates the problems caused by this process.

[0006] Various aspects and features of the present disclosure are defined in the following claims.

[0007] It is to be understood that both the foregoing general description and the following detailed description are exemplary and not restrictive of the technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure becomes better understood, and a more complete appreciation of the present disclosure and many of its attendant advantages will be readily obtained by referring to the following detailed description when considered in conjunction with the accompanying drawings, in which:

[0009] Figure 1 Schematically illustrates an audio / video (A / V) data transmission and reception system using video data compression and decompression;

[0010] Figure 2 schematically illustrates a video display system using video data decompression;

[0011] Figure 3Schematically illustrates an audio / video storage system using video data compression and decompression;

[0012] Figure 4 schematically illustrates a video camera using video data compression;

[0013] Figure 5 and Figure 6 A storage medium is schematically shown;

[0014] Figure 7 A schematic overview of video data compression and decompression equipment is provided;

[0015] Figure 8 The predictor is schematically shown;

[0016] Fig. 9 The use of parameter sets is schematically shown;

[0017] Fig.10 A decoding device is schematically shown;

[0018] Fig.11 schematically illustrates an encoding device; and

[0019] Figures 12 to 16 is a schematic flow chart illustrating various methods. DETAILED DESCRIPTION

[0020] Referring now to the accompanying drawings, there is provided Figures 1 to 4 A schematic diagram of a device or system utilizing the compression and / or decompression device described below in conjunction with an embodiment of the present technology is provided.

[0021] All data compression and / or decompression devices to be described below may be implemented in hardware, software running on a general-purpose data processing device such as a general-purpose computer, as programmable hardware such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), or a combination of these. Where an embodiment is implemented by software and / or firmware, it should be understood that such software and / or firmware, and the non-transitory data storage medium by which such software and / or firmware is stored or otherwise provided, are considered embodiments of the present technology.

[0022] Figure 1 Schematically illustrates an audio / video data transmission and reception system using video data compression and decompression. In this example, the data values ​​to be encoded or decoded represent image data.

[0023] An input audio / video signal 10 is provided to a video data compression device 20 which compresses at least the video component of the audio / video signal 10 for transmission along a transmission path 30 such as a cable, optical fiber, wireless link, etc. The compressed signal is processed by a decompression device 40 to provide an output audio / video signal 50. For the return path, a compression device 60 compresses the audio / video signal for transmission along the transmission path 30 to a decompression device 70.

[0024] Therefore, compression device 20 and decompression device 70 can form one node of the transmission link. Decompression device 40 and compression device 60 can form another node of the transmission link. Of course, in the case where the transmission link is unidirectional, only one node requires a compression device, and the other node only requires a decompression device.

[0025] Figure 2 A video display system using video data decompression is schematically shown. Specifically, a compressed audio / video signal 100 is processed by a decompression device 110 to provide a decompressed signal that can be displayed on a display 120. The decompression device 110 can be implemented as an integral part of the display 120, for example, being arranged in the same housing as the display device. Alternatively, the decompression device 110 can be provided as, for example, a so-called set-top box (STB), it being noted that the expression "set-top box" does not imply a requirement that the box be located in any particular orientation or position relative to the display 120; it is only a term used in the art to indicate a device that can be connected to a display as a peripheral device.

[0026] Figure 3 An audio / video storage system using video data compression and decompression is schematically shown. An input audio / video signal 130 is provided to a compression device 140 that generates a compressed signal for storage by a storage device 150, such as a magnetic disk device, an optical disk device, a magnetic tape device, a solid-state storage device such as a semiconductor memory, or other storage device. For playback, the compressed data is read from the storage device 150 and passed to a decompression device 160 for decompression to provide an output audio / video signal 170.

[0027] It should be understood that a compressed or encoded signal and a storage medium such as a machine-readable non-transitory storage medium storing the signal are considered to be embodiments of the present technology.

[0028] Figure 4 A video camera using video data compression is schematically shown. Figure 4, an image capture device 180, such as a charge coupled device (CCD) image sensor and associated control and readout electronics, generates a video signal that is passed to a compression device 190. A microphone (or microphones) 200 generates an audio signal to be passed to the compression device 190. The compression device 190 generates a compressed audio / video signal 210 (shown generally as schematic stage 220) to be stored and / or transmitted.

[0029] The techniques to be described below relate primarily to video data compression and decompression. It should be understood that many prior art techniques may be used in conjunction with the video data compression techniques to be described for audio data compression to generate a compressed audio / video signal. Thus, a separate discussion of audio data compression will not be provided. It should also be understood that the data rates associated with video data (particularly broadcast quality video data) are typically much higher than the data rates associated with audio data (whether compressed or uncompressed). Therefore, it should be understood that uncompressed audio data may accompany compressed video data to form a compressed audio / video signal. It should further be understood that, although ( Figures 1 to 4 The examples shown in FIG. 1 and FIG. 2 relate to audio / video data, but the techniques described below can be used in systems that simply process (i.e., compress, decompress, store, display, and / or transmit) video data. That is, the embodiments can be applied to video data compression without necessarily having any associated audio data processing.

[0030] therefore, Figure 4 An example of a video capture device including an image sensor and an encoding device of the type discussed below is provided. Figure 2 Examples of decoding devices of the types discussed below and displays to which decoded images are output are provided.

[0031] Figure 2 and Figure 4 A combination of may provide a video capturing apparatus including the image sensor 180 and the encoding device 190, the decoding device 110, and the display 120 to which the decoded image is output.

[0032] Figure 5 and Figure 6 A storage medium is schematically shown, storing, for example, compressed data generated by the device 20, 60, compressed data input to the device 110 or the storage medium or stage 150, 220. Figure 5 A disk storage medium such as a magnetic disk or optical disk is schematically shown, and Figure 6 A solid state storage medium such as flash memory is schematically shown. It should be noted that Figure 5 and Figure 6Examples of non-transitory machine-readable storage media storing computer software that, when executed by a computer, causes the computer to perform one or more of the methods discussed below may also be provided.

[0033] The above arrangements thus provide examples of video storage, capture, transmission or reception devices embodying any of the present techniques.

[0034] Figure 7 A schematic overview of a video or image data compression (encoding) and decompression (decoding) apparatus for encoding and / or decoding video or image data representing one or more images is provided.

[0035] The controller 343 controls the overall operation of the apparatus and, in particular, controls the trial encoding process by acting as a selector to select various modes of operation, such as block size and shape, and whether to encode the video data losslessly or otherwise, when referring to the compression mode. The controller is regarded as forming part of the image encoder or image decoder, as the case may be. Successive images of the input video signal 300 are supplied to the adder 310 and the image predictor 320. Reference will now be made to the Figure 8 The picture predictor 320 is described in more detail. The picture encoder or decoder (as the case may be) plus Figure 8 The intra-image predictor can use the Figure 7 However, this does not mean that the image encoder or decoder must need Figure 7 Each feature.

[0036] The adder 310 actually performs a subtraction (negative addition) operation, since it receives the input video signal 300 on the "+" input and the output of the image predictor 320 on the "-" input, so that the predicted image is subtracted from the input image. The result is a so-called residual image signal 330 representing the difference between the actual image and the predicted image.

[0037] One of the reasons for generating a residual image signal is as follows. The data encoding techniques to be described, i.e., the techniques applied to the residual image signal, tend to work more efficiently when there is less "energy" in the image to be encoded. Here, the term "efficiently" refers to generating a small amount of encoded data; for a particular level of image quality, it is desirable (and considered "efficient") to actually generate as little data as possible. References to "energy" in the residual image relate to the amount of information contained in the residual image. If the predicted image is the same as the true image, then the difference between the two (i.e., the residual image) will contain zero information (zero energy) and will be very easily encoded into a small amount of encoded data. In general, if the prediction process can be made to work reasonably well so that the predicted image content is similar to the image content to be encoded, then it is expected that the residual image data will contain less information (less energy) than the input image, and will therefore be easier to encode into a small amount of encoded data.

[0038] Thus, encoding (using adder 310) involves predicting an image region of an image to be encoded; and generating a residual image region based on the difference between the predicted image region and the corresponding region of the image to be encoded. In conjunction with the techniques discussed below, the ordered array of data values ​​includes data values ​​representing the residual image region. Decoding involves predicting an image region of an image to be decoded; generating a residual image region indicating the difference between the predicted image region and the corresponding region of the image to be decoded; wherein the ordered array of data values ​​includes data values ​​representing the residual image region; and combining the predicted image region and the residual image region.

[0039] The remainder of the device acting as an encoder (encoding the residual image or difference image) will now be described.

[0040] The residual image data 330 is provided to a transform unit or circuit 340, which generates a discrete cosine transform (DCT) representation of a block or region of the residual image data. DCT techniques are themselves well known and will not be described in detail herein. It should also be noted that the use of DCT illustrates only one exemplary arrangement. Other transforms that may be used include, for example, discrete sine transforms (DST). Transforms may also include a sequence or cascade of individual transforms, such as an arrangement in which one transform is followed (whether directly or not) by another transform. The choice of transform may be explicitly determined and / or may depend on side information used to configure the encoder and decoder. In other examples, a so-called "transform skip" mode in which no transform is applied may be selectively used.

[0041] Thus, in an example, the encoding and / or decoding method comprises predicting an image region of an image to be encoded; and generating a residual image region based on a difference between the predicted image region and a corresponding region of the image to be encoded; wherein an ordered array of data values ​​(discussed below) comprises data values ​​representing the residual image region.

[0042] The output of the transform unit 340, i.e. (in the example) a set of DCT coefficients for each transformed block of image data, is supplied to a quantizer 350. Various quantization techniques are known in the art of video data compression, ranging from simple multiplication by a quantization scale factor to the application of complex lookup tables under the control of a quantization parameter. The overall goal is twofold. First, the quantization process reduces the number of possible values ​​for the transformed data. Second, the quantization process can increase the probability that the value of the transformed data is zero. Both of these can make the entropy encoding process described below more efficient in generating a small amount of compressed video data.

[0043] A data scanning process is applied by scanning unit 360. The purpose of the scanning process is to reorder the quantized transform data so as to collect together as many non-zero quantized transform coefficients as possible, and of course therefore collect together as many zero-valued coefficients as possible. These features may allow so-called run-length encoding or similar techniques to be applied efficiently. Thus, the scanning process involves selecting coefficients from the quantized transform data, and specifically from coefficient blocks corresponding to blocks of image data that have been transformed and quantized, according to a "scanning order" such that (a) all coefficients are selected once as part of the scan, and (b) the scan tends to provide the desired reordering. One exemplary scanning order that may tend to give useful results is a so-called top-right diagonal scanning order.

[0044] The scanning order may differ, for example, between transform skipped blocks and transformed blocks (blocks that have undergone at least one spatial frequency transform).

[0045] The scanned coefficients are then passed to an entropy encoder (EE) 370. Again, different types of entropy encoding may be used. Two examples are variants of the so-called CABAC (context-based adaptive binary arithmetic coding) system and variants of the so-called CAVLC (context-based adaptive variable length coding) system. In general, CABAC is seen as providing better efficiency, and in some studies, it has been shown that the amount of encoded output data is reduced by 10-20% for comparable image quality compared to CAVLC. However, CAVLC is seen as representing a much lower level of complexity (in terms of its implementation) than CABAC. It should be noted that the scanning process and the entropy encoding process are shown as separate processes, but can actually be combined or processed together. That is, reading the data into the entropy encoder may occur in a scanning order. Corresponding considerations apply to the various inverse processes described below.

[0046] The output of the entropy encoder 370 , along with additional data (mentioned above and / or discussed below) that, for example, defines the manner in which the predictor 320 generates the predicted picture (whether the compressed data is transformed or the transform is skipped, etc.), provides a compressed output video signal 380 .

[0047] However, a return path 390 is also provided because the operation of the predictor 320 itself depends on the decompressed version of the compressed output data.

[0048] The reason for this feature is as follows. At an appropriate stage in the decompression process (described below), a decompressed version of the residual data is generated. This decompressed residual data must be added to the predicted image to generate the output image (because the original residual data is the difference between the input image and the predicted image). In order to make the process comparable between the compression side and the decompression side, the predicted image generated by the predictor 320 should be the same during the compression process and during the decompression process. Of course, when decompressing, the device does not have access to the original input image, but only to the decompressed image. Therefore, when compressing, the predictor 320 bases its prediction (at least for inter-image coding) on ​​the decompressed version of the compressed image.

[0049] The entropy encoding process performed by the entropy encoder 370 is (in at least some examples) "lossless", that is, it can be reversed to obtain exactly the same data that was first supplied to the entropy encoder 370. Therefore, in such examples, the return path can be implemented before the entropy encoding stage. In fact, the scanning process performed by the scanning unit 360 is also considered to be lossless, so in this embodiment, the return path 390 is from the output of the quantizer 350 to the input of the complementary inverse quantizer 420. In examples where losses or potential losses are introduced by a stage, the stage (and its inverse) can be included in the feedback loop formed by the return path. For example, the entropy encoding stage can be lossy at least in principle, for example, by a technique of encoding bits within the parity information. In this case, entropy encoding and decoding should form part of the feedback loop.

[0050] In general, the entropy decoder 410, the inverse scan unit 400, the inverse quantizer 420, and the inverse transform unit or circuit 430 provide the respective inverse functions of the entropy encoder 370, the scan unit 360, the quantizer 350, and the transform unit 340. The discussion of the compression process will now continue; the process of decompressing an input compressed video signal will be discussed separately below.

[0051] During compression, the scan coefficients are passed from the quantizer 350 via a return path 390 to an inverse quantizer 420 which performs the inverse operation of the scanning unit 360. The inverse quantization and inverse transform processes are performed by units 420, 430 to generate a compressed-decompressed residual image signal 440.

[0052] At adder 450, the image signal 440 is added to the output of predictor 320 to generate a reconstructed output image 460 (although this may be subjected to so-called loop filtering and / or other filtering before output, see below). This will form one input to the image predictor 320, as will be described below.

[0053] Turning now to the decoding process applied to decompress the received compressed video signal 470, the signal is supplied to the entropy decoder 410 and thence to the chain of inverse scanning unit 400, inverse quantizer 420 and inverse transform unit 430, before being added to the output of the image predictor 320 by an adder 450. Thus, on the decoder side, the decoder reconstructs a version of the residual image, which is then applied (by adder 450) to the predicted version of the image (on a block-by-block basis) in order to decode each block. In simple terms, the output 460 of the adder 450 forms the output decompressed video signal 480 (subject to the filtering process discussed below). In practice, the signal may optionally be filtered (e.g. by a filter) before being output. Figure 8 The loop filter 565 shown in FIG. Figure 7 The clarity of the higher-level diagram, from Figure 7 This loop filter is omitted in Figure 3) to apply more filtering.

[0054] Figure 7 and Figure 8 The device can act as a compression (encoding) device or a decompression (decoding) device. The functions of the two types of devices are substantially overlapping. Scanning unit 360 and entropy encoder 370 are not used in decompression mode, and the operation of predictor 320 and other units (described in detail below) follows the mode and parameter information contained in the received compressed bit stream, rather than generating such information themselves.

[0055] Figure 8 The generation of a predicted image, and in particular the operation of the image predictor 320 , is schematically illustrated.

[0056] There are two basic prediction modes performed by the image predictor 320: so-called intra-image prediction and so-called inter-image or motion compensated (MC) prediction. On the encoder side, each involves detecting the prediction direction with respect to the current block to be predicted and generating a predicted block of samples from other samples (in the same (intra) or another (inter) image). The difference between the predicted block and the actual block is encoded or applied by means of the unit 310 or 450 in order to encode or decode the block, respectively.

[0057] (At the decoder, or on the reverse decoding side of the encoder, detection of the prediction direction may be responsive to data associated with data encoded by the encoder, indicating which direction to use at the encoder. Alternatively, the detection may be responsive to the same factors as those used to make the decision at the encoder).

[0058] Intra-frame picture prediction predicts the content of a block or region of a picture based on data from within the same picture. This corresponds to so-called I-frame coding in other video compression techniques. However, in contrast to I-frame coding, which involves encoding the entire picture by intra-coding, in this embodiment the selection between intra-frame coding and inter-frame coding can be made on a block-by-block basis, although in other embodiments the selection is still made on a picture-by-picture basis.

[0059] Motion compensated prediction is an example of inter-image prediction and exploits motion information from a source in another adjacent or nearby image that attempts to define image detail to be encoded in the current image. Thus, in an ideal example, the contents of a block of image data in the predicted image can be encoded very simply as a reference (motion vector) pointing to a corresponding block at the same or slightly different location in an adjacent image.

[0060] A technique called "block copy" prediction is in some ways a hybrid of the two, as it uses a vector to indicate a block of samples at a position displaced from the current prediction block within the same picture that should be copied to form the current prediction block.

[0061] return Figure 8 , two image prediction arrangements are shown (corresponding to intra-frame and inter-frame image prediction), the results of which are selected by multiplexer 500 under the control of mode signal 510 (e.g., from controller 343) to provide predicted image blocks for supply to adders 310 and 450. The selection is made according to which selection gives the lowest "energy" (which can be regarded as the information content that needs to be encoded, as described above), and the selection is signaled to the decoder within the encoded output data stream. In this context, the image energy can be detected, for example, by performing trial subtraction of a region of two versions of the predicted image from the input image, squaring each pixel value of the difference image, summing the squared values, and identifying which of the two versions produces a lower mean square value of the difference image associated with the image region. In other examples, trial encoding can be performed for each selection or potential selection, and then a selection is made according to the cost of each potential selection in terms of one or both of the number of bits required for encoding and the distortion of the picture.

[0062] In an intra coding system, the actual prediction is performed based on the image blocks received as part of the signal 460 (e.g. filtered by loop filtering; see below), i.e. the prediction is based on the coded-decoded image blocks so that an identical prediction can be performed at the decompression device. However, data may be derived from the input video signal 300 by the intra mode selector 520 to control the operation of the intra image predictor 530.

[0063] For inter picture prediction, a motion compensated (MC) predictor 540 uses motion information such as motion vectors derived from the input video signal 300 by a motion estimator 550. Those motion vectors are applied by the motion compensated predictor 540 to a processed version of the reconstructed picture 460 to generate an inter picture prediction block.

[0064] Thus, units 530 and 540 (operating in conjunction with estimator 550) each act as a detector to detect a prediction direction with respect to a current block to be predicted, and as a generator to generate a prediction block of samples (forming part of the prediction passed to units 310 and 450) based on other samples defined by the prediction direction.

[0065] The processing applied to signal 460 will now be described.

[0066] First, the signal may be filtered by a so-called loop filter 565. Various types of loop filters may be used. One technique involves applying a "deblocking" filter to remove, or at least tend to reduce, the effects of block-based processing and subsequent operations performed by the transform unit 340. Another technique involving applying a so-called sample adaptive offset (SAO) filter may also be used. Generally speaking, in a sample adaptive offset filter, filter parameter data (derived at an encoder and transmitted to a decoder) defines one or more offsets to be selectively combined with a given intermediate video sample (a sample of signal 460) by the sample adaptive offset filter as a function of the values ​​of: (i) a given intermediate video sample; or (ii) one or more intermediate video samples having a predetermined spatial relationship with the given intermediate video sample.

[0067] Furthermore, an adaptive loop filter is optionally applied using coefficients derived by processing the reconstructed signal 460 and the input video signal 300. An adaptive loop filter is a type of filter that uses known techniques to apply adaptive filter coefficients to the data to be filtered. In other words, the filter coefficients may vary depending on various factors. Data defining which filter coefficients are used is included as part of the encoded output data stream.

[0068] The techniques discussed below relate to the processing of parameter data related to the operation of the filter. The actual filtering operation (such as SAO filtering) may use other known techniques.

[0069] When the device operates as a decompression device, the filtered output from the loop filter unit 565 actually forms the output video signal 480. It is also buffered in one or more image or frame memories 570; the storage of consecutive images is a requirement for motion compensated prediction processing, and in particular the generation of motion vectors. In order to save storage requirements, the images stored in the image memory 570 can be saved in compressed form and then decompressed for the generation of motion vectors. For this particular purpose, any known compression / decompression system can be used. The stored images can be passed to the interpolation filter 580, which generates a higher resolution version of the stored images; in this example, intermediate samples (subsamples) are generated so that the resolution of the interpolated images output by the interpolation filter 580 is 4 times (in each dimension) the resolution of the images stored in the image memory 570 for the 4:2:0 luminance channel, and 8 times (in each dimension) the resolution of the images stored in the image memory 570 for the 4:2:0 chrominance channels. The interpolated image is passed as input to the motion estimator 550 and also to the motion compensated predictor 540 .

[0070] The manner of segmenting an image for compression processing will now be described. At a basic level, the image to be compressed is viewed as an array of blocks or regions of samples. The division of an image into such blocks or regions may be performed by a decision tree, such as described in SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Infrastructure of audiovisual services-Coding of moving video High efficiency video coding Recommendation ITU-TH.265 12 / 2016. And High Efficiency Video Coding (HEVC) Algorithms and Architectures, chapter 3, Editors: Madhukar Budagavi, Gary J. Sullivan, Vivienne Sze; ISBN 978-3-319-06894-7; 2014, the entire contents of each of which are incorporated herein by reference. More background information is provided in "Versatile Video Coding (Draft 8)", JVET-R2001-vA, B. Bross, J. Chen, S. Liu and YK. Wang, which is incorporated herein by reference in its entirety.

[0071] In some examples, the resulting blocks or regions have a size and in some cases a shape that, with the aid of a decision tree, can generally follow the arrangement of image features within the image. This in itself can allow for improved coding efficiency, as samples representing or following similar image features will tend to be grouped together by such an arrangement. In some examples, square blocks or regions of different sizes (e.g., 4×4 samples, up to 64×64 or larger blocks) can be used for selection. In other exemplary arrangements, blocks or regions of different shapes can be used, such as rectangular blocks (e.g., vertically or horizontally oriented). Other non-square and non-rectangular blocks are contemplated. The result of dividing the image into such blocks or regions is that (at least in this example) each sample of the image is assigned to and only to one such block or region.

[0072] Embodiments of the present disclosure to be discussed below relate to techniques for representing encoding levels of encoders and decoders.

[0073] Coding Tree Unit (CTU)

[0074] CTU stands for reference Figure 7 The basic processing unit of the device under discussion. They may also be referred to as Largest Decoding Units (LCUs). Typically, CTUs may be between 16×16 pixels and 128×128 samples, with larger sizes typically (although not exclusively, e.g., for smaller image sizes) providing increased coding efficiency.

[0075] A CTU can be subdivided down into other units such as coding tree blocks (CTBs), coding units (CUs) (where the CTU size is also the maximum CU size), prediction units (PUs), and transform units (TUs) using established techniques.

[0076] Parameter Sets and Coding Profiles, Levels and Constraints

[0077] When video data is encoded for subsequent decoding by the techniques discussed above, it is appropriate for the encoding side of the process to communicate some parameters of the encoding process to the eventual decoding side of the process. Given that these encoding parameters are needed whenever the encoded video data is decoded, it is useful to associate the parameters with the encoded video data stream itself, for example (although not necessarily exclusively, as they may be sent "out of band" by a separate transmission channel) by embedding the parameters in the encoded video data stream itself as so-called parameter sets.

[0078] Parameter sets may be represented as a hierarchy of information, for example as a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS). The PPS is expected to appear once per picture and is expected to contain information about all coded slices in that picture, the SPS is less frequent (once per picture sequence) and the VPS is less frequent still. More frequently occurring parameter sets (such as the PPS) may be implemented as references to previously encoded examples of the parameter set to avoid the cost of re-encoding. Each coded image slice references a single valid PPS, SPS, and VPS to provide information to be used for decoding the slice. Specifically, each slice header may contain a PPS identifier to reference the PPS, which in turn references the SPS, which in turn references the VPS.

[0079] Another form of parameter information is the so-called Decoding Capability Information (DCI), which is actually a parameter set (although no longer called that by name) indicating the maximum requirements that will be passed along a particular data connection. This general definition can be useful because video streams (VPSs) can be started and stopped, but docked with each other. For example, a change in video channels viewed via a particular broadcast or video on demand provider may switch VPSs without actually making a new connection, or in the case of a live TV broadcast being interrupted by a commercial, the interruption may be achieved by switching or tying VPSs.

[0080] The VPS gives information about video sequences. This may be scalable, in which case there are multiple sequences within the video stream. If not scalable, there will be only one sequence within the video stream. The SPS gives information about one of the video sublayers - described as a sequence. Then, there will usually be only one layer in the VPS, so there will be only one SPS. The PPS gives information about a single picture.

[0081] Further examples include the so-called Adaptive Parameter Set (APS), which is applicable to each slice and is expected to contain information that changes more often than other parameter sets.

[0082] Apart from the APS, all may be transmitted rarely - but decoding cannot start until the required information has been sent. Thus, for broadcast, the information may be repeated, for example, every 1 second, with random entry into the stream (e.g., the viewer may initiate decoding at any point in time).

[0083] The DCI, VPS and SPS may each contain a set of constraints of the form discussed below. The DCI must contain a set of these constraints, representing the worst case for the connection.

[0084] Among these parameter sets, the SPS contains example information related to some of the information discussed below, namely data defining the so-called profiles, tiers and coding levels to be used.

[0085] A profile defines a set of decoding tools or functions to be used. Exemplary profiles include the "Main Profile" which is associated with 8-bit 4:2:0 video, and the "Main 10 Profile" which allows 10-bit resolution and other extensions relative to the Main Profile.

[0086] In general, profiles, tiers and levels specify restrictions on the coded bitstream. Indirectly, they define the capabilities required to decode the bitstream. A decoder operating on a specific profile, tier and level needs to be able to decode all examples of the coded bitstream that conform to that profile, tier and level.

[0087] The decoding process is specified by the relevant standards and draft standards such that all decoders conforming to the specified profile and level (and operating with respect to a bitstream conforming to that profile and level) will generate numerically identical output.

[0088] A profile may define a set of parameters including, for example, the chroma subsampling scheme (4:4:4, 4:2:2, 4:2:0, etc.).

[0089] It is also possible to encode so-called constraints. Constraints can refer to one of two categories of information:

[0090] a) Selective disabling of specific features of encoders and decoders; in this context, constraints are generally considered to be reducing because they allow or cut off features that would otherwise be available within a specific profile. Examples of features that can be selectively disabled in this way include SAO (sample adaptive offset) filters, ALF (adaptive loop filter), the use of multiple transforms such as DCT and DST, and so-called palette modes. It should be noted that in the so-called Versatile Video Coding (VVC) arrangement, the constraint flag makes a requirement on another flag. For example, the constraint no_alf_constraint_flag equal to 1 specifies that sps_alf_enabled_flag (its ALF is valid) should be equal to 0. The flag no_alf_constraint_flag equal to 0 does not impose such a constraint. In principle, if a full decoder can handle every possible option, it does not need to pay attention to constraints. However, if there is a decoder that can only handle some aspects, it can (does not have to) check these flags. In addition, if a profile (or sub-profile) is defined that requires some tools to be turned off, the bitstream will not be consistent unless the constraint flag is set. Constraint flags may also exist at higher levels and represent multiple streams. They may also be used for scalable / layered coding, where they describe constraints on scalable streams that may represent a single scalable video stream.

[0091] Therefore, they describe the maximum value used, and then all bit depths and chroma formats of the sub-part of the stream are constrained to this maximum value.

[0092] b) Selective variation of a particular parameter (i.e., a default parameter associated with a profile may be changed to another value for that parameter); examples here include chroma subsampling format and bit depth or maximum bit depth (which may actually be specified as "bit depth minus 8" in some examples). For example, such a constraint may define encoder settings associated with a 10-bit profile of (e.g.) 4:2:0 but connected to 8-bit video data of 4:0:0. Of course, these constraints may be defined separately and independently of other constraints as the technical background allows. The constraints may relate to the difference between the encoding / decoding properties applicable to the video data stream and the corresponding properties defined by the profile indicated by the parameter data. It should be noted that "difference" does not necessarily mean a numerical or mathematical difference. In other words, defining "the difference between A and B" and indicating it through constraint data indicates that A is not equal to B, and the constraint data informs what the difference is, for example, by mapping to an alternative value, rather than the constraint data providing a direct numerical indication of the alternative value.

[0093] The exemplary embodiment involves constraints of the type (b) just described.

[0094] The coding level provides constraints on things like maximum sampling rate and picture size. The level imposes a maximum data rate.

[0095] In the JVET (Joint Video Experts Team) recommendations for Versatile Video Coding (VVC), such as those defined by the above-referenced specification JVET-R2001-vA (at the date of submission), different levels are defined from 1 to 6.2.

[0096] Example use of parameter sets

[0097] Exemplary implementations will now be described with reference to the accompanying drawings.

[0098] Fig. 9 The use of video parameter sets and sequence parameter sets discussed above is schematically illustrated. Specifically, these form part of the hierarchy of parameter sets mentioned above, so that multiple sequence parameter sets (SPS) 900, 910, 920 can refer to the example of decoding capability information (DCI) 930 and are then referenced by the corresponding sequences 902, 912, 922 themselves. In an exemplary embodiment, the profile information is defined in the DCI. So-called constraint flags or other forms of constraint data are provided in the DCI and optionally in the SPS, or alternatively (or in addition) by a mechanism that allows scalable coding (such as, a so-called video parameter set (VPS)). Level information applicable to the corresponding sequence can be provided in the sequence parameter set. However, it should be understood that these information types can be provided in these and / or other parameter sets or positions relative to the encoded bitstream.

[0099] However, in other embodiments, it should be appreciated that the profile information may be provided in a different form or a different set of parameters.

[0100] Note that a "profile" may actually be defined by a hierarchy of profile information, e.g., as profiles and sub-profiles, where the net effect of two (or more) instances is considered to be effectively a "profile" applicable to a particular stream.

[0101] Similarly, although Fig. 9 The schematic representation of shows that the DCI and sequence parameter sets are provided as part of the entire video data stream 940, but the DCI and sequence parameter sets (or other data structures carrying the information discussed herein) may alternatively be provided by separate communication channels. In either case, the profile information and constraints are associated with the video data stream 940.

[0102] Fig. 9 Also provided are illustrative examples of a data area 950 within a DCI 930 that may carry at least profile information and / or a constraint flag or other constraint data, and an example of a data area 960 within an SPS that may optionally carry at least a constraint flag or other constraint data.

[0103] Example Operation - Decoder

[0104] Fig.10 Schematically illustrates aspects of a decoding device configured to receive an input (encoded) video data stream 1000 and to decode the video data using the above referenced Figure 7 The decoder 1020 of the discussed form is used to generate the output decoded video data stream 1010. For clarity of this description, Figure 7 The control circuit or controller 343 is drawn separately to the rest of the decoder 1020.

[0105] Within the functionality of the controller or control circuit 343 is a parameter set (PS) detector 1030 that detects various parameter sets including one or more of DCI, VPS, SPS, and PPS from appropriate fields of the input video data stream 1000. The parameter set detector 1030 derives information from the parameter sets including the levels discussed above. This information is passed to the rest of the control circuit 343. It should be noted that the parameter set detector 1030 may decode the levels or may simply provide the encoded levels to the control circuit 343 for decoding.

[0106] The control circuit 343 is also responsive to one or more decoder parameters 1040 that define at least, for example, the level at which the decoder 1020 is capable of decoding.

[0107] The control circuit 343 detects whether the decoder 1020 is capable of decoding a given or current input video data stream 1000 and controls the decoder 1020 accordingly. The control circuit 343 may also provide various other operating parameters to the decoder 1020 in response to information obtained from the parameter set detected by the parameter set detector 1030.

[0108] therefore, Fig.10 An example of a device is provided, the device comprising:

[0109] a video data decoder circuit 1020 configured to decode an input video data stream, the video data decoder being responsive to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format;

[0110] a detector circuit 1030 configured to detect constraint data associated with the input video data stream, the constraint data defining a difference between decoding properties applicable to the input video data stream and decoding properties defined by a profile indicated by the parameter data; and

[0111] The control circuit 343 is configured to control the video data decoder to decode the input video data stream to generate a decoded video data stream having decoding properties defined by the encoding profile provided by the parameter data and modified by the difference value defined by the constraint data.

[0112] Example Operation - Encoder

[0113] In a similar manner, Fig.11 Schematically illustrates aspects of an encoding device, the encoding device comprising, for example, the above reference Figure 7 An encoder 1100 of the type discussed. For clarity of explanation, the control circuitry 343 of the encoder is drawn separately. The encoder acts on an input video data stream 1110 to generate and output an encoded video data stream 1120 under the control of the control circuitry 343, which in turn responds to encoding parameters 1130 including definitions of encoding profiles and / or levels to be applied.

[0114] It should be noted that Fig.11 The parameters provided by the exemplary unit 1130 for use at the encoder may be referred to as "encoding parameters" (e.g., Fig.11 ), but in the context of transmission to an encoder, the parameters are also effectively called "decoding attributes" because they control both the encoding and decoding processes.

[0115] The control circuit 343 also includes or controls a parameter set generator 1140 that generates parameter sets including, for example, DCI, VPS, SPS and PPS to be included in the output encoded video data stream.

[0116] therefore, Fig.11 An example of a device is provided, the device comprising:

[0117] a video data encoder circuit 1100 configured to encode an input video data stream to generate an output encoded video data stream, the video data encoder being responsive to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format; and

[0118] parameter encoder circuit 1140 configured to encode decoding properties associated with the output encoded video data stream according to the configuration data;

[0119] wherein the parameter encoder circuit is configured to encode parameter data indicating a profile selected from a plurality of profiles, each profile defining a decoding property, and constraint data defining a difference between the decoding property defined by the configuration data and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference in the decoding property by a zero value of the constraint data.

[0120] Constraint Encoding

[0121] Figures 12 to 16 is a schematic flow chart defining a corresponding method involving various aspects of encoding and decoding constraints such as those of type (b) discussed above, although Fig.12 and Fig.13 The general principles of the specific flowchart of FIG. 1 are applied to constraints of type (a) and type (b). The subsequent figures ( Figures 14 to 16 ) will refer to a specific constraint of type (b).

[0122] refer to Fig.12 At the encoder, in step 1200, a configuration file referred to as a "base configuration file" in the figure is established as a base configuration of the encoder. For example, the configuration may be Fig.11 The configuration data maintained by unit 1130 also defines any differences from the base configuration file in exemplary step 1210. It should be noted that the encoder may in fact operate exactly according to the base configuration file, in which case there are no differences to note in step 1210. However, even in these cases, a constraint flag or other constraint data indicating "no difference" is still used.

[0123] In step 1220, parameter data indicating a base profile and constraint data indicating whether there is a difference with the base profile are encoded into one or more parameter sets, such as DCI and / or optionally VPS and / or optionally SPS. The generation of parameter data and constraint data may be performed by one or both of the control circuit 343 and the parameter set generator 1140, wherein the parameter set generator 1140 participates in encoding the parameter data and constraint data into the encoded stream 1120.

[0124] It should be noted that the constraint data is provided in a predetermined format that can be generated by the encoder and recognized by the decoder, for example, as an ordered sequence of data fields in an order established in both the encoder and the decoder. This means that within the constraint data, each data field does not need to be marked or identified, because the decoder knows their respective meaning or function by their position relative to other data fields and / or header fields of the constraint data.

[0125] In step 1230, the encoder 1100, under the control of the control circuit 343, encodes the required input video data stream to generate an output encoded video data stream according to the base profile modified by any changes defined by the constraint data.

[0126] Fig.13 Corresponding operations on the decoder side are involved. The decoder receives an input coded video data stream and generates output decoded video data. The input stream may include parameter sets such as DCI and / or VPS and / or SPS discussed above.

[0127] In step 1300, the decoder (specifically, the parameter set detector 1030) detects a base profile defined by parameter data in one or more of the parameter sets such as DCI and / or VPS and / or SPS. In step 1310, the decoder (again, the parameter set detector 1030) detects constraint data including one or more constraint flags or other constraint information from the parameter sets such as DCI and / or VPS and / or SPS, the constraint data indicating any differences from the base profile.

[0128] In step 1320, decoder parameters relating to the base profile modified by any modifications defined by the constraint data are generated by the parameter set detector 1030 and / or the control circuit 343, and in step 1330, decoding is performed by the unit 1020 according to the decoder parameter set in step 1320 under the control of the control circuit 343.

[0129] It should be noted that Fig.12 and Fig.13In the embodiment, steps 1200, 1210, 1220 and steps 1300, 1310, 1320 are shown as separate sequential steps purely for illustrative purposes and clarity of illustration, but these groups of steps may be performed as a single composite operation in a working embodiment.

[0130] Constrained Data Generation - Encoder

[0131] Fig.14 and Fig.15 is a schematic flow chart related to the operation of generating constraint data of type (b) at the encoder side, that is, the operation performed in the above step 1220. Fig.14 Flowchart or Fig.15 , and the operations defined by the corresponding flowchart may be used for each example of constraint data of type (b). For example, in a system where constraint data of type (b) relates to each of a chroma subsampling format and a bit depth or a maximum bit depth (e.g., represented by "maximum bit depth - 8"), with respect to each of these two examples of constraint data of type (b), the operations defined by the corresponding flowchart may be followed. Fig.14 or Fig.15 operation.

[0132] First reference Fig.14 In step 1400, any difference in the parameter under consideration is detected relative to the dominant base profile. In step 1410, if there is no difference, control passes to step 1420, where the corresponding constraint is encoded as a constraint value of 0. However, if a difference is detected in step 1410, control passes to step 1430, where the corresponding constraint is encoded as another value, such as a value from a predetermined set of candidate constraint values, where each set is not equal to 0.

[0133] Potential advantages of using a value of 0 to indicate "no changes to base profile" may include the relative ease of encoding zeros into the encoded output data stream and / or the relative ease with which a decoder detects blocks or groups of zeros in the constraint data, indicating that no changes need to be implemented to the dominant profile.

[0134] Therefore, in an example, a mapping of a set of candidate values ​​of constraint data to corresponding difference amounts is used.

[0135] In other words, the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data. Similarly, in an example, the constraint data is configured to indicate a non-zero difference value for the decoding property by a non-zero value of the constraint data.

[0136] Examples of candidate constraint values ​​and exemplary mappings of these values ​​to corresponding meanings are provided below. It should be noted that exemplary embodiments may implement one or more of these examples. If more than one example is implemented, each example may have its own corresponding constraint data, as described above, or in other arrangements, the examples may share constraint data. The term "constraint data" may refer to individual constraints such as (i) or (ii) below, or to an overall set of constraint data.

[0137] (i) Chroma subsampling format

[0138]

[0139] (ii) Bit depth (which may actually be represented by "bit depth - 8")

[0140]

[0141] This range of possible values ​​does not explicitly prohibit bad practices at the encoder, for example by not allowing combinations of base profile and constraint data which, when combined, result in technically impossible or at least technically insensitive operating modes, for example operation with a 10-bit base profile in 4:2:0, but with constraint data defining a bit depth of 16 bits. Fig.15 Resolve this potential problem. Fig.15 In the embodiment, steps 1500, 1510, 1540 and 1550 correspond to respective steps 1400, 1410, 1430 and 1420 and will not be further described herein. However, at a "yes" result in step 1510, control schematically proceeds to step 1520 where the detected difference is tested to see if it is an allowable difference.

[0142] Here, the allowable difference may be considered a "subtractive" difference, such that the base profile is not enhanced in terms of capabilities by the constraint data.

[0143] For example, when the decoding attribute is bit depth, the corresponding difference amounts of the candidate values ​​mapped to the constraint data all represent negative differences in the bit depth, so that the bit depth modified by the difference defined by the constraint data as defined by the encoding profile provided by the parameter data is a bit depth that is no greater than the bit depth defined by the encoding profile provided by the parameter data.

[0144] As an example, when the base profile involves (for example) 10-bit data, the allowable values ​​for the constraint of bit depth would be 0, 1, and 2. Such restriction on the range of allowable values ​​can potentially make the encoding of the constraint more efficient.

[0145] In the context of chroma subsampling formats, respective difference amounts mapped to candidate values ​​of the constraint data all represent difference values, such that the chroma subsampling format defined by the encoding profile provided by the parameter data and modified by the difference value defined by the constraint data is a chroma subsampling format that is no higher in the predetermined hierarchy of chroma subsampling formats than the chroma subsampling format defined by the encoding profile provided by the parameter data. For example, in the predetermined hierarchy of chroma subsampling formats, the predetermined hierarchy of chroma subsampling formats, in order from low to high, may include:

[0146] i.4:0:0 (monochrome) chroma subsampling format;

[0147] ii.4:2:0 chroma subsampling format;

[0148] iii. 4:2:2 chroma subsampling format; and

[0149] iv.4:4:4 chroma subsampling format.

[0150] As an example, when the base profile involves (for example) a 4:2:0 sub-sampling format, the allowable constraint values ​​according to this level will be 0 and 1. Again, this restriction on the range of allowable values ​​can potentially make the encoding of the constraints more efficient.

[0151] Assuming the result of step 1520 is "yes", i.e., the detected difference is an allowable difference, e.g., as described above, control proceeds to step 1540, where the difference is encoded by mapping to a constraint value not equal to 0. However, if the result of step 1520 is negative, such that the detected difference is not an allowable difference, various results may occur. One example is triggering a fatal error on the encoder side. Fig.15 Another example schematically shown in is an error condition being triggered, but in encoder operation, the handling of the error condition does not result in a fatal error. Instead, the detected difference is set to a zero difference (so that the particular parameter in question operates according to its definition by the base configuration file) and control is returned to step 1510, which will then be followed by a "no" result to step 1550. Also, returning to reference Fig.12 , and note that Fig.15 This represents an example of the operation of step 1220, in which the encoding in step 1230 is performed based on a zero difference value in the parameter in question.

[0152] therefore, Fig.14 or Fig.15 Combination Fig.12 An example of a method is provided, the method comprising:

[0153] encoding 1230 an input video data stream to generate an output encoded video data stream in response to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format; and

[0154] The decoded attributes are encoded 1220 (e.g., by encoding parameter data and constraint data indicating a profile selected from a plurality of profiles) according to the configuration data. Fig.14 or Fig.15 Further defined) is associated with the output encoded video data stream, each profile defines a decoding attribute, the constraint data defines a difference between the decoding attribute defined by the configuration data and the decoding attribute defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference in the decoding attribute by a zero value of the constraint data.

[0155] In and by Fig.15 In more examples related to the defined operations, the present disclosure provides a method, including:

[0156] encoding 1230 an input video data stream to generate an output encoded video data stream in response to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format; and

[0157] The decoded attribute is encoded 1220 (in accordance with the configuration data) by encoding parameter data and constraint data indicating a configuration file selected from a plurality of configuration files. Fig.15 ) is associated with an output coded video data stream, each profile defining decoding properties, the constraint data defining a difference between the decoding properties defined by the configuration data and the decoding properties defined by the profile indicated by the parameter data, wherein the constraint data comprises a set of candidate constraint data values ​​all relating to differences in corresponding decoding properties such that the difference in capability defined by the constraint data does not enhance the profile.

[0158] The present disclosure also provides a method for operating according to the method Fig.11 corresponding device of the type shown in .

[0159] Constrained Data Detection-Decoder

[0160] Fig.16 Provides Fig.13 Examples of operations associated with steps 1300 ... 1320. Fig.16, on the decoder side, in step 1600, the base profile and constraints are detected, and for a given constraint of type (b), it is detected in step 1610 whether the constraint is equal to 0. If the result is "yes", control proceeds to step 1620, in which the corresponding parameters are used according to their values ​​defined by the base profile. If the result of step 1610 is "no", control proceeds to step 1630, in which the constraints are decoded by applying an inverse mapping using, for example, the mapping information discussed above, and the difference defined by the constraints is applied to the corresponding attributes or parameters defined by the base profile. Then, control proceeds to Fig.13 Step 1330 is used to decode according to the derived decoder parameters.

[0161] therefore, Fig.16 Combination Fig.13 An example of a method is provided, the method comprising:

[0162] decoding the input video data stream in response to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format;

[0163] detecting 1310, 1600 constraint data associated with the input video data stream, the constraint data defining a difference between a decoding property applicable to the input video data stream and a decoding property defined by a profile indicated by the parameter data, wherein the constraint data is configured such that a zero value of the constraint data indicates a zero difference value of the decoding property; and

[0164] The decoding steps are controlled 1320, 1620, 1630, 1330 to generate a decoded video data stream having decoding properties defined by the encoding profile provided by the parameter data, modified by the difference values ​​defined by the constraint data.

[0165] Naming convention

[0166] It should be understood that in the context of different video encoding and decoding settings (such as standards, draft standards, ad hoc standards, industry practices, etc.) or even different generations or implementations of a particular setting, the naming conventions applied to different examples of configuration or parameter data may vary. Therefore, although the present disclosure and embodiments refer to a "configuration file", it should be understood that the use of this term does not imply a requirement to adhere to any particular setting (although exemplary settings are mentioned in the description given above), and data structures that provide defined parameters (including decoding properties of one or more of bit depth and chroma subsampling format) are considered to represent examples of "configuration files", even if they are not referred to as "configuration files" in the relevant documents defining the encoding and decoding settings for the reasons given above or other reasons.

[0167] Encoded video data

[0168] Video data encoded by any of the techniques disclosed herein is also considered to represent embodiments of the present disclosure. A non-transitory machine-readable storage medium (such as a Figure 5 and Figure 6 One of the media schematically shown in ) is also considered to represent an embodiment of the present disclosure.

[0169] Summarize

[0170] To the extent that embodiments of the present disclosure are described as being implemented at least in part by a data processing device controlled by software, it should be understood that non-transitory machine-readable media carrying such software, such as optical disks, magnetic disks, semiconductor memories, etc., are also considered to represent embodiments of the present disclosure. Similarly, data signals including encoded data generated according to the methods discussed above (whether or not embodied on a non-transitory machine-readable medium) are also considered to represent embodiments of the present disclosure.

[0171] Obviously, in light of the above teachings, various modifications and variations of the present disclosure are possible.Therefore, it should be understood that, within the scope of the appended clauses, the present technology can be practiced in ways other than those specifically described herein.

[0172] It should be understood that for clarity, the above description has described embodiments with reference to different functional units, circuits and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuits and / or processors may be used without departing from the embodiments.

[0173] The described embodiments may include any suitable form of hardware, software, firmware or any combination of these. The described embodiments may optionally be implemented at least in part as computer software running on one or more data processors and / or digital signal processors. The elements and components of any embodiment may be implemented physically, functionally and logically in any suitable manner. In fact, the functions may be implemented in a single unit, in multiple units or as a part of other functional units. Thus, the disclosed embodiments may be implemented in a single unit, or may be physically and functionally distributed between different units, circuits and / or processors.

[0174] Although the present disclosure has been described in conjunction with some embodiments, it is not intended to be limited to the specific form set forth herein. In addition, although features may appear to be described in conjunction with specific embodiments, those skilled in the art will recognize that the various features of the described embodiments may be combined in any manner suitable for implementing the technology.

[0175] The corresponding aspects and features are defined by the following numbered clauses:

[0176] 1. A device comprising:

[0177] a video data decoder circuit configured to decode an input video data stream, the video data decoder responsive to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties including one or more of a bit depth and a chroma subsampling format;

[0178] a detector circuit configured to detect constraint data associated with the input video data stream, the constraint data defining a difference between a decoding property applicable to the input video data stream and a decoding property defined by said profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference in the decoding property by a zero value of the constraint data; and

[0179] The control circuit is configured to control the video data decoder to decode the input video data stream to generate a decoded video data stream having decoding properties defined by the encoding profile provided by the parameter data and modified by the difference value defined by the constraint data.

[0180] 2. Apparatus according to clause 1, wherein the constraint data is configured to indicate a non-zero difference value of the decoding property by a non-zero value of the constraint data.

[0181] 3. Apparatus according to clause 2, wherein the detector circuit is configured to generate the difference value of the decoding property by mapping a set of candidate values ​​of the constraint data to corresponding difference quantities.

[0182] 4. A device according to clause 3, wherein, when the decoding attribute is bit depth, the corresponding difference quantities mapped to the candidate values ​​of the constraint data all represent negative differences in the bit depth, so that the bit depth modified by the difference defined by the constraint data as defined by the encoding profile provided by the parameter data is a bit depth that is no greater than the bit depth defined by the encoding profile provided by the parameter data.

[0183] 5. A device according to clause 3 or clause 4, wherein, when the decoding attribute is a chroma subsampling format, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent differences, so that the chroma subsampling format modified by the difference defined by the constraint data, as defined by the encoding profile provided by the parameter data, is a chroma subsampling format that is not higher than the chroma subsampling format defined by the encoding profile provided by the parameter data in a predetermined hierarchy of chroma subsampling formats.

[0184] 6. The apparatus of clause 5, wherein, in the predetermined hierarchy of chroma subsampling formats, the predetermined hierarchy of chroma subsampling formats comprises, in order from low to high:

[0185] i.4:0:0 chroma subsampling format;

[0186] ii.4:2:0 chroma subsampling format;

[0187] iii. 4:2:2 chroma subsampling format; and

[0188] iv.4:4:4 chroma subsampling format.

[0189] 7. Apparatus according to any of the preceding clauses, wherein:

[0190] The input video data stream has an associated parameter set; and

[0191] The video data decoder is configured to decode parameter data from the parameter sets and to decode constraint data from the parameter sets.

[0192] 8. A video storage, capture, transmission or reception device comprising a device according to any of the preceding clauses.

[0193] 9. A device comprising:

[0194] a video data encoder circuit configured to encode an input video data stream to generate an output encoded video data stream, the video data encoder being responsive to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format; and

[0195] parameter encoder circuitry configured to encode decoding attributes associated with an output encoded video data stream based on the configuration data;

[0196] wherein the parameter encoder circuit is configured to encode parameter data indicating a profile selected from a plurality of profiles, each profile defining a decoding property, and constraint data defining a difference between the decoding property defined by the configuration data and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference in the decoding property by a zero value of the constraint data.

[0197] 10. Apparatus according to clause 9, wherein the constraint data is configured to indicate a non-zero difference value of the decoding property by a non-zero value of the constraint data.

[0198] 11. Apparatus according to clause 10, wherein the parameter encoder circuit is configured to generate a difference value for the decoding property by mapping a set of candidate values ​​of the constraint data to corresponding difference quantities.

[0199] 12. A device according to clause 11, wherein, when the decoding attribute is bit depth, the corresponding difference quantities mapped to the candidate values ​​of the constraint data all represent negative differences in the bit depth, so that the bit depth modified by the difference defined by the constraint data as defined by the encoding profile provided by the parameter data is a bit depth that is no greater than the bit depth defined by the encoding profile provided by the parameter data.

[0200] 13. A device according to clause 11 or clause 12, wherein, when the decoding attribute is a chroma subsampling format, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent differences, so that the chroma subsampling format modified by the difference defined by the constraint data, as defined by the encoding profile provided by the parameter data, is a chroma subsampling format that is not higher than the chroma subsampling format defined by the encoding profile provided by the parameter data in a predetermined hierarchy of chroma subsampling formats.

[0201] 14. The apparatus of clause 13, wherein, in the predetermined hierarchy of chroma subsampling formats, the predetermined hierarchy of chroma subsampling formats comprises, in order from low to high:

[0202] i.4:0:0 chroma subsampling format;

[0203] ii.4:2:0 chroma subsampling format;

[0204] iii. 4:2:2 chroma subsampling format; and

[0205] iv.4:4:4 chroma subsampling format.

[0206] 15. Apparatus according to any one of clauses 9 to 14, wherein:

[0207] The output coded video data stream has an associated parameter set; and

[0208] The parameter data encoder is configured to encode the parameter data into parameter sets and to encode the constraint data into parameter sets.

[0209] 16. A video storage, capture, transmission or reception device comprising a device according to any one of clauses 9 to 15.

[0210] 17. A method comprising:

[0211] decoding the input video data stream in response to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format;

[0212] detecting constraint data associated with the input video data stream, the constraint data defining a difference between a decoding property applicable to the input video data stream and a decoding property defined by a profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference in the decoding property by a zero value of the constraint data; and

[0213] The decoding step is controlled to generate a decoded video data stream having decoding properties defined by the encoding profile provided by the parameter data, modified by the difference values ​​defined by the constraint data.

[0214] 18. Computer software which, when executed by a computer, causes the computer to perform the method according to clause 17.

[0215] 19. A machine-readable non-transitory storage medium storing computer software according to clause 18.

[0216] 20. A method comprising:

[0217] responsive to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format, encoding an input video data stream to generate an output encoded video data stream; and

[0218] A decoding property is encoded as being associated with an output encoded video data stream according to the configuration data by encoding parameter data and constraint data indicating a profile selected from a plurality of profiles, each profile defining a decoding property, the constraint data defining a difference between a decoding property defined by the configuration data and a decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data.

[0219] 21. An encoded video data stream generated by the method according to clause 20.

[0220] 22. A machine-readable non-transitory storage medium storing a coded video data stream according to clause 21.

[0221] 23. Computer software which, when executed by a computer, causes the computer to perform the method according to clause 20.

[0222] 24. A machine-readable non-transitory storage medium storing computer software according to clause 23.

Claims

1. A device for video data, comprising: a video data decoder circuit configured to decode an input video data stream, the video data decoder being responsive to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format; a detector circuit configured to detect constraint data associated with the input video data stream, the constraint data defining a difference between a decoding property applicable to the input video data stream and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data; and A control circuit is configured to control the video data decoder to decode the input video data stream to generate a decoded video data stream having decoding properties, wherein the decoding properties are defined by the profile provided by the parameter data and modified by the difference value defined by the constraint data.

2. The device according to claim 1, wherein: The constraint data is configured to indicate a non-zero difference value of the decoding property by a non-zero value of the constraint data.

3. The device according to claim 2, wherein: The detector circuit is configured to generate the difference value of the decoding property by mapping a set of candidate values ​​of the constraint data to corresponding difference quantities.

4. The device according to claim 3, wherein: When the decoding attribute is the bit depth, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent negative differences in bit depth, so that the bit depth defined by the profile provided by the parameter data as modified by the difference defined by the constraint data is a bit depth that is not greater than the bit depth defined by the profile provided by the parameter data.

5. The device according to claim 3, wherein: When the decoding attribute is the chroma subsampling format, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent difference values, so that the chroma subsampling format modified by the difference value defined by the constraint data defined by the configuration file provided by the parameter data is a chroma subsampling format that is not higher than the chroma subsampling format defined by the configuration file provided by the parameter data in a predetermined hierarchy of chroma subsampling formats.

6. The device according to claim 5, wherein: In the predetermined levels of the chroma subsampling format, the predetermined levels of the chroma subsampling format include, in order from low to high: i.4:0:0 chroma subsampling format; ii.4:2:0 chroma subsampling format; iii. 4:2:2 chroma subsampling format; and iv.4:4:4 chroma subsampling format.

7. The apparatus of claim 1, wherein: The input video data stream has an associated parameter set; and The video data decoder is configured to decode the parameter data from a parameter set and to decode the constraint data from a parameter set.

8. A video storage, capture, transmission or reception device comprising the device according to claim 1.

9. A device for video data, comprising: a video data encoder circuit configured to encode an input video data stream to generate an output encoded video data stream, the video data encoder being responsive to configuration data defining decoding properties, the decoding properties including one or more of bit depth and chroma subsampling format; and parameter encoder circuitry configured to encode the decoding attributes to be associated with the output encoded video data stream based on the configuration data; wherein the parameter encoder circuit is configured to encode parameter data indicating a profile selected from a plurality of profiles, each profile defining a decoding property, and constraint data defining a difference between the decoding property defined by the configuration data and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data.

10. The device according to claim 9, wherein: The constraint data is configured to indicate a non-zero difference value of the decoding property by a non-zero value of the constraint data.

11. The device according to claim 10, wherein: The parameter encoder circuit is configured to generate the difference value for the decoding property by mapping a set of candidate values ​​for the constraint data to corresponding difference quantities.

12. The device according to claim 11, wherein When the decoding attribute is the bit depth, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent negative differences in bit depth, so that the bit depth defined by the profile provided by the parameter data as modified by the difference defined by the constraint data is a bit depth that is not greater than the bit depth defined by the profile provided by the parameter data.

13. The device according to claim 11, wherein: When the decoding attribute is the chroma subsampling format, the corresponding difference amounts mapped to the candidate values ​​of the constraint data all represent difference values, so that the chroma subsampling format modified by the difference value defined by the constraint data defined by the configuration file provided by the parameter data is a chroma subsampling format that is not higher than the chroma subsampling format defined by the configuration file provided by the parameter data in a predetermined hierarchy of chroma subsampling formats.

14. The device according to claim 13, wherein: In the predetermined levels of the chroma subsampling format, the predetermined levels of the chroma subsampling format include, in order from low to high: i.4:0:0 chroma subsampling format; ii.4:2:0 chroma subsampling format; iii. 4:2:2 chroma subsampling format; and iv.4:4:4 chroma subsampling format.

15. The apparatus of claim 9, wherein: The output coded video data stream has an associated parameter set; and The parameter data encoder is configured to encode the parameter data into a parameter set and to encode the constraint data into a parameter set.

16. A video storage, capture, transmission or reception device comprising the device according to claim 9.

17. A method for video data, comprising: decoding an input video data stream in response to parameter data associated with the input video data stream, the parameter data indicating a profile selected from a plurality of profiles, each profile defining decoding properties, the decoding properties including one or more of a bit depth and a chroma subsampling format; detecting constraint data associated with the input video data stream, the constraint data defining a difference between a decoding property applicable to the input video data stream and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data; as well as A decoding step is controlled to generate a decoded video data stream having decoding properties defined by said profile provided by said parameter data, modified by said difference value defined by said constraint data.

18. A machine-readable non-transitory storage medium storing computer software, which, when executed by a computer, causes the computer to perform the method according to claim 17.

19. A method for video data, comprising: encoding an input video data stream to generate an output encoded video data stream in response to configuration data defining decoding properties including one or more of bit depth and chroma subsampling format; as well as The decoding property is encoded to be associated with the output encoded video data stream according to the configuration data by encoding parameter data and constraint data indicating a profile selected from a plurality of profiles, each profile defining a decoding property, the constraint data defining a difference between the decoding property defined by the configuration data and the decoding property defined by the profile indicated by the parameter data, wherein the constraint data is configured to indicate a zero difference value for the decoding property by a zero value of the constraint data.

20. A machine-readable non-transitory storage medium storing computer software, which, when executed by a computer, causes the computer to perform the method according to claim 19.

Images