Image data encoding device, image data encoding method, computer software, and non-transitory storage medium

Abstract

1. An image data encoding apparatus comprising: an entropy encoder configured to selectively encode data items representing image data to generate encoded binary symbols for successive output data portions; an attribute detector configured to detect encoding attributes applicable to a given output data portion; and a selector configured to select a constraint for use with the given output data portion from two or more candidate constraints in response to the detected encoding attribute, wherein the entropy encoder is configured to generate an output data stream subject to a constraint defining an upper limit on the number of binary symbols represented by the byte size of any individual output data portion relative to the byte size of the output data portion, and wherein the entropy encoder is configured to provide padding data for each output data portion that does not satisfy the constraint, so as to increase the size in bytes of the output data portion to satisfy the constraint.

Description

[Technical field]

[0001] This disclosure relates to encoding and decoding image data. [Background technology]

[0002] The "Background Art" section of this specification is intended to generally describe the background of this application. The inventors' techniques, to the extent described in this Background Art section, are not admitted, expressly or impliedly, as prior art to this application, as are aspects of the description that should not be considered prior art if they were not prior art at the time of filing this application.

[0003] There are several video data encoding and decoding systems that convert video data into a frequency domain representation, quantize the resulting frequency domain coefficients, and then apply some kind of entropy coding to the quantized coefficients, thereby compressing the video data. By applying corresponding decoding or decompression techniques, the original video data is reconstructed and restored.

[0004] High Efficiency Video Coding (HEVC), also known as H.265 or MPEG-H Part 2, has been proposed as a successor to H.264 / MPEG-4 AVC. It aims to improve the video quality of HEVC while doubling the data compression ratio of H.264, and to make H.264 scalable in the range of resolutions from 128x96 to 7680x4320 pixels, which corresponds to a bitrate of roughly 128 Kbit / s to 800 Mbit / s. Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure aims to address or mitigate problems with the above processes. [Means for solving the problem]

[0006] The present disclosure relates to a method for encoding an image data block comprising: an entropy encoder configured to selectively encode data items representing image data to generate encoded binarized symbols of successive output data portions; an attribute detector configured to detect encoding attributes applicable to a given output data portion; a selector configured to select a constraint for use with said given output data portion from two or more candidate constraints in response to the detected encoding attribute; Equipped with the entropy encoder is configured to generate an output data stream, the output data stream being subject to a constraint defining an upper limit on the number of binarized symbols represented by the byte size of any individual output data portion relative to a byte size of the output data portion; To satisfy this constraint, the entropy encoder is configured to provide padding data for each output data portion that does not satisfy the constraint, to increase the size in bytes of that output data portion. An image data encoding device is provided.

[0007] The present disclosure provides a method for generating a digital signal processing circuit comprising the steps of: selectively encoding data items representing image data to generate encoded binary symbols for successive output data portions; generating an output data stream subject to a constraint defining an upper limit on the number of binary symbols represented by any individual output data portion relative to the size in bytes of said output data portions; providing padding data for each output data portion that does not satisfy the constraint so as to increase the size in bytes of that output data portion to satisfy the constraint; Detecting encoding attributes applicable to a given portion of output data; selecting a constraint from two or more candidate constraints for use with a given output data portion in response to the detected encoding attribute; Includes A method for encoding image data is also provided.

[0008] The present disclosure further provides a suitable decoding device for decoding a data signal generated by the above-mentioned method or device.

[0009] Further respective aspects and features of the present disclosure are defined in the appended claims.

[0010] It should be understood that the above general description and the following detailed description are examples of the present technology and are not intended to limit the present technology. [Brief description of the drawings]

[0011] A complete understanding of the present technology and many of its advantages will be readily appreciated by considering the following detailed description in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a schematic diagram showing an audio / video (A / V) data transmission and reception system that compresses and decompresses video data. [Diagram 2] FIG. 1 is a schematic diagram illustrating a video display system with video data decompression. [Diagram 3] FIG. 1 is a schematic diagram illustrating an audio / video storage system that compresses video data and decompresses video data. [Figure 4] FIG. 1 is a schematic diagram showing a video camera that performs compression of video data. [Diagram 5] FIG. 2 is a schematic diagram showing a storage medium. [Figure 6] FIG. 2 is a schematic diagram showing a storage medium. [Figure 7] 1 is a schematic diagram showing a video data compression and decompression device; [Figure 8] FIG. 2 is a schematic diagram showing a prediction unit. [Figure 9] FIG. 2 is a schematic diagram showing a partially encoded image; [Figure 10] FIG. 1 is a schematic diagram illustrating a set of possible intra-prediction directions. [Figure 11]FIG. 2 is a schematic diagram showing a set of prediction modes. [Figure 12] FIG. 13 is a schematic diagram illustrating another set of prediction modes. [Figure 13] FIG. 2 is a schematic diagram illustrating an intra-prediction process. [Figure 14] FIG. 1 is a schematic diagram showing a CABAC encoder. [Figure 15] FIG. 1 is a schematic diagram illustrating the CABAC encoding technique. [Figure 16] FIG. 1 is a schematic diagram illustrating the CABAC encoding technique. [Figure 17] FIG. 1 is a schematic diagram illustrating the CABAC decoding technique. [Figure 18] FIG. 1 is a schematic diagram illustrating the CABAC decoding technique. [Figure 19] FIG. 2 is a schematic diagram showing a segmented image. [Figure 20] FIG. 1 is a schematic diagram showing one apparatus. [Figure 21] FIG. 2 is a schematic diagram showing a controller. [Figure 22] A and B represent the output data section diagrammatically. [Figure 23] 1 is a schematic flow chart illustrating one method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Referring now to the drawings, FIGS. 1 to 4 show schematic diagrams of an apparatus or system that utilizes a compression device and / or a decompression device according to each embodiment of the present technology described below.

[0013] All of the data compression and / or decompression devices described below may be implemented in hardware or as programmable hardware, such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA), or a combination thereof, or in software running on a general-purpose data processing device, such as a general-purpose computer. For embodiments implemented in software and / or firmware, it will be understood that such software and / or firmware, as well as non-transitory data recording media on which such software and / or firmware is stored or provided, are considered embodiments of the present technology.

[0014] FIG. 1 is a schematic diagram showing an audio / video data transmission / reception system that performs video data compression and video data decompression.

[0015] An input audio / video signal 10 is provided to a video data compression device 20 which compresses at least the video element of the audio / video signal 10 and is transmitted along a transmission route 30, e.g. a cable, optical fiber, wireless link, etc. The compressed signal is processed by a decompressor 40 which provides an output audio / video signal 50. On the return path, a compressor 60 compresses the audio / video signal which is transmitted along the transmission route 30 to a decompressor 70.

[0016] Thus, compressor 20 and decompressor 70 may constitute one node of a transmission link, and decompressor 40 and compressor 60 may constitute another node of the transmission link. Of course, if the transmission link is unidirectional, then only one of these nodes will require a compressor and only the other will require a decompressor.

[0017] 2 is a schematic diagram of a video display system that performs video data decompression. Specifically, a compressed audio / video signal 100 is processed by a decompressor 110, which provides a decompressed signal that can be displayed on a display device 120. The decompression device 110 may be formed integrally with the display device 120, for example, by being provided in the same housing as the display device 120. Alternatively, the decompression device 110 may be provided (for example) as a so-called set-top box (STB). Note that the term "set-top" does not imply that the box must be placed in a particular orientation or position relative to the display device 120. The term is simply used in the art to denote a device that can be connected to a display as a peripheral.

[0018] 3 is a schematic diagram of an audio / video storage system that compresses and decompresses video data. An input audio / video signal 130 is provided to a compressor 140 which produces a compressed signal which is stored in a storage device 150, for example a magnetic disk drive, optical disk drive, magnetic tape drive, or a solid state storage device such as semiconductor memory or other storage device. During playback, the compressed data is read from the storage device 150 and sent to a decompressor 160 for decompression, thereby providing an output audio / video signal 170.

[0019] It will be appreciated that compressed or encoded signals and storage media, such as non-transitory machine-readable storage media, that store the signals are considered embodiments of the technology.

[0020] Figure 4 is a schematic diagram of a video camera that compresses video data. In Figure 4, an image capture device 180, such as a Charge Coupled Device (CCD) image sensor and associated control and readout electronics, generates a video signal that is sent to a compression device 190. One or more microphones 200 generate audio signals that are sent to the compression device 190. The compression device 190 generates a compressed audio / video signal 210 (collectively represented as stage 220) for storage and / or transmission.

[0021] The techniques described below are primarily directed to the compression and decompression of video data, and it will be appreciated that many existing techniques may be used to generate compressed audio / video signals in conjunction with the video data compression techniques described below to effect compression of audio data. Therefore, compression of audio data will not be discussed separately, and it will also be appreciated that the data rates associated with video data are typically much higher than the data rates associated with audio data (whether compressed or uncompressed), particularly in broadcast quality video data. It will therefore be appreciated that uncompressed audio data can be added to compressed video data to result in a compressed audio / video signal. Furthermore, although the embodiments of the present invention (see Figures 1-4) relate to audio / video data, it will be appreciated that the techniques described below may also be used in systems that simply process (i.e. compress, decompress, store, display and / or transmit) video data. That is, the embodiments do not necessarily relate to audio data processing, but may also be applied to the compression of video data.

[0022] Figure 4 thus provides an example of a video capture device including an image sensor and an encoding device of the type described below, and Figure 2 thus provides an example of a decoding device of the type described below, and a display device on which the decoded image is output.

[0023] The combination of Figures 2 and 4 can provide a video capture device including an image sensor 180 and an encoding device 190, a decoding device 110, and a display device 120 where the decoded images are output.

[0024] 5 and 6 are schematic diagrams showing storage media for storing (for example) compressed data generated by device 20, 60, compressed data input to device 110, or storage media i.e. storage 150, 220. FIG. 5 is a schematic diagram showing a disk-type storage medium such as a magnetic disk or an optical disk. FIG. 6 is a schematic diagram showing a solid-state storage medium such as a flash memory. Note that FIG. 5 and 6 also show examples of non-transitory device-readable storage media for storing computer software that, when executed by a computer, causes the computer to perform one or more of the methods described below.

[0025] Thus, the above arrangements provide examples of video storage, capture, transmission and reception devices embodying any of the present techniques.

[0026] FIG. 7 is a schematic diagram of a video data compression and decompression device.

[0027] The controller 343 controls the overall operation of the video data compression and decompression device, and in particular the compression mode, by acting as a selector to select various operational aspects such as block size and shape, thereby controlling the trial encoding process. The control unit 343 also controls whether the video data is encoded lossily or not. The control unit 343 may be considered as forming part of an image encoder or image decoder (as the case may be). Successive images of the input video signal 300 are provided to a summing unit 310 and to an image prediction unit 320, which will be described in more detail below with reference to Figure 8. An image encoder or image decoder (as the case may be) may use features from the apparatus shown in Figure 7 together with the intra image predictor of Figure 8. However, the image encoder or image decoder does not necessarily require all the features shown in Figure 7.

[0028] The adder 310 effectively performs a subtraction (negative addition) operation, receiving the input video signal 300 on its "+" input and the output of the image predictor 320 on its "-" input, thereby subtracting the predicted image from the input image, resulting in a so-called residual image signal 330, which represents the difference between the actual image and the predicted image.

[0029] One reason for generating a residual image signal is the following: the described data coding techniques, i.e. techniques applied to the residual image signal, tend to work more efficiently when there is less "energy" in the image being coded, where the term "efficient" refers to the small amount of coded data produced. For a particular image quality level, it is desirable (and considered "efficient") to generate as little data as possible. The "energy" of a residual image refers to the amount of information contained in the residual image: if the predicted image and the actual image were identical, the difference between these two images (i.e., the residual image) would contain zero information (zero energy) and could be very easily coded into a small amount of coding data. In general, if the prediction process can be performed reasonably well, such that the content of the predicted image is similar to the content of the image to be encoded, it is expected that the residual image data will contain less information (less energy) than the input image and can be easily encoded into a smaller amount of encoding data.

[0030] The remainder of the apparatus, which operates as an encoder (encoding the residual or difference image), will now be described. The residual image data 330 is fed to a transform section or circuit 340 which generates a Discrete Cosine Transform (DCT) representation of a block or region of the residual image data. The DCT technique itself is well known and will not be described in detail here. Also, the use of the DCT is merely an example of one possible configuration. Other transformations could be used, such as the Discrete Sine Transform (DST). Transforms may be combined, such as one transformation followed (directly or indirectly) by another. The choice of the transformation scheme may be decided explicitly and / or may depend on side information used to configure the encoder and decoder.

[0031] The output of the transform unit 340, i.e., the set of DCT coefficients for each transform block in the image data, is provided to a quantizer unit 350. A variety of quantization techniques are well known in the field of video data compression, ranging from simple multiplication by a quantization scaling factor to the application of complex look-up tables under the control of a quantization parameter. They generally serve two purposes: first, the quantization process reduces the possible values ​​that the transform data can take. Second, the quantization process increases the likelihood that the transformed data will have a zero value, allowing the entropy coding process (described below) to be more efficient in generating small amounts of compressed video data.

[0032] A data scanning process is applied by a scanner 360. The purpose of the scanning process is to reorder the quantized transformed data in order to group together as many non-zero quantized transformed coefficients as possible, and of course, to group together as many zero-valued coefficients as possible. These functions allow the efficient application of so-called run-length coding or similar techniques. The scanning process thus involves selecting coefficients from the quantized transformed data, and in particular blocks of coefficients that correspond to blocks of transformed and quantized image data, in a "scan order" such that (a) all coefficients are selected once as part of the scan, and (b) the scan can effect any desired reordering. One example of a scan order that produces useful results is the so-called Up-right Diagonal scan order.

[0033] The scanned coefficients are then sent to an entropy encoder (EE) 370. Again, various entropy coding techniques may be performed. Two examples are variations of the so-called Context Adaptive Binary Coding (CABAC) system and variations of the so-called Context Adaptive Variable-Length Coding (CAVLC) system. CABAC is generally considered efficient. Studies have shown that the amount of coded output data in CABAC is 10-20% less than CAVLC for comparable image quality. However, CAVLC is believed to exhibit a much lower level of complexity (in terms of implementation) than CABAC. Note that although the scanning and entropy coding processes are shown as separate processes, in practice they can be combined or treated together, i.e., data can be read into the entropy encoder in scan order, as can the inverse processes described below.

[0034] The output of the entropy encoder 370 provides a compressed output video signal 380, together with additional data (described above and / or below), for example, that defines how the prediction unit 320 generates the predicted image.

[0035] However, since the operation of the predictor 320 itself depends on the decompressed compressed output data, a return path is also provided.

[0036] The reason for this feature is as follows: At an appropriate stage in the decompression process (described below), decompressed residual data is generated. This decompressed residual data needs to be added to the predicted image to generate the output image (because the original residual data was the difference between the input image and the predicted image). To make this process equivalent on the compression and decompression side, the predicted image generated by the predictor 320 should be the same during the compression and decompression processes. Of course, the device does not have access to the original input image at decompression time, it can only access the decompressed image, so at compression time, the predictor 320 bases its predictions (at least for inter-image coding) on ​​the decompressed compressed image.

[0037] The entropy encoding process performed by entropy encoder 370 is considered (at least in some instances) to be "lossless," i.e., the data originally provided to entropy encoder 370 can be replaced with exactly the same data. Thus, in such instances, a return pass can be implemented prior to the entropy encoding stage. Indeed, the scanning process performed by the scanner 360 is also considered to be lossless, but in this embodiment the return path 390 runs from the output of the quantizer 350 to the input of the supplemental inverse quantizer 420. If a stage is lossy or potentially lossy, it may be included in the feedback loop formed by the return path. For example, the entropy encoding stage may, at least in principle, be made lossy, for example by techniques that encode bits in parity information. In such instances, the entropy encoding and decoding must form part of a feedback loop.

[0038] In general, entropy decoder 410, inverse scan unit 400, inverse quantizer unit 420, and inverse transform unit or circuit 430 provide the inverse functions of entropy encoder 370, scanner unit 360, quantizer unit 350, and transform unit 340, respectively. The compression process will continue here; the process for decompressing the input compressed video signal will be described separately below.

[0039] In the compression process, the scanned coefficients are sent from the quantizer 350 via a return path 390 to an inverse quantizer 420 which performs the inverse operation of the scanner 360. The inverse quantization and inverse transform processes are performed by the inverse quantizer 420 and the inverse transformer 430 to generate a compressed-decompressed residual image signal 440.

[0040] The image signal 440 is added to the output of the prediction unit 320 at adder 450 to produce a reconstructed output image 460. This forms one input to the image prediction unit 320, as will be described below.

[0041] The process applied to decompress a received compressed video signal 470 will now be described. The compressed video signal 470 is first supplied to an entropy decoder 410, which in turn supplies it to an inverse scanning unit 400, an inverse quantization unit 420, and an inverse transform unit 430. It is then added to the output of the image prediction unit 320 by an adder 450. Thus, at the decoder side, the decoder reconstructs the residual image and decodes each block by applying it (block by block) to the predicted image (by adder 450). In brief, the output 460 of adder 450 forms the output decompressed video signal 480. In practice, further filtering may optionally be applied before the signal is output (e.g., using filter 560), which is shown in Figure 8. In Figure 7, which shows the overall configuration in comparison with Figure 8, filter 560 has been omitted for clarity.

[0042] The apparatus shown in Figures 7 and 8 can operate as a compression (encoding) apparatus or a decompression (decoding) apparatus. The functions of the two types of apparatus are substantially overlapped. The scanner 360 and the entropy encoder 370 are not used in the decompression mode. The operation of the predictor 320 (described in more detail below) and other sections depends on the mode and parameter information contained in the received compressed bitstream, and they do not generate this information themselves.

[0043] FIG. 8 is a schematic diagram illustrating the generation of a predicted image, and in particular the operation of the image predictor 320.

[0044] Two basic prediction modes are performed by the image prediction unit 320: the so-called intra-image prediction and the so-called inter-image prediction or Motion-Compensated (MC) prediction. On the encoder side, these predictions each involve finding a prediction direction for the current block to be predicted and generating a predictive block of samples depending on other samples (in the same (intra) or another (inter) image). The adder unit 310 or 450 respectively encodes or decodes the block by encoding or decoding the difference between the predicted block and the actual block.

[0045] (At the decoder side or inverse decoding side of the encoder, this detection of the prediction direction may be in response to data associated with the data encoded by the encoder indicating which direction was used by the encoder, or the detection of the prediction direction may be in response to the same factors determined by the encoder.)

[0046] Intra-image prediction is based on the prediction of the content of image blocks or regions on data taken from within the same image. This corresponds to the so-called I-frame coding in other video compression techniques. However, in contrast to I-frame coding, where the whole image is coded by intra-coding, in this embodiment the choice between intra-coding and inter-coding can be made on a block-by-block basis. In other embodiments, the choice is still made on a picture-by-picture basis.

[0047] Motion compensated prediction is an example of inter-image prediction, where motion information is used that seeks to define the source, in other adjacent or nearby images, of the image details to be coded in the current image. Thus, in an ideal case, the content of a block of image data in a predicted image can be very easily coded as a reference (a motion vector) that points to a corresponding block in the same or slightly different position in an adjacent image.

[0048] The technique known as "block copy" prediction is in some ways a hybrid of the two above predictions, as it uses a vector that indicates a block of samples, displaced from the current predicted block within the same image, that should be copied to generate the current predicted block.

[0049] Returning to Figure 8, there are shown two image prediction configurations (corresponding to intra image prediction and inter image prediction) whose prediction results are selected by multiplication unit 500 under control of mode signal 510 (e.g. in control unit 343) to provide a block of predicted images for supply to adders 310 and 450. The selection is made based on which selection results in the least "energy" (which, as discussed above, may be thought of as the amount of information that needs to be coded), and is signalled to the decoder in the coded output data stream. In this regard, the image energy can be found, for example, by trial subtracting regions of the two versions of the predicted image from the input image, squaring each pixel value of the difference image, summing the multiplication values, and identifying which of the two versions has a lower average multiplication value of the difference image associated with that image region. In another example, a trial encoding can be performed for each selection or possible selection, and the selection is made depending on the cost of each possible selection in terms of either or both of the number of bits required for encoding and the distortion to the image.

[0050] In an intra prediction system, the actual prediction is made based on the image blocks received as part of the signal 460, i.e. the prediction is made based on the encoded-decoded image blocks so that the exact same prediction can be made in the decompressor. However, data can also be derived from the input video signal 300 and the operation of the intra image predictor 530 can be controlled by the intra mode selector 520.

[0051] In inter-image prediction, a motion compensated (MC) prediction unit 540 uses motion information, e.g., motion vectors, derived from the input video signal 300 by a motion estimation unit 550. The motion compensated prediction unit 540 applies these motion vectors to the reconstructed image 460 to generate blocks of inter-image prediction.

[0052] Thus, the intra-image prediction unit 530 and the motion compensation prediction unit 540 (operating together with the estimation unit 550) respectively operate as a detection unit that detects the prediction direction for the current block to be predicted, and as a generation unit that generates a prediction block of samples (which forms part of the prediction result sent to the addition units 310 and 450) depending on other samples defined by the prediction direction.

[0053] We now turn to the processing applied to the signal 460. First, the signal 460 is optionally filtered by a filter unit 560, which will be described in more detail below. In this processing, a "deblocking" filter is applied to eliminate, or at least reduce, the effect on the block-based processing and subsequent operations performed by the transform unit 340. A Sample Adaptive Offsetting (SAO) filter may also be used. An adaptive loop filter is optionally applied using coefficients obtained by processing the reconstructed signal 460 and the input video signal 300. The adaptive loop filter is a type of filter that applies adaptive filter coefficients to the data to be filtered using known techniques. That is, the filter coefficients may vary based on various factors. Data defining which filter coefficients to use is inserted as part of the encoded output data stream.

[0054] When the device is operating as a decompressor, the filtered output from filter 560 actually forms the output video signal 480. This signal is stored in one or more image or frame stores 570. The storage of successive images is required for the motion compensated prediction process, in particular for the generation of motion vectors. To conserve the necessary memory, the images stored in image store 570 may be kept in compressed form and then decompressed for use in the generation of motion vectors. Any known compression / decompression system may be used for this particular purpose. The stored image is sent to an interpolation filter 580 which generates a higher resolution stored image. In this example, intermediate samples (sub-samples) are generated so that the resolution of the interpolated image output by the interpolation filter 580 is four times (in each dimension) the image stored in the image store 570 if the luminance channel is 4:2:0, and eight times (in each dimension) the image stored in the image store 570 if the chrominance channels are 4:2:0. The interpolated image is sent as input to a motion estimation unit 550 and a motion compensated prediction unit 540.

[0055] We now describe how to partition an image for compression purposes. At a basic level, an image to be compressed can be thought of as an array of blocks or regions of samples. Such partitioning of an image into blocks or regions can be accomplished using decision trees such as those described in Bross et al., High Efficiency Video Coding (HEVC) text specification draft 6, JCTVC-H1003_d0 (November 2011), the contents of which are incorporated herein by reference. In some examples, the resulting blocks or regions have different sizes and, in some cases, shapes that generally follow the arrangement of image features in the image due to the decision tree. This alone can improve coding efficiency, since samples that represent or follow similar image features tend to be grouped together due to such an arrangement. In some examples, square blocks or regions of different sizes (e.g., from 4x4 samples to, for example, 64x64, or larger blocks) are available for selection. Other configurations may use blocks or regions of different shapes, such as rectangular blocks (e.g., vertically or horizontally oriented). Other non-square and non-rectangular blocks are also encompassed. As a result of such a division of the image into blocks or regions, (at least in this example) each sample of the image is assigned to one, and even only one, such block or region sample.

[0056] We now turn to the intra prediction process. In general, intra prediction involves generating a prediction of a current block of samples from previously coded and decoded samples in the same image.

[0057] 9 is a schematic diagram of a partially coded image 800, where the image is coded block by block from the top left to the bottom right. An example of a block that is partway through coding in the processing of the entire image is shown as block 810. The upper shaded region 820 to the left of block 810 has already been coded. Any of the shaded regions 820 can be used for intra-image prediction of the content of block 810, but not the unshaded regions below.

[0058] In some examples, an image is coded block-wise such that larger blocks (called Coding Units (CUs)) are coded in an order such as that described with reference to Figure 9. For each CU, it may be processed as a set of two or more smaller blocks or Transform Units (TUs) (depending on the block partitioning process performed). This provides a hierarchical coding order in which the image is coded CU-wise. Each CU is potentially coded on a TU-by-TU basis. Note that for each TU (the largest node in the tree structure of block division) in a current Coding Tree Unit (CTU), the above-mentioned hierarchical coding order (CU-by-CU, then TU-by-TU) means that there may be previously coded samples in the current CU that are available for coding the TU. These samples may be, for example, at the top right or bottom left of the TU.

[0059] Block 810 represents a CU. As mentioned above, for intra-image prediction processing, this may be subdivided into smaller units of a set. An example of a current TU 830 is shown in CU 810. More generally, an image is divided into regions or groups of samples so that the signaling information and the transformed data can be efficiently coded. Signaling information may require a different tree structure consisting of a subdivision for the structure of the transformation, and indeed for the structure of the prediction information or the prediction itself. For this reason, a CU may have different tree structures for the structures of transform blocks or regions, prediction blocks or regions, and prediction information. In some examples, such as HEVC, the structure may be a so-called quadtree of CUs, where leaf nodes contain one or more prediction units and one or more TUs. A TU may contain multiple transform blocks corresponding to luma and chroma representations of an image, and prediction schemes may be considered as applicable at the transform block level. In some instances, the parameters that are applied to a particular set of samples can be considered to be defined primarily at the block level, which may not be at the same granularity as the transform structure.

[0060] Intra-image prediction considers samples that have been coded before considering the current TU, which may be samples above and / or to the left of the current TU. The samples from which the required sample is predicted may be in different positions or orientations relative to the current TU. To determine which orientation is suitable for the current prediction unit, the mode selection unit 520 of the exemplary encoder may try all combinations of available TU structures for each candidate orientation and select the prediction direction and TU structure that provides the best compression efficiency.

[0061] Images may be coded per "slice". In one example, a slice is a group of horizontally adjacent CUs. More generally, however, the entire residual image may constitute a slice, or a slice may be a single CU or a row of CUs, etc. Slices are coded as independent units, providing some degree of resilience to errors. Encoder and decoder states are fully reset at slice boundaries. For example, intra prediction is not performed across slice boundaries. Thus, slice boundaries are treated as image boundaries.

[0062] Fig. 10 is a schematic diagram showing a possible (candidate) set of prediction directions. All possible directions are available for a prediction unit. The direction is determined by the horizontal and vertical translation with respect to the current block position, and is coded as a prediction "mode". The set of directions is shown in Fig. 11. Note that the so-called DC mode represents a simple arithmetic mean of the surrounding top and left samples. Also, the set of directions shown in Fig. 10 is only an example. Another example is a set of (for example) 65 angular modes plus DC and planar (total of 67 modes), as shown diagrammatically in Fig. 12. Other numbers of modes are also possible.

[0063] In general, the system is operable to generate a predictive block of samples after detecting the prediction direction, depending on other samples determined by the prediction direction. In some examples, the image encoder is configured to encode data identifying the selected prediction direction for each sample or region of the image (and the image decoder is configured to detect such data).

[0064] 13 is a schematic diagram illustrating an intra-prediction process in which a sample 900 of a block or region of samples 910 is derived from other reference samples 920 of the same image according to a direction 930 determined by the intra-prediction mode associated with said sample. In this example, the reference samples 920 originate from blocks above and to the left of the block under consideration 910, and a prediction of the sample 900 is obtained by tracking along the direction 930 relative to the reference samples 920. The direction 930 may indicate a single individual reference sample, but more typically an interpolated value of surrounding reference samples is used as the prediction value. Note that the block 910 may be a square as shown in FIG. 13, or may be of other shapes such as a rectangle.

[0065] FIG. 14 is a schematic diagram illustrating the operation of the CABAC entropy encoder.

[0066] A CABAC encoder works on binary data, i.e. data that is represented only by two symbols: 0 and 1. The encoder performs a so-called context modeling process, which selects a "context", i.e. a probability model, for the next data based on the data already encoded. The selection of the context is performed in a deterministic manner, based on already decoded data, such that the same decision can be made at the decoder without requiring additional data (that identifies the context) to be added to the encoded data stream passed to the decoder.

[0067] Referring to Figure 14, the input data to be encoded may be passed to a binary converter 1400 unless it is already in binary form, in which case converter 1400 is bypassed (by switch 1410 in the figure). In this embodiment, the conversion to binary form is actually accomplished by representing the quantized DCT coefficient data as a series of binary "maps", which are described below.

[0068] The binary data may then be processed by one of two processing paths (shown diagrammatically as separate paths, but which in some embodiments of the invention described below may actually be performed in the same processing stages with only slightly different parameters) a "normal" path and a "bypass" path. The bypass path employs a so-called bypass coder 1420 which does not necessarily utilise the same form of context modelling as the normal path. In some instances of CABAC encoding, such a bypass path can be selected when a sequence of data needs to be processed particularly quickly. However, in this embodiment, we note two characteristics of the so-called "bypass" data. The first characteristic is that the bypass data is processed by the CABAC encoder (950, 1460) only using a fixed context model that represents a 50% probability. The second feature is that bypass data relates to certain categories of data. A particular example of such data is coefficient code data. If the bypass path is not selected, the normal path is selected by the illustrated switches 1430, 1440 operating under the control of control circuitry 1435. This includes data that is processed by context modeler 1450 and subsequently by coding engine 1460.

[0069] The entropy encoder shown in Figure 14 encodes blocks of data (i.e., data corresponding to a block of coefficients for a block of a residual image, for example) as a single value if the entire block is composed of zero-valued data. For each block that does not fall into this category, i.e., a block that contains at least some non-zero data, a "significance map" is created. The significance map indicates, for each position in the block of data to be encoded, whether the corresponding coefficient in the block is non-zero or not. The importance map data itself, which is in binary form, is CABAC coded. Using the importance map aids in compression since no data needs to be coded for coefficients whose magnitudes are indicated to be zero by the importance map. The importance map may also contain a special code to indicate the last non-zero coefficient in a block. This allows all of the last high frequency / trailing zero coefficients to be omitted from coding. In the coded bitstream, the importance map is followed by data defining the values ​​of the non-zero coefficients as dictated by the importance map.

[0070] Other levels of map data are also created and CABAC coded. One example is a map that defines as a binary value (1=yes, 0=no) whether the coefficient data at the map locations indicated by the significance map to be "non-zero" actually has a value of "1". Another map specifies whether the coefficient data at the map locations indicated by the significance map to be "non-zero" actually has a value of "2". Yet another map indicates whether the coefficient data at those map locations indicated by the significance map to be "non-zero" actually has a value of "3 or greater". Yet another map indicates the sign of the data value (using a predefined binary representation such as 1 for +, 0 for -, or vice versa) for the data identified as "non-zero".

[0071] In an embodiment of the present invention, the significance map and other maps are assigned in a predetermined manner to either the CABAC encoder or the bypass encoder, and all represent different respective attributes or value ranges of the same initial data item. In one example, at least the significance map is CABAC encoded and at least a portion of the remaining map (e.g., the signature data) is bypass encoded. Thus, each data item is divided into respective subsets of data, and each subset is encoded by a first (e.g., CABAC) and a second (e.g., bypass) encoding system. The nature of the data, CABAC and bypass coding is such that for a given amount of CABAC encoded data, a variable amount of zero or more bypass data is generated for the same initial data item. Thus, for example, if the quantized, reordered DCT data contains substantially all zero values, then no bypass data or a very small amount of bypass data may be generated. This is because the bypass data only pertains to those map locations for which the significance map indicates a non-zero value. In another example, in quantized reordered DCT data with many high value coefficients, a significant amount of bypass data may be generated.

[0072] In an embodiment of the present invention, the significance map and other maps are generated from the quantized DCT coefficients, for example by scanning unit 360, and subjected to a zigzag scanning process (or a scanning process selected from zigzag, horizontal raster and vertical raster scanning) before being subjected to CABAC encoding.

[0073] In general, CABAC coding involves predicting the context, or probability model, of the next bit to be coded based on other previously coded data. If the next bit is the same as the bit identified as "most likely" by the probability model, then coding the information "the next bit matches the probability model" can be coded with great efficiency. The derivation of context data is important for the good operation of the encoder, since it is inefficient to code "the next bit does not match the probability model." The term "adaptive" means that the context or probability model is adapted or changed during encoding in an attempt to provide a good match with the next data (which has not yet been coded).

[0074] Using a simple analogy, in written English, the letter "U" is relatively rare. However, it is very common in the letter position immediately following the letter "Q." Therefore, a probabilistic model can set the probability of "U" to a very low value, but if the current letter is "Q," the probabilistic model for "U" as the next letter can be set to a very high probability value.

[0075] CABAC coding is used in the current implementation for at least the significance map and the map indicating whether a non-zero value is a 1 or a 2. Bypassing - In these embodiments, identical to CABAC encoding, but for the fact that the probability model is fixed with an equal (0.5:0.5) probability distribution of 1s and 0s, a separate so-called escape data encoding is used at least for the sign data and a map indicating whether the value is >2. For those data positions identified as >2, a separate so-called escape data encoding can be used to encode the actual value of the data. This may include Golomb-Rice encoding techniques.

[0076] The CABAC context modeling and encoding process is described in further detail in WD4: High-Efficiency Video Coding, Working Draft 4, JCTVC-F803_d5, Draft ISO I / EC 23008-HEVC; 201x(E) 2011-10-28.

[0077] Now, referring to Figures 15 and 16, an entropy encoder forming part of a video encoding apparatus includes a first encoding system (e.g., an arithmetic encoding system such as the CABAC encoder 1500) and a second encoding system (such as the bypass encoder 1510) arranged such that a particular data word or value is encoded into the final output data stream by either the CABAC encoder or the bypass encoder, but not by both.

[0078] In an embodiment of the invention, the data values ​​passed to the CABAC encoder and the bypass encoder are respective subsets of ordered data values ​​split or derived from the initial input data (in this example the reordered quantized DCT data) and represent different ones of a set of "maps" generated from the input data.

[0079] The schematic diagram treats the CABAC encoder and the bypass encoder as separate arrangements, which is often the case in practice, but another possibility, shown diagrammatically in Figure 16, is that a single CABAC encoder 1620 is used as both the CABAC encoder 1500 and the bypass encoder 1510 of Figure 15. The encoder 1620 operates under the control of the encoder 1630 such that when in CABAC encoder 1500 mode it operates with an adaptive context model (as described above), and when in bypass encoder 1510 mode it operates with a fixed 50% probability context model.

[0080] A third possibility combines these two in that two substantially identical CABAC encoders can be run in parallel (similar to the parallel arrangement of Figure 15), the difference being that the CABAC encoder operating as the bypass encoder 1510 has its context model fixed at a 50% probability context model.

[0081] The outputs of the CABAC encoding process and the bypass encoding process may be stored (at least temporarily) in respective buffers 1540, 1550. In Figure 16, a switch or demultiplexer 1660 functions under the control of a mode signal 1630 to route CABAC encoded data to buffer 1550 and bypass encoded data to buffer 1540.

[0082] 17 and 18 illustrate generally an example of an entropy decoder forming part of a video decoding apparatus. 17, respective buffers 1710, 1700 pass data to a CABAC decoder 1730 and a bypass decoder 1720, arranged so that a particular encoded data word or value is decoded by either the CABAC decoder or the bypass decoder, but not both. The decoded data is reordered by logic 1740 into the proper order for the subsequent decoding stages.

[0083] The schematic diagram of Figure 17 treats the CABAC decoder and the bypass decoder as separate arrangements, which is often the case in practice, but in another possibility, shown diagrammatically in Figure 18, a single CABAC decoder 1850 is used as both the CABAC decoder 1730 and the bypass decoder 1720 of Figure 17. The decoder 1850 operates under the control of the decoder 1860 such that when in CABAC decoder 1730 mode it operates with an adaptive context model (as described above), and when in bypass encoder 1720 mode it operates with a fixed 50% probability context model.

[0084] As mentioned above, a third possibility is that two substantially identical CABAC decoders can be operated in parallel (similar to the parallel arrangement of FIG. 17), with the difference being that the CABAC decoder operating as the bypass decoder 1720 has a fixed context model at a 50% probability context model.

[0085] In FIG. 18, a switch or multiplexer 1870 acts under the control of a mode signal 1860 to route the CABAC encoded data from buffer 1700 or buffer 1710, as appropriate, to decoder 1850.

[0086] FIG. 19 illustrates a picture 1900 in a schematic manner and is used to illustrate various picture partitioning schemes relevant to the following discussion.

[0087] One example of a picture division is in slices or "regular slices". Each regular slice is encapsulated in its own Network Abstraction Layer (NAL) unit. Prediction within a picture (e.g., intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across slice boundaries are not allowed. That is, regular slices can be reconstructed independently from other regular slices in the same picture. So-called tiles define horizontal and vertical boundaries for dividing a picture into rows and columns of tiles. In a manner that corresponds to regular slices, no in-picture prediction dependencies are allowed across tile boundaries, and no entropy decoding dependencies. However, tiles are not constrained to be contained in individual NAL units.

[0088] Generally speaking, there can be multiple tiles within a slice, multiple slices within a tile, and one or more slices within a picture.

[0089] The schematic example of Figure 19 shows four slices 1910, 1920, 1930, 1940, where slice 1940 includes two tiles 1950, 1960. However, as mentioned, this is merely an arbitrary schematic example.

[0090] In some example configurations, there is a threshold on the number of bins (either EP or CABAC) that may be coded into a slice or picture according to the following formula:

[0091] BinCountsinNalUnits <= (4 / 3) * NumBitsInVclNalUnits + (RawMinCuBits*PicSizelnMinCbsY) / 32 (Formula 1)

[0092] The right hand side of the formula depends on the sum of two parts: a constant value for a particular image region (RawMinCuBits*PicSizelnMinCbsY), a dynamic value that is related to the size of the slice or picture, and the number of bits coded in the output stream for the slice or picture (NumBitsInVclNalUnits). Note that the value 4 / 3 represents the number of bins per bit.

[0093] RawMinCuBits is the number of bits in a minimum size (usually 4*4) raw CU, and PicSizelnMinCbsY is the number of minimum size CUs in a slice or picture.

[0094] When this threshold is exceeded, CABAC zero words (three bytes with values ​​00 00 03) are appended to the stream until the threshold is reached. Each such zero word increases the dynamic value by three.

[0095] This constraint may be expressed as follows: N <= K1 * B + (K2 * CU) Here, N = number of binary symbols in the output data section, K1 is a constant, B = number of encoded bytes in the output data section, K2 is a variable that depends on the characteristics of the minimum size coding unit employed by the image data coding device, CU = size of a picture, slice, or tile represented in the output data portion expressed in number of minimum size coding units.

[0096] In the previously proposed example, this threshold check is performed at the picture and slice level.

[0097] Alternatively, the following formula for the same threshold may be used, where the difference is in the reference to bytes rather than bits: BinCountsinNalUnits <= (32 / 3) * NumByteslnVclNalUnits + (RawMinCuBits * PicSizelnMinCbsY) / 32 (Formula 2)

[0098] The threshold in Equation 1 (or equivalently expressed in Equation 2) may be applied uniformly, ie, regardless of other encoding or decoding parameters, attributes, etc.

[0099] One example of a technical justification for using such a threshold is that the threshold itself indirectly defines the maximum level of processing performance required by a real-time decoder. Real-time decoding relies on decoding each frame in time to achieve a particular output frame rate (e.g., frames per second). The constraint on the processing performance of a real-time decoder is driven by the speed at which two values ​​(binary) need to be decoded, not by the amount of resulting decoded data. A technique used to control this situation is to limit (e.g., as an upper limit or threshold or constraint) the number of binaries per slice or picture, to allow relatively simple rate control on the encoder side. The maximum processing performance required by a CABAC decoder can then be considered as a product or function of this constraint and the rate at which slices or pictures must be decoded.

[0100] It is therefore important to set the threshold (or upper limit or constraint) appropriately. A threshold that is too high can lead to implementation or correct operation challenges for real-time decoders. Similarly, use of an incorrect or inappropriate threshold can lead to excessive use of padding data (e.g., dummy or other data whose sole purpose is to occupy parts of the encoded data stream in order to increase the size of the encoded output data portion), reducing the efficiency of the encoding.

[0101] As noted with reference to Figure 19, a picture or slice may be divided into multiple tiles. One example of why this is done is to allow the use of multiple simultaneous (parallel) decoders.

[0102] Under the arrangement proposed above, each tile does not necessarily meet the threshold calculation discussed above: for example, if a tile is used or decoded alone as a picture, or if different tiles (e.g., with different quantization parameters or different sources) are composited together, there may be no guarantee that the composite slice or picture will conform to the above specifications.

[0103] To address this issue, in an exemplary embodiment, the CABAC threshold is applied at the end of each tile, rather than at the end of each slice or image alone. Thus, the application of the threshold occurs at the end of the encoding of either a tile, slice, or picture. With this said, for pictures divided into slices or tiles, it is not necessary to apply the threshold again at the end of the encoding of the picture, since it can be assumed that if each tile in an image complies with the threshold, then the entire picture must also be compliant.

[0104] The terms "tile" and "slice" refer to independently decodable units and refer to the names used as of the priority date of this application. In the event of subsequent or other variations of the names, the arrangements are applicable to other such independently decodable units. Thus, in an exemplary configuration, the output data portion may be an independently decodable data unit, e.g., the image portion (represented by the output data portion) may be one of an image, a slice, or a tile.

[0105] For the purposes of applying the above formula, the dynamic value represents the number of bytes coded in the output stream of the tile, while the fixed value depends on the number of minimum size coding units (CUs) in the tile.

[0106] FIG. 20 shows a schematic of an apparatus configured to perform this test. Referring to Figure 20, at input 2000, a CABAC / EP encoded stream is received from an encoder. A padding data detector 2010 detects whether the above mentioned threshold calculation is compliant at a given stage with reference to the completion of a slice or tile, such as the end of encoding a slice or tile. The controller 2020, in response to detection by the detector 2010, controls a padding data generator 2030 to generate padding data 2040, such as the CABAC zero words described above, which is added to the stream by a combiner 2050 to form an output stream 2060. The generation of the zero word may also include sending a signal back to the padding data detector 2010 to continue monitoring compliance with the threshold as the zero word is added, and to have the controller 2020 stop generating the zero word once compliance with the threshold has been achieved.

[0107] In another example, the controller 2020 may also function as a predictor configured to generate a prediction of whether a constraint will be satisfied by an output data portion during generation of the output data portion in connection with the embodiments described herein.

[0108] The predetermined stage may be, for example, every n (where n is an integer greater than or equal to 1) binary values ​​encoded, but in an embodiment the predetermined stage is the end of the encoding of the current output data portion.

[0109] Thus, Figure 20 discloses an example of a padding data detector 2010 configured to detect whether a constraint is satisfied by the current output data portion at a given stage in the encoding of the current output data portion, and a padding data generator 2030 configured to generate and insert sufficient padding data into the current output data portion such that the output data portion including the padding data to be inserted satisfies the constraint.

[0110] Referring to FIG. 21, in some examples, the controller 2020 includes an attribute detector 2070 configured to detect encoding attributes applicable to a given output data portion, and a selector 2080 configured to select a constraint from two or more candidate constraints 2082 for use with the given output data portion in response to the detected encoding attributes.

[0111] The controller 2020 may also include a comparator 2090 for comparing a threshold derived from the currently selected constraint with the detection by the padding data detector 2010 to derive a control signal for controlling the operation of the padding data generator 2030.

[0112] The attribute detected by detector 2070 may be, for example, a coding attribute (e.g., a coding mode or profile, e.g., enablement of dependent quantization, to be discussed below), which is an operating mode in which the selection of a quantization parameter to use for quantizing a current data value depends at least in part on characteristics of previously encoded data values ​​and is subsequently included in the encoded data stream or represented by flag data (shown generally as 2072) associated with the encoded data stream and subsequently detectable by a decoder. For example, the flag data may be included in header data, such as header data of the output data portion (e.g., slice header data). The detector 2070 need not itself generate or insert the flag data. This aspect of the process is shown in FIG. 21 for schematic purposes only for the benefit of the present description. An attribute may be considered to be applicable to a given output data portion even if (as in some examples) the attribute is also applicable to other (eg, preceding or following) output data portions.

[0113] Thus, the apparatus of Figures 7 and 14 operating according to the techniques described with respect to Figures 19 and 20 (and those described below) provides an example of an apparatus for encoding image data. The system includes a first data encoder 1450, 1460 and a second data encoder 1420 configured to generate output data bits representing binary symbols from successive symbols representing image data. The first data encoder is configured to generate output data bits representing the encoding symbols with a variable ratio of data bits to a number of encoding symbols. The second data encoder is configured to generate a fixed number of output data bits representing each encoding symbol. The entropy encoder is configured (e.g., using controller 2020 and / or controller 1435) to generate an output data stream, the output data stream being subject to a constraint defining an upper bound on the number of binarized symbols represented by the byte size of any individual output data portion relative to the byte size of that output data portion, for which the entropy encoder is configured to provide padding data for each output data portion that does not satisfy the constraint, to increase the size in bytes of that output data portion to satisfy the constraint. The device is an attribute detector 2070 configured to detect encoding attributes applicable to a given output data portion; and a selector 2080 configured to select a constraint from two or more candidate constraints for use with a given output data portion in response to the detected encoding attribute.

[0114] Using the technique shown in Figure 14, the first data encoder / decoder may be a context-adaptive binary arithmetic coding (CABAC) encoder / decoder. The second data encoder / decoder may be a bypass encoder / decoder. The second data encoder / decoder may be a binary arithmetic coder / decoder using a fixed 50% probability context model.

[0115] Examples of applying thresholds or constraints are as follows:

[0116] Typically the test is: "Does the amount of output data produced satisfy a threshold test?" This can be performed, for example, at the end of encoding the output data portion.

[0117] The apparatus of FIG. 7 operating according to the techniques discussed herein provides one example of an image data encoding device. The image data encoding device comprises an entropy encoder configured to selectively encode data items representing image data to generate encoded binary symbols of successive output data portions. the entropy encoder is configured to generate an output data stream, the output data stream being subject to a constraint defining an upper bound on the number of binarized symbols that can be represented by the byte size of any individual output data portion relative to a byte size of the output data portion; To satisfy this constraint, the entropy encoder is configured to provide padding data for each output data portion that does not satisfy this constraint, in order to increase the size in bytes of that output data portion. The image data encoding device includes: an attribute detector configured to detect encoding attributes applicable to a given output data portion; and a selector configured to select a constraint for use with a given output data portion from two or more candidate constraints depending on the detected encoding attribute.

[0118] The present disclosure also provides a decoding device suitable for decoding a data signal generated by the methods or devices described herein.

[0119] Further examples of constraints are now described.

[0120] In the above examples, a single threshold derivation or formula is used throughout. In alternative examples described below, a choice between two or more candidate thresholds or constraints is implemented. The choice or selection may be responsive to one or more coding attributes (e.g., a parameter, attribute or mode, e.g., a parameter or mode signaled from the encoder side to the decoder side by a flag or the like, or a parameter, attribute or mode derivable in a corresponding or matching manner at the encoder and decoder sides).

[0121] An example of a further candidate equation for a threshold or constraint is as follows: BinCountsinNalUnits <= 10 * NumByteslnVclNalUnits + (RawMinCuBits * PicSizelnMinCbsY) / 16 (Formula 3)

[0122] Thus, although this Equation 3 can be used uniformly as described above, in embodiments, a selection is made between two or more candidate expressions, for example Equation 1 / 2 and Equation 3. In some examples, the selection may be made depending on whether a predefined coding attribute signaled in or along with the encoded data stream is in a first or second state at the encoder and decoder sides, with the respective state of the predefined attribute corresponding to the selection (e.g., Equation 1 / 2 or Equation 3).

[0123] An example of such an attribute is the so-called "dep_quant_enabled_flag", which indicates in the slice header whether a technique called dependent quantization is enabled for the slice to which the slice header applies.

[0124] Instead of using dep_quant_enabled_flag, it actually depends on the availability of the tool, not whether this (dep_quant) tool is enabled. So for example in case of a profile, the profile itself may define "dep_quant_enabled_flag should be off (not enabled)". Also, there may be no such constraint and the dep_quant tool may be on (enabled) or off (disabled). Therefore, this choice can be made by profile constraints, not by whether the dep_quant tool is currently enabled or disabled.

[0125] Dependent quantization is defined in "Versatile Video Coding (Draft 5), Bross et al, JVET-N1001-v10, July 2019 (incorporated herein by reference)." See, e.g., Section 8.7.3, which relates to techniques in which the decoding process selects between multiple possible quantization parameters or a set of quantization parameters, e.g., in response to properties (e.g., parity properties) of previously encoded and decoded sample values. Thus, when dep_quant_enabled_flag = 1 (enabled), such ongoing dependent quantization selection occurs. When dep_quant_enabled_flag = 0 (disabled), such ongoing dependent quantization selection does not occur. As mentioned above, the flag dep_quant_enabled_flag (as an example of a coding attribute) is provided, for example, in the slice header, so that enabling or disabling of dependent quantization applies to the entire slice.

[0126] When dependent quantization is used, or at least when it is possible, different constraints may be relevant due to the empirical observation that the expected relationship between the encoded and decoded binary values ​​can be modified when dependent quantization is applied. A different constraint, such as equation (3), may be more appropriate for use with dependent quantization.

[0127] When multiple candidate constraints are applicable, the decoder is expected to be subject to design constraints to provide sufficient processing power, speed or capacity to deal with the encoded data produced under the harder (or most difficult) of the different available constraints.

[0128] More generally, however, any such attribute (e.g., flag or parameter) may be used, whether or not explicitly signaled in or along with the data stream. For example, different candidate formulas may be selected for each different instance of a so-called "profile," where a profile in this context defines a set or basket of parameters such as bit depth, chrominance sampling (e.g., 4:0:0 (monochrome), 4:2:0, 4:2:2, 4:4:4, etc.), coding type restrictions (intra-picture coding only, etc.), etc.

[0129] In an exemplary embodiment, attributes may be conveniently used to define some aspect of an output data portion, since an associated threshold derivation may be applied to that output data portion. Examples of output data portions in this context may include output data portions that represent respective image portions, such as slices, tiles, or pictures. Other examples of suitable attributes include, for slices, that the slice type is either intra only, or unrestricted (can include intra). As an example of tiling, consider the case of composition of an image from multiple sources (one tile per source). Each tile can have its own threshold derivation depending on the original encoding method. Alternatively, the attributes can be determined using the value of the attribute of the previous tile / output data part at the same position in the picture.

[0130] In some examples, as described above, the constraint or threshold is defined by an equation such as: N <= K1 * B + (K2 * CU) Where: N = number of binary symbols in the output data section, K1 is a constant, B = number of encoded bytes in the output data portion, K2 is a variable that depends on the characteristics of the minimum size coding unit employed by the image data coding device. CU = the size of the picture, slice, or tile represented by the output data portion expressed in the number of minimum sized coding units.

[0131] Note that this is actually a generalization of the above Equations 1, 2, and 3. An example list of constraint candidates that reference K1 and K2 is as follows: [Table 1]

[0132] For the example of Equation 5, the variable vcIByteScaleFactor can be expressed as: vcIByteScaleFactor = (32 + 4 * general_tier_flag) / 3 Here, general_tier_flag is an encoding tier indicator, which (at least in some instances) varies as a flag value of 0 or 1, where 0 indicates the so-called main tier and 1 indicates the so-called high tier. For a particular coding level (representing a maximum dimension of the image to be coded), the higher layers generally correspond to a higher bitrate representation than the main layer. Thus, in this example, the image data coding device is configured to operate on a coding layer selected from the at least two candidate coding layers and to generate layer parameters (e.g., coded within or at least associated with the coded image data or bitstream) defining a currently selected coding layer, where at least the constant K1 depends on the layer parameters. For example, a higher tier parameter may indicate a higher quality encoded output for a particular image size, and the parameter K1 may increase depending on the tier parameter.

[0133] Thus, the process, circuitry, code or logic used to generate the thresholds is conveniently the same or substantially the same in each case, with the parameters K1 and K2 simply being varied for each of the example equations (1 / 2)-(7). However, it will be appreciated that one or more different equations or expressions (or potentially different fixed thresholds) can be used, such as between the different candidate thresholds or constraints discussed above.

[0134] Thus, the candidate constraints 2082 may be stored or represented as (K1, K2) pairs to be selected by the selector 2080. The (K1, K2) pairs may be predetermined and stored in the encoder and decoder, or the pairs (or an indication of a larger set of subsets of the predefined pairs) may be transmitted from the encoder to the decoder as (for example) part of a profile or parameter setting data. The expression N <= K1 * B + (K2 * CU) can then be tested in comparator 2090 using the commonly selected (K1, K2).

[0135] 22A and 22B provide two schematic examples of output data portions, extending across a page, including binarized data 2200 and pad data 2210 provided and inserted by the techniques described above. The amount of padding data can be determined by the particular image data being encoded (in terms of how efficiently it can be encoded) and the constraints in use.

[0136] FIG. 23 is a schematic flow chart illustrating an image data encoding method, the method comprising: selectively encoding (at step 2300) data items representing image data to generate encoded binary symbols for successive output data portions; generating (at step 2310) an output data stream subject to a constraint defining an upper limit on the number of binarized symbols represented by any individual output data portion relative to the size in bytes of that output data portion; providing (at step 2320) padding data for each output data portion that does not satisfy this constraint so as to increase the size in bytes of that output data portion to satisfy the constraint; Detecting (at step 2330) encoding attributes applicable to a given portion of output data; selecting (at step 2340) a constraint for use with a given output data portion from two or more candidate constraints in response to the detected encoding attribute; Includes.

[0137] Exemplary embodiments also provide an image decoder including circuitry configured to interpret an encoded signal generated by controlling an image data encoding device of any one or more embodiments described herein and to output a decoded video image.

[0138] Exemplary embodiments also provide an image decoder including circuitry configured to interpret encoded signals generated by controlling the first and second data encoders and any one or more of the controllers of the embodiments described herein and to output a decoded video image.

[0139] The embodiments described herein may be implemented in any suitable form including hardware, software, firmware or any combination of these. The embodiments described herein may optionally be implemented at least partly as computer software running on one or more data processors and / or digital signal processors. The components and elements of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed functionality may be implemented in a single unit, in multiple units or as part of other functional units. Thus, embodiments of the present disclosure may be implemented in a single unit or may be physically and functionally distributed between different units, circuits and / or processors.

[0140] Obviously, numerous modifications and variations of the present disclosure are possible in light of the above teachings, and it is therefore to be understood that, within the scope of the appended provisions, the technology may be practiced otherwise than as specifically described herein.

[0141] Each aspect and feature is defined by the following numbered paragraphs. (1) an entropy encoder configured to selectively encode data items representing image data to generate encoded binary symbols for successive output data portions; an attribute detector configured to detect encoding attributes applicable to a given output data portion; a selector configured to select a constraint for use with said given output data portion from two or more candidate constraints in response to the detected encoding attribute; Equipped with the entropy encoder is configured to generate an output data stream, the output data stream being subject to a constraint defining an upper limit on the number of binarized symbols that can be represented by the byte size of any individual output data portion relative to a byte size of the output data portion; To satisfy this constraint, the entropy encoder is configured to provide padding data for each output data portion that does not satisfy the constraint, to increase the size in bytes of that output data portion. Image data encoding device. (2) the entropy encoder is configured to selectively encode data items representing image data to be encoded by a first, context-adaptive binary arithmetic coding (CABAC), encoding system or a second, bypass, encoding system to generate encoded binarized symbols. An image data encoding device according to (1). (3) the image data represents one or more pictures, each picture comprising: (i) one or more slices within each Network Abstraction Layer (NAL) portion; (ii) zero or more tiles, each defining respective horizontal and vertical boundaries of a picture region and each being unconstrained to be encapsulated within a respective NAL unit; contains data representing each slice of a picture is decodable independently of other slices of the same picture, and the tiles are decodable independently of other tiles of the same picture; The output data portion includes one or more pictures, slices, and tiles. An image data encoding device according to (1). (4) The second coding system is a binary arithmetic coding system that uses a fixed 50% probability context model. An image data encoding device according to (1) or (2). (5) The above restriction is: N <= K1 * B + (K2 * CU) (constraint equation 1) Here, N=the number of binarized symbols in the output data section. K1 is a constant, B = the number of encoded bytes in the output data portion, K2 is a variable that depends on the characteristics of the minimum size coding unit employed by the image data coding device; CU=the size of the picture, slice or tile represented in the output data portion expressed in the number of minimum size coding units It is defined as The image data encoding device according to any one of (1) to (4). (6) at least two candidate constraints are defined by the constraint equation 1; each of the set (K1, K2) is associated with each of the at least two candidate constraints; The selector is configured to select a sequence of (K1, K2) for the given output data portion. An image data encoding device according to (5). (7) the controller is configured to encode, in association with an output data stream representative of the predetermined output data portion, a representation of the encoding attributes applicable to the predetermined output data portion. The image data encoding device according to any one of (1) to (6). (8) The image data encoding device includes a quantization unit configured to selectively operate in a dependent quantization mode; The coding attribute indicates whether the dependent quantization mode is enabled or disabled for a given output data portion. An image data encoding device according to (7). (9) The entropy encoder a detector configured to detect, at a given stage in the encoding of a current output data portion, whether said constraint is satisfied by said current output data portion; a padding data generator configured to generate and insert sufficient padding data into the current output data portion such that the output data portion including padding data to be inserted satisfies the constraint; Includes The image data encoding device according to any one of (1) to (8). (10) The predetermined stage is the end of the encoding of the current output data portion. An image data encoding device according to (6). (11) A video storage device, a capture device, or a transmission / reception device, comprising an image data encoding device according to any one of (1) to (10). (12) selectively encoding data items representative of image data to generate encoded binary symbols for successive output data portions; generating an output data stream subject to a constraint defining an upper limit on the number of binary symbols represented by any individual output data portion relative to the size in bytes of said output data portions; providing padding data for each output data portion that does not satisfy the constraint so as to increase the size in bytes of that output data portion to satisfy the constraint; Detecting encoding attributes applicable to a given portion of output data; selecting a constraint from two or more candidate constraints for use with a given output data portion in response to the detected encoding attribute; Includes Image data encoding method. (13) Computer software that, when executed by a computer, causes the computer to carry out the method according to (12). (14) A machine-readable non-transitory storage medium for storing the computer software described in (13). (15) A data signal including encoded data generated according to the method described in (12). (16) An image data decoder configured to decode a data signal according to (15).

Claims

1. An entropy encoder configured to encode data items representing image data to be encoded by a first, context-adaptive binary arithmetic coding (CABAC) encoding system or a second, bypass encoding system to generate encoded binary symbols; an attribute detector configured to detect encoding attributes applicable to a given output data portion; a selector configured to select a constraint for use with said given output data portion from two or more candidate constraints in response to the detected encoding attribute; Equipped with the entropy encoder is configured to generate an output data stream according to a constraint defining an upper limit on the number of binary symbols represented by any individual output data portion relative to a byte size of the output data portion; To satisfy this constraint, the entropy encoder is configured to provide padding data for each output data portion that does not satisfy the constraint, to increase the size in bytes of that output data portion. Image data encoding device.

2. The image data represents one or more pictures, each picture comprising: (i) one or more slices within each Network Abstraction Layer (NAL) portion; (ii) zero or more tiles, each defining respective horizontal and vertical boundaries of a picture region, and each being unconstrained to be encapsulated within a respective NAL unit; contains data representing each slice of a picture is decodable independently of other slices of the same picture, and the tiles are decodable independently of other tiles of the same picture; The output data portion includes one or more pictures, slices, and tiles.

2. The image data encoding device according to claim 1.

3. The bypass coding system is a binary arithmetic coding system that uses a fixed 50% probability context model.

2. The image data encoding device according to claim 1.

4. The constraint is: N <= K1 * B + (K2 * CU) (Constraint equation 1) Here, N=the number of binary symbols in the output data section. K1 is a constant, B = the number of encoded bytes in the output data portion, K2 is a variable that depends on the characteristics of the minimum size coding unit employed by the image data coding device; CU = the size of a picture, slice or tile represented in the output data portion expressed in the number of coding units of the smallest size It is defined as 2. The image data encoding device according to claim 1.

5. At least two candidate constraints are defined by the constraint equation 1; each of the set (K1, K2) is associated with each of the at least two candidate constraints; The selector is configured to select a sequence of (K1, K2) for the given output data portion.

5. The image data encoding device according to claim 4.

6. The entropy encoder is configured to encode, in association with an output data stream representative of the given output data portion, a representation of the encoding attributes applicable to the given output data portion.

2. The image data encoding device according to claim 1.

7. the image data encoding device includes a quantization unit configured to selectively operate in a dependent quantization mode; The coding attribute indicates whether the dependent quantization mode is enabled or disabled for a given output data portion.

7. The image data encoding device according to claim 6.

8. The entropy encoder comprises: a detector configured to detect, at a given stage in the encoding of a current output data portion, whether said constraint is satisfied by said current output data portion; a padding data generator configured to generate and insert sufficient padding data into the current output data portion such that the output data portion including padding data to be inserted satisfies the constraint; Includes 2. The image data encoding device according to claim 1.

9. The predetermined stage is the end of the encoding of the current output data portion.

9. The image data encoding device according to claim 8.

10. A video storage device, a capture device, and a transmitting / receiving device, comprising the image data encoding device according to claim 1.

11. A method for producing encoded binary symbols, comprising: encoding data items representing image data to be encoded by a first, context-adaptive binary arithmetic coding (CABAC) coding system or a second, bypass coding system; generating an output data stream according to a constraint defining an upper limit on the number of binary symbols represented by any individual output data portion relative to a byte size of said output data portion; providing padding data for each output data portion that does not satisfy said constraint to increase the size in bytes of that output data portion so as to satisfy said constraint; Detecting encoding attributes applicable to a given portion of output data; selecting a constraint from two or more candidate constraints for use with a given output data portion in response to the detected encoding attribute; Includes Image data encoding method.

12. Computer software which, when executed by a computer, causes the computer to carry out the method of claim 11.

13. A machine-readable non-transitory storage medium storing the computer software of claim 12.

Images