encoding a data array

Abstract

The data array is encoded. When encoding a block of data elements in the data element array, the data values ​​of the data elements in the block are represented and stored as truncated data values ​​in the data group using one or more subsets of the most significant bits of the corresponding bit sequence of the data values ​​of the data elements. A rounding mode is selected from several available rounding modes that can be applied when decoding the block of data elements, and an indication of the selected rounding mode is provided along with the encoded data group. The rounding mode is associated with one or more rounding bit sequences that can be applied to the truncated data values ​​when decoding the data group to obtain the decoded data values ​​of the data elements in the block.

Description

Technical Field

[0001] This invention relates to methods and apparatus for encoding data in a data processing system, and more particularly to methods for compressing and storing image data such as texture or frame buffer data in a graphics processing system. Corresponding decoding methods and apparatus are also described. Background Technology

[0002] Graphics processing systems typically store the generated image data in a frame buffer. The frame buffer usually contains a complete dataset of the frame (image) to be displayed, including, for example, the brightness values ​​of each pixel within that frame (e.g., when the image data represents a grayscale image). A suitable display driver can then read the contents of the frame buffer and use the image data stored therein to drive the display to show the desired frame (image).

[0003] However, the storage and access of image data (“frame buffer data”) in the frame buffer places relatively high demands on the storage and / or bandwidth resources of, for example, data processing systems, or conversely, leads to performance degradation when such demands cannot be met.

[0004] To reduce the burden on graphics processing systems, it is desirable to be able to store such frame buffer data in a “compressed” format. This is particularly desirable in data processing devices such as portable devices like digital cameras or mobile devices that include such cameras, where processing resources and power may be relatively limited.

[0005] Another example in the context of graphics processing would be storing texture data, for example, as an array of texture elements (or "texels"), each representing given texture data (e.g., color, brightness, etc.), which can then be mapped to individual sampling locations (e.g., pixels) in the resulting rendered output. Again, storing and accessing this texture data places relatively high storage and / or bandwidth requirements on the data processing system.

[0006] Of course, similar considerations apply to various other instances of data processing where it is desirable to reduce the amount of data required to store and / or transmit certain information.

[0007] Therefore, it is common practice to process data by encoding arrays of data elements to compress the data and reduce bandwidth and storage consumption. Various data compression schemes have been developed for this purpose.

[0008] Some data compression schemes encode data in a lossless manner, allowing the original array of data elements to be perfectly reconstructed from the encoded data. However, for some applications, losing some output fidelity may be acceptable, and it may be more desirable to guarantee a given bandwidth (bit rate).

[0009] Therefore, other data compression schemes can encode data into fixed-size data blocks. In that case, compressing to fixed-size data blocks is inherently "lossy" (because when the fixed-size data block is full, any remaining bits will not be included in the data block, so these bits will be effectively discarded, and therefore, the information associated with the discarded bits will be lost). Summary of the Invention

[0010] The applicant believes that there is still room for improvement in the settings for encoded data, especially when using this lossy compression scheme.

[0011] According to a first aspect of the invention, a method is provided

[0012] A method for processing an array of data elements, each data element having an associated data value represented by a corresponding bit sequence, the method comprising the following steps:

[0013] For the data element block in the data element array:

[0014] Using a subset of one or more most significant bits of the corresponding bit sequence of the data value of the data element, the data value of each data element in the data element block is represented as a truncated data value; and

[0015] The rounding mode to be applied when decoding the data element block is selected from a plurality of available rounding modes, wherein each rounding mode is associated with one or more rounding bit sequences, and wherein the rounding mode to be applied when decoding the data element block is selected by the following process:

[0016] Based on the data values ​​of the data elements in the data element block, determine the first representation value of the data element block;

[0017] For at least one of the plurality of available rounding modes, a corresponding representation value for the data element block is determined, wherein the representation value of the rounding mode is determined based on the truncated data value of the data element in the data element block and a corresponding rounding bit sequence associated with the rounding mode;

[0018] The first representation value of the data element block determined based on the data value is compared with the corresponding representation value determined for at least one of the plurality of available rounding modes to determine which of the plurality of available rounding modes gives a representation value closest to the first representation value; and

[0019] Based on which of the plurality of available rounding modes is determined to give a representation value that is closest to the first representation value determined for the data element block, the rounding mode to be applied when decoding the data element block is selected.

[0020] The method also includes the following steps:

[0021] The data element block is encoded by storing the truncated data value of the data element as a data packet representing the data element block and providing the data packet with data indicating the rounding mode to be used when decoding the data packet; and

[0022] When it is desired to use the data value of one or more data elements that have been encoded into the data element block in the data group, the method then includes the following steps:

[0023] The data packet, along with data indicating the rounding mode to be used when decoding the data packet, is passed to the decoder circuitry for decoding.

[0024] Extract the truncated data values ​​of the one or more data elements; and

[0025] A rounding bit sequence associated with the indicated rounding mode is added to the truncated data value to obtain the decoded data value of one or more data elements in the data element block.

[0026] According to a second aspect of the invention, a method is provided

[0027] A data processing system for processing arrays of data elements, each data element having an associated data value represented by a corresponding bit sequence, the data processing system comprising:

[0028] Encoder circuit, the encoder circuit being configured as follows:

[0029] For the data element block in the data element array:

[0030] Using a subset of one or more most significant bits of the corresponding bit sequence of the data value of the data element, the data value of each data element in the data element block is represented as a truncated data value; and

[0031] The rounding mode to be applied when decoding the data element block is selected from a plurality of available rounding modes, each rounding mode being associated with one or more rounding bit sequences, wherein the selection of the rounding mode is performed by a rounding mode selection circuit configured to:

[0032] Based on the data values ​​of the data elements in the data element block, determine the first representation value of the data element block;

[0033] For at least one of the plurality of available rounding modes, a corresponding representation value for the data element block is determined, wherein the representation value of the rounding mode is determined based on the truncated data value of the data element in the data element block and a corresponding rounding bit sequence associated with the rounding mode;

[0034] The first representation value of the data element block determined based on the data value is compared with the corresponding representation value determined for at least one of the plurality of available rounding modes to determine which of the plurality of available rounding modes gives a representation value closest to the first representation value; and

[0035] Based on which of the plurality of available rounding modes is determined to give a representation value that is closest to the first representation value determined for the data element block, the rounding mode to be applied when decoding the data element block is selected.

[0036] The encoder circuit is also configured such that:

[0037] The data element block is encoded by storing the truncated data values ​​of the data element as data packets representing the data element block and providing the data packets with data indicating the rounding mode to be used when decoding the data packets;

[0038] The data processing system further includes a decoder circuit, which is configured such that:

[0039] When it is desired to use the data value of one or more data elements that have been encoded into the data element block in the data group:

[0040] Extract the truncated data values ​​of one or more data elements from the data element block; and

[0041] A rounding bit sequence associated with the indicated rounding mode is added to the truncated data value to obtain the decoded data value of one or more data elements in the data element block.

[0042] This invention generally relates to encoding blocks of data elements into data packets, for example, to compress data. In this invention, a rounding “mode” is selected for the block of data elements being encoded (from a plurality of available rounding modes), and then the selected rounding mode (i.e., or at least the data indicating that rounding mode) is provided together with or preferably as part of the data packets, for example, to a suitable decoder (circuit). Each rounding mode is associated with one or more rounding bit sequences. Then, when the data packets are subsequently decoded, for example to obtain a decoded output, the rounding bit sequence for the selected rounding mode can (and indeed) be applied to the data elements in the data element block.

[0043] When a data packet that has been encoded in this way is subsequently decoded (i.e., for a data packet, data indicating the selected rounding mode has been provided), the decoder (circuit) is thus able to determine the rounding mode that has been provided for the data packet, and then add the sequence of rounding bits associated with the indicated rounding mode to the (encoded) data value of each data element in the data element block.

[0044] The rounding bit sequence associated with different rounding modes thus acts as a corresponding rounding "modifier" that can (and indeed) be added to the truncated data value at the decoder (circuit).

[0045] Therefore, the selected rounding mode for the data element block can (and indeed can) be applied block by block by the decoder (circuit) to modify the data values ​​of all data elements within the data element block that has been encoded as a data group to obtain the decoded output.

[0046] Thus, by identifying and selecting the (most) appropriate rounding mode to apply to the block of data elements from a set of available rounding modes, it has been found that improved reproduction of the initial data can be facilitated, and in particular, to reduce unwanted coarse steps (or jumps) in the data values, which may otherwise occur at least for some encoding schemes, as will be further explained below.

[0047] Specifically, the rounding mode to be applied to the data element block according to the invention is advantageously combined with a (lossy) encoding scheme, in which the encoding of the data element block involves a truncated bit sequence representation of the data values ​​of the data elements within the block, such that only a subset of one or more most significant bits (“truncated” data values) of the corresponding bit sequence representation of the initial data values ​​is used to represent the data values ​​of the data elements in the data element block, and only these subsets (truncated data values) of the multiple most significant bits of the corresponding data elements are included in the encoded data blocks.

[0048] To avoid confusion, it should be understood that the most significant bit (singular) of a bit sequence is the position of the highest-value bit. Multiple most significant bits (complex) are multiple bits that are closest to and include that most significant bit. For example, for a bit sequence containing N bits, where the Nth bit represents the maximum value (most significant bit), multiple most significant bits may include the Nth bit, the (N-1)th bit, and so on.

[0049] Therefore, the present invention encodes the data element block by truncating the bit sequence representing the data value of each data element within the data element block, so as to give the corresponding "truncated data value" of the data element, which includes only a subset of one or more most significant bits of the bit sequence representation of the initial data value.

[0050] This truncation of the bit sequence (i.e., data values) can thus provide the desired data compression. However, naturally, this truncation results in the loss of any information not included in the data block, represented by the multiple less significant bits. Consequently, less information can be represented in the decoded output.

[0051] This might be the case, for example, when blocks of data elements are encoded in "raw" format as fixed-size data packets. In this case, once the data packet is full, any remaining bits are effectively discarded. Therefore, encoding as fixed-size data packets is inherently lossy.

[0052] Therefore, in a preferred embodiment, the data element array (at least for encoding) is divided into multiple data element blocks that can be independently encoded / decoded, and a certain bit budget (size) is allocated to each block to ensure that each block is encoded into a fixed-size data packet. Then, preferably, starting from the most significant bit (i.e., the most significant bit) of each data element, the multiple most significant bits of the data elements in each block are encoded in their "original" form (by including the bit value "as is" in the data packet), and, as long as there is still available space in the data packet, after adding the most significant bit of each data element, the process moves to the next most significant bit of the data element, and so on, until the data packet is full (the allocated bit budget has been used).

[0053] As part of this encoding scheme, it is preferable to change the order in which the bit values ​​of data elements are added to data blocks, preferably in a random manner. For example, the order in which data elements are processed when the most significant bit value is added to a data block may differ from the order in which the next most significant bit value is added to a data block, and so on. Preferably, the order in which multiple bits are added to data blocks is also changed between different blocks of data elements, for example, within the entire array of data elements and / or within a sequence of such arrays. This helps to avoid introducing repetitive artifacts in the decoded output. For example, if data elements in all blocks within the entire array of data elements are always processed in the same order, a larger number of bits can be included in the data block for the data element processed first, and this will introduce repetitive patterns in the decoded output, for example, based on the division of the array into blocks.

[0054] Therefore, the encoding scheme according to the invention preferably stores as many most significant bits of data elements in a data element block as possible within the allocated bit budget, while still ensuring a fixed-size data block. However, depending on the expected (fixed) size of the data block, i.e., the number of bits already allocated to the data block, this may mean that for each data element within the block, only a subset of the most significant bits, such as one or two most significant bits, are included in the data block (where multiple least significant bits are not included).

[0055] The applicant has recognized that, in this case, when such data packets are subsequently decoded, the decoded output obtained (only) based on the truncated data values ​​may then contain artifacts, particularly in the form of relatively coarse steps in the data values.

[0056] For example, in some preferred embodiments, the data element array may represent a grayscale image data array, wherein each data element is associated with a brightness value (representing the corresponding brightness value at the spatial location (sample) associated with the data element).

[0057] For example, consider a typical scenario where it is desirable to encode an 8×8 block of 8-bit luminance samples into a fixed-size 128-bit data packet. If the entire 128-bit bit budget of the data packet is available for the luminance sample (although it should be understood that this is generally not the case, and is not the case in implementations, for example, because some of the allocated bits are preferably used to signal the encoding scheme and / or rounding mode, etc., for example in a suitable header of the data packet, such that typically less than all 128 bits are available for encoding the luminance value, as will be explained further below), then it means that only two bits are available for each luminance sample.

[0058] Therefore, when the preferred encoding scheme described above is applied, the bit values ​​of the two most significant bits (only) of each brightness sample (i.e., the value of the most significant bit position (most significant bit) and the value of the next most significant bit position) (in its "original" format) are added to the data group.

[0059] However, since the entire bit budget has been used, the data packet is "full", and the remaining (lower significant) bits of the brightness sample are not added to the data packet (therefore, information associated with the multiple lower significant bits will be lost).

[0060] It should be understood that, depending on the desired level of compression, data values ​​may be intentionally truncated, resulting in the loss of a significant amount of information. For example, for the 8-bit sample in this illustrative example, this means that the truncated data values ​​are then restricted to having values ​​“0” (00000000), “64” (01000000), “128” (10000000), or “192” (11000000) (compared to the initial 256 values ​​(from 0 to 255)).

[0061] The result is that the decoded image (or image sequence) will have relatively coarse brightness steps, and it may not be possible to represent smooth brightness changes because multiple least significant bits are not used. For example, these brightness steps are particularly noticeable during brightness fade-in, such as when a particular block in the image sequence fades from black (Y=0) to white (Y=256) over time.

[0062] During this luminance fade-in, it can be expected that all luminance samples within a single block (typically representing only a portion of the entire image) will have similar (or even identical) luminance values ​​(though this is not necessarily the case). When luminance fades in, it is generally expected that the luminance of the block will fade in gradually to create a smooth transition throughout the image sequence. However, in the example above, instead of providing the desired smooth fade-in effect, the luminance of the block will change in relatively coarse steps, for example, jumping from 0 to 64 to 128 to 192.

[0063] Although an example has been given in the context of a luminance sample block, it should be understood that the problem described above can generally be applied to any other type of data (array) that can be properly and desirablely encoded in this way.

[0064] These problems can be mitigated by using the present invention. Specifically, as described above, the present invention proposes the use of a rounding "modifier" that can be applied block-by-block by the decoder (circuit), such that the decoder preferably adds a specific value (i.e., in the form of a specified rounding bit sequence) to the individual truncated data values ​​(bit sequences) of data elements within a data element block. The rounding mode to be used for the block is selected from a set of multiple available rounding modes, and this therefore allows for finer-grained control over the data values ​​of the decoded block (compared to using truncated data values ​​individually), for example, making it possible to reproduce smooth changes in data values ​​between blocks in the decoded output.

[0065] Therefore, when encoding data, the encoder (circuit) analyzes the initial block of data elements and determines which rounding mode will provide the best reconstruction of the initial data when truncated data values ​​are added to that block. The rounding mode determined to provide the best reconstruction of the initial data is then selected as the rounding mode for the data element block. For example, in the case of a luminance sample block, the rounding mode for a specific data element block can be selected such that the apparent (e.g., luminance) of the reconstructed block most closely represents the luminance of the initial block sent to the encoder.

[0066] To select the rounding mode for data element blocks, analysis is performed to allow comparison between the initial data value and the values ​​obtained in the decoded output based on the truncated data value with different rounding modes applied.

[0067] To facilitate this comparison, an initial data value is used to determine a first representation value for the data element block. This first representation value can be determined directly or indirectly based on the data value in any suitable manner. For example, in some implementations, the initial data value can be processed or transformed into a different form, for example, to facilitate the determination of the first representation value. This could include, for example, truncating the initial data value (but to a lesser extent than truncation used for encoding the data value). Various other arrangements are also possible in this regard.

[0068] This representation value can effectively represent the "average" of data values ​​within a block. The representation value can represent the "true" average (e.g., mean) data value of the data elements within the block, or it can simply represent an estimate or approximation of the "average" data value of the block. In practice, generally, this first representation value can be determined in any suitable manner as desired to provide a suitable representation value that can be used in the manner described herein.

[0069] Therefore, in this implementation, a representation value is calculated for the data value block based on at least some of the data values ​​of the data elements within the block. The calculation of the representation value can be performed in any appropriate and desired manner, depending on what the data value block represents.

[0070] For example, in some implementations, the representation value may simply include the sum of the data values ​​of the data elements in the block (or the sum of at least some of the data values ​​of the data elements in the block). Generally, it will be appreciated that any suitable algebraic function can be applied to the data values ​​of the data elements in the block to determine the representation value. For example, this may involve determining a weighted sum, a sum of squares, or other power values.

[0071] Specifically, the method of determining the representation value can depend on the data type, and for example, the required level of precision. For instance, when the data value represents a luminance value, it is preferable to use the sum of squares of the data values ​​to determine the representation. This may be suitable for the non-linear relationship between the approximate luminance value and the perceived luminance (i.e., a luminance value of 128 is not half as bright as a luminance value of 256; in fact, this relationship is described by a gamma curve involving powers of 2.2, so the summation and square provides a reasonable approximation). It should be understood that color (e.g., RGBA) and chromaticity (UV) values, for example, can be handled in a similar manner.

[0072] Therefore, in an implementation, determining the first representation value includes summing the squares of the data values ​​of at least some of the data elements within the block; and determining the representation value for a corresponding rounding mode includes summing the squares of the modified data values ​​of at least some of the data elements within the block, wherein the modified data values ​​of the data elements are obtained by adding a corresponding rounding bit sequence associated with the rounding mode to the truncated data values ​​of the data elements.

[0073] However, there are certainly various other settings, and the representation value can be determined in a more or less complex way depending on expectations.

[0074] For example, various other algebraic optimizations can be performed to facilitate hardware computation of the representation values. This can be done, for instance, by algebraically expanding the function used to determine the representation values ​​and removing terms that contribute only a small amount to the result.

[0075] What will be understood is that it is not necessary to use every data element in a block to compute the representation value, and the computation can use only a specific subset of the data elements within the block. For example, the representation value can be computed using every other data element. Alternatively, the representation value can be computed using sub-blocks of data elements within the block. This helps simplify computation while still providing appropriate representation values ​​for blocks of data elements as expected (especially since data elements within a single block can generally be expected to have similar data values).

[0076] Once the first representation value of the block has been determined using (at least some) initial data values ​​of the data elements in the block, regardless of the method used, the corresponding representation value of at least one of the multiple available rounding modes, and preferably the corresponding representation value of each available rounding mode, is determined in a similar manner. The first representation value determined based on the initial data value is then compared with the corresponding representation value obtained for the at least one of the multiple available rounding modes, and preferably each rounding mode, to determine the rounding mode that gives the closest match to the first representation value. The rounding mode that best matches the first representation value is then selected as the rounding mode to be used for the block.

[0077] Therefore, the corresponding representation value determined for different available rounding modes is preferably determined in the same way as the first representation value (e.g., using the same function), but the data value of the data element in the block is replaced with a truncated value associated with the corresponding ingest bit sequence associated with the rounding mode; that is, the initial data value is replaced with the modified data value that will be generated in the decoding output for the corresponding rounding mode.

[0078] That is, in the implementation, the representation value associated with a specific rounding mode is determined by modifying the truncated data value by adding an associated rounding bit sequence to the truncated data value, and then determining the representation value based on such modified truncated data value.

[0079] For example, if the first representation value X is derived from the initial data value {x} i A function f is defined in the array X = f(x1, x2, ..., xn). n If the rounding mode is specified, then the corresponding value Y is represented by the rounding mode. R It can be determined by the same function, but operates under the appropriate rounding modifier R used for that data value. i Modified truncated data value {y i}, that is: Y R = f(y1+R1, y2+R2, ..., y n +R n ).

[0080] In some preferred embodiments, each rounding mode is associated with a specific (single) rounding bit sequence, which is then applied to each data element in the data element block (although potentially at different bit positions, as will be further explained below).

[0081] However, it is also contemplated that a given rounding mode can be associated with multiple rounding bit sequences, which can be applied to different data elements in a block of data elements as desired, and this is the case in some implementations. In this case, which rounding bit sequence to apply to a particular data element when using the associated rounding mode can be selected as desired based on any suitable criterion.

[0082] For example, a simple criterion could be to select the rounding bit sequence to be applied to a data element based on how many most significant bits are used to represent the truncated data value of that data element (so that different rounding bit sequences can be applied to different data elements within a block, depending on how many most significant bits are stored for different data elements).

[0083] Depending on the desired outcome, the criteria for determining which rounding sequence to apply to which data element can be more or less complex. For example, in some cases, the rounding sequence associated with a particular rounding mode can be indexed based on the data element, such that when that rounding mode is selected, the index can be used to determine the rounding sequence to be applied to the data element (and such that the particular rounding mode potentially applies different rounding sequences to the various data elements).

[0084] Therefore, in general, each rounding mode can be associated with one or more rounding bit sequences. Furthermore, for a set of multiple available rounding modes, some rounding modes can be associated with a single rounding bit sequence, and some rounding modes can be associated with multiple rounding bit sequences.

[0085] There are various settings available in this regard.

[0086] Therefore, each available rounding mode can be associated with one or more predetermined sequences of four, six, etc. (one or more rounding bit sequences) to be added to the truncated data value of the data element to obtain the decoded data value.

[0087] Rounding bits can be added to truncated data values ​​in any suitable manner (when determining the representation value, and when decoding data blocks using the selected rounding mode).

[0088] In a particularly preferred embodiment, a rounding bit sequence for the data element is immediately added after the plurality of most significant bits stored for the truncated data value of the data element. This means that even if the rounding bit sequence to be added to the truncated data values ​​of different (e.g., all) data elements in the block is the same (as is the case in some preferred embodiments), the rounding bit sequence can be added at different bit positions, for example, depending on how many most significant bits are used to represent the truncated data value of the data element. Therefore, although in a preferred embodiment the same rounding bit sequence is applied to all data elements in the block, the actual rounded value added for different data elements can be different.

[0089] However, other settings are certainly possible. For example, if the data value is represented in 8-bit form and it is expected that only one or two most significant bits are stored for each data element, then the corresponding rounding bit sequence can be added to (and, for example, each) the truncated data value of the data element, replacing the third, fourth, etc., most significant bits, i.e., at the same bit position, regardless of whether one or two most significant bits are stored for that data element. In that case, the same rounding value can be added to each data element (i.e., the same rounding bit sequence at the same bit position). In other examples, instead of adding a rounding bit sequence after the multiple most significant bits sent, a rounding bit sequence can be added in place of multiple least significant bits.

[0090] Preferably, a comparison is made between the first representation value and the corresponding representation values ​​obtained for each available rounding mode. However, this is not always necessary. For example, if the first tested available rounding mode gives an exact match to the initial data, it may not be necessary to check other available rounding modes (and therefore may not be necessary in this implementation).

[0091] In addition, one of the available rounding modes may include a "zero" mode, where the added rounding bit sequence is "0" (or alternatively, where no rounding bit sequence is added). In that case, the representation value for the zero mode can be calculated (only) based on the truncated data value, and then compared with other available rounding modes.

[0092] While the implementation has been described above with respect to individual data element blocks, it should be understood that it is preferable to select a rounding mode for each data element block in the entire data element array. Therefore, different rounding modes can be appropriately selected (and preferably indeed) for different data element blocks in the entire data element array to provide an improved reconstruction of the initial data values ​​for each block. However, in principle, only a subset of the blocks, i.e., less than all, can be handled in this way.

[0093] This scheme has been found to represent initial data values ​​better in the decoded output, for example, with improved granularity between blocks, such as from block to block in an array of data elements (e.g., on a single image) or from block to block in a sequence of data arrays (e.g., an image sequence).

[0094] For example, in some cases, it can be expected that data elements within a single block will generally have similar data values ​​(although this is not necessary). This might be the case, for example, when a block represents a portion of an image and the data values ​​are luma values, since the luma across a specific part of the image (i.e., a block) can be substantially constant. Therefore, applying an appropriate rounding mode at the block level can provide finer control over the block granularity and better reproduction of the initial data values.

[0095] Accordingly, the block size can be adjusted to ensure this is the case. That is, in an implementation, the size of the data element block can be selected such that all data elements within the data element block (desired) have similar data values, for example, falling within a certain range. Therefore, the present invention attempts to mitigate the above-mentioned problems and thus allows for better reproduction of the initial data array, for example, by providing better reproduction of the apparent brightness level of the blocks throughout the data array, even if the bit sequence representation of the data values ​​is strongly truncated during the encoding of the data element blocks (so that multiple least significant bits are not included in the data block).

[0096] The selected rounding mode of the block can then be indicated to the decoder in any suitable manner. Sideband information can be used, for example, if needed.

[0097] However, preferably, in the data packet, the rounding mode to be used for the block is signaled in the appropriate header of the data packet. Therefore, the decoder is able to read this information from the header when processing the data packet and apply the appropriate rounding mode when decoding the block.

[0098] The improved granularity according to the invention is achieved by selecting the (most) appropriate rounding mode from a plurality of available rounding modes. Therefore, a larger number of available rounding modes can further improve granularity. However, there may be a trade-off between the increase in granularity and the cost of calculating the representation values ​​of the available rounding modes, and the increased data storage requirements of signaling the selected rounding mode for data grouping.

[0099] Furthermore, when the rounding mode is signaled in the data packet itself (e.g., in the header), increasing the number of rounding modes beyond a certain point may significantly impair the overall quality of the decoded output, as this would mean sending fewer bits for the data elements.

[0100] Therefore, preferably, only a limited number of rounding modes are available, such as two, four, eight, or sixteen rounding modes. This has been found to provide improved data fidelity while still allowing rounding modes to be signaled relatively inexpensively. Furthermore, because rounding modes are applied and signaled at the block level, they can be conveniently signaled (i.e., since this is the level at which data elements are encoded / decoded).

[0101] For example, in a preferred embodiment, there may be four available rounding modes. In that case, the selected rounding mode can then be signaled using (only) two bits. In principle, the rounding bit sequence of the block can be directly encoded. For example, in some preferred embodiments, the selected rounding mode is associated with a specific bit sequence in the truncated data values ​​of the data elements to be added to the encoded block. In that case, the multiple bits used to signal the rounding mode can directly reflect the rounding bit sequence to be added. However, preferably, it is the rounding mode that is signaled (rather than attempting to signal the rounding bit sequence itself), which then allows any rounding mode (as associated with any desired rounding bit sequence) to be signaled accordingly. For example, four pairs of bits "00", "01", "10", and "11" can be used to represent four available rounding modes, where each pair represents a different rounding mode associated with a different set of one or more predetermined rounding bit sequences. Generally, any suitable association can exist that allows identification of the rounding bit sequence associated with a particular rounding mode. For example, in one implementation, the rounding bit sequence associated with a particular rounding mode can be obtained from a suitable lookup table, for example. However, various other arrangements are possible in this regard.

[0102] Therefore, only two bits can be used to signal any of the four rounding modes (rounding bit sequences). In the illustrative example given above, this means that 126 bits will still be available for the sample data (and / or any other information that may need to be added to the data blocks, such as a bit sequence indicating the encoding mode, etc.). This means that for some samples, only a single most significant bit is sent (and for others, two most significant bits are sent). However, this is still acceptable and information loss can be at least partially compensated for by using the (most) appropriate rounding mode.

[0103] Similarly, using three bits, eight different rounding modes can then be signaled, and so on. As mentioned above, there is a trade-off between increasing the number of bits required to signal the rounding mode and leaving multiple bits idle to encode as many of the most significant bits of the data element as possible.

[0104] Therefore, this invention allows for improved data fidelity, and in particular, improved block-to-block granularity in reconstructed data, even when the bit sequence representation of the data values ​​of data elements within a data element block is very strongly truncated during the encoding of the data element block. Furthermore, this can be achieved with virtually no additional data storage and / or data transfer overhead.

[0105] In a preferred embodiment, as described above, the invention employs an appropriate data element block (e.g., an N×N block) that will have a given size in the original format, and encodes this block into data packets of a desired size, for example, to meet the required compression ratio. Then, preferably, this process is repeated for the other data element blocks in the entire data element array to generate an encoded data stream comprising multiple (independently) encoded data packets representing the individual blocks into which the entire data element array has been divided.

[0106] Embodiments of the present invention thus enable the compression of a given data array into a stream of data packets, each with a fixed size. For example, the present invention can compress to a fixed (selected) bit rate, such as half or one-third of the initial size. For example, a 64-byte block in its original form can be compressed to a fixed ratio (half-ratio compression), such as 32 bytes per block. Similarly, an original block of 96 bytes (e.g., an 8×8 YUV420 block might be this case) can be compressed to 48 bytes (half-ratio) or 32 bytes (one-third ratio).

[0107] The aforementioned "primitive" encoding scheme may be particularly suitable when the array of data elements represents highly complex or random data. However, it will be recognized that various other, more complex encoding schemes exist that may be better suited to certain other types of data. For example, such schemes can leverage expected behavior in the data to provide more efficient encoding (however, these schemes may therefore be less efficient when the data does not match the expected behavior). For instance, for natural image data that has been converted to the frequency domain, the existence of a leading sequence of "0"s that can be efficiently encoded can be expected.

[0108] That is, the applicant has also recognized that the “original” encoding settings of the present invention may not be suitable for a particular form of input data. Therefore, in an embodiment, the encoder can identify instances where it may not be suitable to use a particular encoding scheme described according to the present invention, and in such cases, preferably encode the input data in a different manner and indicate the processing to be performed to the decoder.

[0109] Therefore, the encoder may be able to select from several different available encoding schemes based on the input data. The encoder can select an encoding scheme based on the input data in various suitable ways, such as based on a suitable metric. For example, in one implementation, the encoder may perform a frequency transformation operation on the array of data elements and then select an encoding scheme based on a suitable metric such as the sum of the absolute values ​​of the transform coefficients. However, various other arrangements are certainly possible.

[0110] For example, when the data is image data, in some cases, the data can be encoded more efficiently by first performing a frequency domain transform and then bit-plane encoding of the frequency domain coefficients. Whether this is more efficient than simply encoding the raw data values ​​will generally depend on the complexity of the image data. Therefore, checks can be performed, for example by summing the data values ​​(in the frequency domain), to determine which encoding scheme is likely to give the best results, and then that encoding scheme can be selected.

[0111] Therefore, in some preferred embodiments, the original encoding scheme described above can be used in combination with another encoding scheme. The encoder (circuit) can thus analyze the array of data elements and determine (from multiple available encoding schemes) which encoding scheme to use. For example, the determination of which encoding scheme to use can be performed in the initial steps early in the encoding process. Alternatively, in some preferred embodiments, the determination of which encoding scheme to use can be performed in parallel with the application of multiple different available encoding schemes, and then the encoding scheme to be used can be selected at the end. This can help reduce input latency. Various arrangements are possible in this regard.

[0112] Therefore, in implementations, the encoder can select between two (or more) encoding schemes. This can also be indicated by signals for data packets (and preferably within data packets). For example, the header can also indicate which encoding scheme is being used.

[0113] In any of the embodiments described herein, the data element array can take any desired and suitable form. Therefore, while examples have been given above in the context of grayscale image data, it should be understood that the invention is not limited to such data and can generally be used when processing any suitable image or non-image data as desired. Thus, while preferred embodiments relate to data arrays comprising image and / or video data, other examples of data array configurations are possible if desired, and generally, the data element array can comprise any data array that can be suitably or desiredly encoded according to the invention.

[0114] The data element array preferably represents the spatial distribution of values. That is, the data element array represents a set of data values ​​distributed in a spatial domain. Therefore, each data element can represent a data value at a certain location within the spatial distribution. Thus, in an implementation, the data element array can (each) correspond to a data location array.

[0115] In a preferred embodiment, the data elements or position array may correspond to all or part of the desired output (e.g., an image), such as a still image or video frame (e.g., for display). That is, the data element array preferably includes an image data array, i.e., data that can be used to generate the image for display. Therefore, in one embodiment, the data array may correspond to a single still image to be encoded. In other embodiments, the data array may correspond to a video frame of a video frame stream to be encoded.

[0116] In some preferred embodiments, the data element array comprises a grayscale image data array. Specifically, the data values ​​may represent brightness values. However, it should be understood that the data does not need to include grayscale image data, and the invention is equally applicable to color (e.g., RGBA) image data. In that case, the above scheme can be used either separately for each channel (RGBA) or collectively for multiple channels, for example by determining an appropriate representation of a "common" value for the data values ​​across all color channels.

[0117] Generally, any desired and appropriate correspondence can exist between data elements and the desired output. For example, data elements in a block can each correspond to a pixel in the desired output. A data element block can have any desired and appropriate size or shape in terms of data elements or location, but is rectangular (including square) in implementations. Data elements can represent two-dimensional data, but can also represent one-dimensional, three-dimensional, or higher-dimensional data. Data elements can also have any desired and appropriate format, such as a format representing image data values ​​(e.g., brightness or color values).

[0118] In any of the embodiments described herein, the array of data elements may be provided in any desired and appropriate manner. Embodiments may include generating (at least some or all) of the data elements of the array. Embodiments may also, or alternatively, include reading (at least some or all) of the data elements of the array, for example, from memory.

[0119] The data elements of the array can be generated in any desired and suitable manner. In one implementation, the data elements of the array can be generated by a camera, such as a video camera. In other implementations, generating the data elements of the array can include rendering processing. Rendering processing can include deriving data values ​​represented by the data elements of the array (e.g., by rasterizing primitives to generate graphic fragments and / or by rendering graphic fragments). A graphics processor (graphics processing pipeline) can be used to generate the data elements of the array. To generate the data elements of the array, the graphics processing pipeline can contain any suitable and desired processing stages that can be included by the graphics pipeline and the processor, such as vertex shaders, rasterization stages (rasterizers), rendering stages (rasterizers), etc.

[0120] In this invention, data elements of a data array are encoded as data element “blocks,” for example, block by block. For instance, the data element array may be divided into multiple source blocks to be encoded block by block. Therefore, the processing or encoding of a data array or its data elements herein should be understood to include and generally involve processing or encoding such data element blocks. A “block” can typically comprise an N×N array of data elements.

[0121] Therefore, when encoding the (entire) data element array, such as representing an entire frame (image), the (entire) data element array is divided into multiple blocks, and then each block is encoded according to the encoding scheme of the present invention to provide a corresponding set of data packets. In a preferred embodiment, each data element block within a larger array of data elements is encoded (compressed) into fixed-size data packets. Thus, in a preferred embodiment, the data element blocks encoded into fixed-size data packets include data element blocks from the larger entire data array (and preferably, this process is repeated for each block among the multiple blocks constituting the entire data array). The data packets of the individual blocks can then be appropriately combined in a certain order to form an encoded data stream representing the entire data element array.

[0122] Therefore, this invention is a block-based scheme, wherein each block is preferably encoded independently, so that the blocks can be decoded independently subsequently. This facilitates random access to blocks within a frame that have been encoded using this invention. For example, the number of bytes in each data packet (block) is typically known, and therefore the location of the respective blocks in memory can be easily identified, making them easy to load and randomly access.

[0123] Data elements may include differences, such as comparisons with a reference frame or block. However, more often, an encoding scheme is used to encode the raw data, and the data elements can therefore represent (raw) pixel values.

[0124] The invention is further extended to the processing of encoding the data element array itself, and to the encoder (circuit) used to perform this method.

[0125] According to another aspect of the invention, a method is provided.

[0126] A method for encoding an array of data elements, each data element having an associated data value represented by a corresponding bit sequence, the method comprising the following steps:

[0127] For the data element block in the data element array:

[0128] Using a subset of one or more most significant bits of the corresponding bit sequence of the data value of the data element, the data value of each data element in the data element block is represented as a truncated data value; and

[0129] The rounding mode to be applied when decoding the data element block is selected from a plurality of available rounding modes, wherein each rounding mode is associated with one or more rounding bit sequences, and wherein the rounding mode to be applied when decoding the data element block is selected by the following process:

[0130] Based on the data values ​​of the data elements in the data element block, determine the first representation value of the data element block;

[0131] For at least one of the plurality of available rounding modes, a corresponding representation value for the data element block is determined, wherein the representation value of the rounding mode is determined based on the truncated data value of the data element in the data element block and a corresponding rounding bit sequence associated with the rounding mode;

[0132] The first representation value of the data element block determined based on the data value is compared with the corresponding representation value determined for at least one of the plurality of available rounding modes to determine which of the plurality of available rounding modes gives a representation value closest to the first representation value; and

[0133] Based on which of the plurality of available rounding modes is determined to give a representation value that is closest to the first representation value determined for the data element block, the rounding mode to be applied when decoding the data element block is selected.

[0134] The method also includes the following steps:

[0135] The data element block is encoded by storing the truncated data value of the data element as a data group representing the data element block and providing the data group with data indicating the rounding mode to be used when decoding the data group.

[0136] According to another aspect of the invention, a method is provided

[0137] An encoder for encoding an array of data elements, each data element having an associated data value represented by a corresponding bit sequence, the encoder comprising:

[0138] Encoding circuit, the encoding circuit being configured as follows:

[0139] For the data element block in the data element array:

[0140] Using a subset of one or more most significant bits of the corresponding bit sequence of the data value of the data element, the data value of each data element in the data element block is represented as a truncated data value; and

[0141] A rounding mode to be applied when decoding the data element block is selected from a plurality of available rounding modes, each rounding mode being associated with one or more rounding bit sequences, wherein the selection of the rounding mode is performed by a rounding mode selection circuit configured to:

[0142] Based on the data values ​​of the data elements in the data element block, determine the first representation value of the data element block;

[0143] For each of the plurality of available rounding modes, a corresponding representation value for the data element block is determined, wherein the representation value of the rounding mode is determined based on the truncated data value of the data element in the data element block and the corresponding rounding bit sequence associated with the rounding mode;

[0144] The first representation value determined for the data element block is compared with the corresponding representation value determined for at least one of the plurality of available rounding modes to determine which of the plurality of available rounding modes gives a representation value closest to the first representation value; and

[0145] Based on which of the plurality of available rounding modes is determined to give a representation value that is closest to the first representation value determined for the data element block, the rounding mode to be applied when decoding the data element block is selected.

[0146] The encoding circuit is further configured such that:

[0147] The data element block is encoded by storing the truncated data value of the data element as a data group representing the data element block and providing the data group with data indicating the rounding mode to be used when decoding the data group.

[0148] The encoders (encoding circuits) and encoding methods according to these other aspects may include any of the features described above with respect to other aspects of the invention.

[0149] Therefore, the encoding of the present invention produces a data stream, which preferably comprises a series of fixed-size data packets (e.g., each data packet represents an array (or block) of data elements). The encoded data stream generated by the encoder can be output to a memory for storage (e.g., for subsequent decoding and / or transmission), or it can be output to an external device (e.g., for subsequent decoding and optionally display).

[0150] When it is desired to subsequently access or use data that has been encoded into data packets, the data packets are then passed to the decoder (circuit) and can be decoded therefrom.

[0151] Therefore, the decoder (circuit) can first analyze the data packets, for example by reading their headers, to determine whether the data packets are encoded in the manner described above and whether a rounding mode has been selected.

[0152] When the decoder (circuit) identifies the rounding pattern for the data to be grouped, it extracts the truncated data values ​​of each data element in the encoded block and then adds the rounding bit sequence associated with the indicated rounding pattern to the data element to produce the decoded output.

[0153] The present invention is also extended to corresponding methods and apparatus for decoding data that has been encoded in accordance with the present invention.

[0154] According to another aspect of the invention, a method is provided for decoding a data stream comprising one or more data packets, each data packet encoding a block of data elements in an array of data elements, wherein each data element in the block of data elements has an associated encoded data value, and wherein data is provided for the data packets indicating a rounding mode to be used when decoding the data packets, the rounding mode being associated with one or more rounding bit sequences, the method comprising the steps of:

[0155] When you want to use the data values ​​of one or more data elements that have been encoded into a data element block in a data group:

[0156] Read the data group to extract the encoded data value of one or more data elements from the data element block;

[0157] Identify the rounding mode to be used when decoding the data packets; and

[0158] A rounding bit sequence associated with the identified rounding pattern is added to the encoded data value to obtain the decoded data value of the one or more data elements in the data element block.

[0159] According to another aspect of the invention, a decoder is provided for decoding a data stream comprising one or more data packets, each data packet encoding a block of data elements in an array of data elements, wherein each data element in the block of data elements has an associated encoded data value, and wherein data indicating a rounding mode to be used when decoding the data packets is provided for the data packets, the rounding mode being associated with one or more rounding bit sequences, the decoder comprising:

[0160] When you want to use the data values ​​of one or more data elements that have been encoded into a data element block in a data group:

[0161] Read the data group to extract the encoded data value of one or more data elements from the data element block;

[0162] Identify the rounding mode to be used when decoding the data packets; and

[0163] A rounding bit sequence associated with the identified rounding pattern is added to the encoded data value to obtain the decoded data value of the one or more data elements in the data element block.

[0164] Generally, a decoder decodes data packets during operation to recover a (lossy) representation of the encoded original source data (e.g., an image), and then uses (e.g., displays) that data as desired. Therefore, a decoder can be configured to display or output data obtained using the decoded data packets as soon as the data packets have been decoded.

[0165] The encoding and decoding devices discussed above can typically be provided as part of a data processing system. For example, these devices may include part of an image and / or video processing system that includes a camera, such as a digital camera, mobile phone, or tablet computer.

[0166] Generally, the processing described herein in any embodiment can be performed by any desired and suitable means. For example, in any embodiment, the process described herein can be performed by an image or video processor (codec). Therefore, in any embodiment, the data processing apparatus described herein may include or may be a video processor. Thus, the processing (encoding / decoding) circuitry, output circuitry, header generation circuitry, header reading circuitry, or input circuitry described herein in any embodiment may form part of a video processor. In any embodiment, the data processing apparatus described herein may include, or may be, a system-on-a-chip (SoC), or may form part of a system-on-a-chip (SoC).

[0167] As those skilled in the art will understand, the encoding / decoding apparatus of the present invention described herein may be part of an overall data processing system including, for example, a host (e.g., central) processor. The host processor may, for example, execute applications that require data processing by the encoding / decoding apparatus. The host processor may send appropriate commands and data to the encoding / decoding apparatus to control its performance of data encoding / decoding operations, and to generate and / or use the output required by the applications executing on the host processor. To facilitate this, the host processor may execute a driver for the encoding / decoding apparatus.

[0168] In embodiments, the apparatus or system may include and / or be able to communicate with one or more memory and / or storage devices that store the data described herein, and / or store software for performing the processes described herein. The apparatus or system may include and / or be able to communicate with a display to display images based on an array of data elements. The apparatus or system may include and / or be able to communicate with a camera that generates the array of data elements.

[0169] The memory referred to herein can be that of a data processing device or any desired and suitable memory for a data processing device. The memory can be external to the data processing device, such as being stored in memory, a video processor, or a system-on-a-chip (SoC). The memory can be, for example, main system memory.

[0170] The invention described herein can be used by data processing devices to provide and / or use all forms of data arrays, such as images or frames for display. Therefore, as described above, a data element array can include image data and / or images or frames that may correspond to the image data.

[0171] In implementation, the various functions of the invention described herein are performed on a single data (e.g., image) processing platform that provides and / or uses an array of data elements.

[0172] The invention described herein can be implemented in any suitable system, such as a system based on a suitably configured computer or microprocessor. In practice, the invention described herein is implemented in a computer and / or microprocessor-based system.

[0173] The various functions of the invention described herein can be performed in any desired and suitable manner. For example, the steps and functions of the invention described herein can be implemented in hardware or software as needed. Thus, for example, unless otherwise indicated, the various circuits, functional elements, stages, and “devices” of the invention described herein may include suitable processors, controllers, functional units, circuits, processing logic, microprocessors, etc., operable to perform various steps or functions, etc., such as suitable dedicated hardware elements (processing circuits / circuit systems) and / or programmable hardware elements (processing circuits / circuit systems) that can be programmed to operate in a desired manner.

[0174] Specifically, it can be anticipated that the encoder can be implemented in hardware or software as needed. Thus, for example, the encoder may include suitable processors, controllers, functional units, (encoding) circuits / circuits, processing logic, microprocessor settings, etc., operable to perform the various encoding steps or functions described herein, such as suitable dedicated hardware elements (processing circuits / circuit systems) and / or programmable hardware elements (processing circuits / circuit systems) that can be programmed to operate in a desired manner.

[0175] Similarly, decoders can be implemented in hardware or software as needed. Thus, for example, a decoder may include suitable processors, controllers, functional units, (decoding) circuits / circuit systems, processing logic, microprocessor devices, etc., operable to perform the various encoding steps or functions described herein, such as suitable dedicated hardware elements (processing circuits / circuit systems) and / or programmable hardware elements (processing circuits / circuit systems) that can be programmed to work in a desired manner.

[0176] The various steps or functions of the invention described herein can be copied and / or executed in parallel on a given processor. Similarly, processing levels can share processing circuitry / systems, etc., if desired.

[0177] Subject to any hardware required to perform the specific steps or functions described above, the system may additionally include data processing equipment and / or any or more conventional functional units included in the system.

[0178] Therefore, various data processing levels can be implemented as desired and in any suitable manner, and can each perform any desired and suitable function. Similarly, various types of data can be defined and stored in any suitable and desired manner.

[0179] Those skilled in the art will also recognize that all described embodiments of the invention herein may and do include any one or more of the features described herein.

[0180] The method according to the invention described herein can be implemented at least in part using software, such as a computer program. Therefore, further embodiments of the invention described herein include computer software particularly suitable for executing the method described herein when installed on a data processor. The computer program elements include computer program code portions that, when run on the data processor, are used to execute the method described herein, and a computer program including code that, when run on the data processor, is adapted to perform all steps of the method or the method described herein. The data processor may be a microprocessor system, a programmable FPGA (Field-Programmable Gate Array), etc.

[0181] The invention described herein also extends to computer software carriers that include software that, when used to operate a data processing device or system including a data processor, causes the device or system to perform the steps of the methods of the invention described herein together with the data processor. Such computer software carriers can be physical storage media such as ROM chips, CD-ROMs, RAM, flash memory, or hard disks, or signals such as wired electronic signals, optical signals, or radio signals such as those to satellites.

[0182] It will also be recognized that not all steps of the methods of the invention described herein need to be performed by computer software, and therefore, in other embodiments, computer software and such software mounted on a computer software carrier for performing at least one step of the methods set forth herein are included.

[0183] Therefore, the invention described herein can be suitably implemented as a computer program product for use with a computer system. Such implementation may include a series of computer-readable instructions fixed on a tangible, non-transitory medium, such as a computer-readable medium like a floppy disk, CD, DVD, ROM, RAM, flash memory, or hard disk. It may also include a series of computer-readable instructions that can be transmitted to the computer system via a modem or other interface device through a tangible medium (including, but not limited to, optical or analog communication lines) or intangibly using wireless technologies (including, but not limited to, microwave, infrared, or other transmission technologies). This series of computer-readable instructions performs all or part of the functions previously described herein.

[0184] Those skilled in the art will understand that such computer-readable instructions can be written in a variety of programming languages ​​for use in many computer architectures or operating systems. Furthermore, any current or future storage invention can be used to store such instructions, including but not limited to semiconductor, magnetic, or optical, or any current or future communication invention can be used to transmit such instructions, including but not limited to optical, infrared, or microwave. It is contemplated that such computer program products can be distributed as removable media with accompanying printed or electronic documentation, such as shrink-wrapped software, pre-installed computer systems, for example, on system ROM or a fixed disk, or distributed from servers or electronic bulletin boards via networks (e.g., the Internet or the World Wide Web). Attached Figure Description

[0185] Various embodiments of the invention described herein will now be described by way of example only and with reference to the accompanying drawings, wherein:

[0186] Figure 1 An example of a data processing system in which embodiments of the invention described herein may be implemented is illustrated schematically;

[0187] Figure 2 This is a flowchart illustrating an encoding scheme according to an implementation method.

[0188] Figure 3 An example of compressed data packets generated according to an implementation method is illustrated schematically.

[0189] Figure 4 This is a flowchart illustrating a decoding scheme according to an implementation method. And...

[0190] Figure 5 A comparison is shown between the implementation method and the coding scheme that does not use the technology of the present invention.

[0191] The accompanying drawings illustrate elements of a data processing apparatus and system relating to embodiments of the invention described herein. Where appropriate, similar reference numerals in the drawings denote similar elements.

[0192] As those skilled in the art will understand, there may be other elements of the data processing apparatus and system not illustrated in the figures. It should also be noted that the figures are merely schematic, and, for example, even if elements are schematically shown as separate elements in the figures, they may actually share essential hardware circuitry. Or conversely, elements shown as sharing essential hardware circuitry may actually comprise separate elements. Detailed Implementation

[0193] Figure 1An embodiment of a data processing system 100 is illustrated schematically, which can provide and use data arrays encoded in the manner of the present invention, such as image or video frames for display. It should be understood that... Figure 1 Only one example of a system suitable for use with the present invention has been shown, and various other configurations are also possible.

[0194] In this embodiment, system 100 includes a data processing device in the form of a system-on-a-chip (SoC) 102. System 100 also includes off-chip (main) memory 116, a display device 118, and a video camera 120.

[0195] The SoC 102 includes a central processing unit (CPU) 104, a graphics processing unit (GPU) 106, an image / video processor 108, a display controller 110, an interconnect 112, and a memory controller 114.

[0196] like Figure 1 As shown, CPU 104, GPU 106, image / video processor 108, and display controller 110 communicate with each other via interconnect 112, and communicate with memory 116 via interconnect 112 and memory controller 114. Display controller 110 also communicates with display device 118. Video camera 120 also communicates with SoC 102 via interconnect 112.

[0197] In one implementation, the image / video processor 108 reads image data from memory 116, encodes the image data using a suitable encoder circuit 122 (e.g., in the manner of the present invention), and then outputs the encoded image data, for example, for storage in memory 116 or for streaming to another device. The encoded image data can be retrieved and decoded later, for example by a suitable decoder circuit 124 of the image / video processor 108, or received and decoded by another device. Although the image / video processor 108 is shown as having separate encoder circuit 122 and decoder circuit 124, it should be understood that this is merely illustrative and they can form part of a shared encoding / decoding circuit. The decoded image data can then be output, for example, via display controller 110, to display device 118 or other devices for display.

[0198] Therefore, within such a data processing system, it may be desirable to compress the image data being processed at various points. Furthermore, to facilitate higher throughput, such as for high-bandwidth media applications, it may be desirable to guarantee a given bandwidth (bit rate).

[0199] In this embodiment, the following encoding scheme is used, which encodes image data into data blocks of fixed size to ensure the desired bit rate. This encoding scheme is block-based. When encoding image data blocks, at least some image data is encoded using the "original" encoding scheme, wherein multiple most significant bits representing data values ​​(e.g., brightness values) at different sampling points within the image are added "as is" to the data blocks until the data blocks are full.

[0200] This "primitive" encoding scheme therefore adds as many most significant bits as possible to the data elements until the desired size is reached. However, depending on the expected size of the data blocks, this can result in severely truncated data values. For example, in common examples, it is possible to add only one or two most significant bits from each sample point to the data block, while all other numerous less significant bits are discarded.

[0201] This can lead to artifacts in the reconstructed image data. For example, when the data values ​​represent brightness values, relatively coarse steps in brightness may appear across the entire image or between different images in a sequence of images.

[0202] This implementation typically mitigates this situation and provides improved granularity between blocks by selecting an appropriate rounding modifier value that can be applied block by block to better reproduce the initial data value of that block.

[0203] Figure 2 This is a flowchart illustrating the encoding process for encoding data element blocks according to this embodiment.

[0204] As shown, the process begins with a set of input samples for the block (step 201). As described above, the encoding of the present invention can store only a subset of the multiple most significant bits of a data element. Therefore, the initial data values ​​are truncated (step 203), and only these truncated data values ​​are stored for the data elements in the data block (step 204). Thus, at this stage, the multiple less significant bits in the data block will not be stored and are therefore discarded.

[0205] However, it is used to determine the rounding mode to be used for the data element block before several lower significant bits are discarded (step 202). The rounding mode cost for the block can therefore be calculated as follows. First, the representation value for the data element block is determined. For example, this could include the sum of data values ​​(although any other suitable representation value could be used).

[0206] Then, for each available rounding mode, the corresponding representation value for the data element block is determined. This is done by calculating the representation value of the data element block in the same way as the first representation value, but instead of using the initial data value, using a modified data value obtained by adding the corresponding rounding bit sequence for that rounding mode to the truncated data value.

[0207] Then, the first representation value determined based on the initial data value is compared with the representation values ​​obtained for each available rounding mode to determine which rounding mode best reproduces the initial data.

[0208] Encoded data packets can then be generated, including a suitable header and truncation values ​​for the data elements in the block, the header identifying the rounding mode to be used for the block (step 204). The encoded data packets can then be provided for output (step 205), for example, and then decoded.

[0209] Figure 3 An example of an encoded data packet generated according to this embodiment is shown. Specifically, such as Figure 3 As shown, the data grouping includes a header 300 indicating the selected rounding mode, for example, using the rounding_mode field 302. For example, there might be four available rounding modes, so two bits could be used to indicate this. However, generally there might be more or fewer available rounding modes. Then, multiple most significant bits applied to the encoded data elements are stored in the grouping in a specific order, for example, in the sample most significant bit field 305. Figure 3 The load is marked as 304.

[0210] The header can also include any other suitable information as needed. For example, when the encoder is able to select between different encoding modes, the encoding mode can be indicated at the beginning of the header using an appropriate data structure (e.g., when using the “raw” encoding scheme described above, it can be signaled with four “0000” bits, such as using the is_raw? field 301, although of course any other suitable signaling can be used).

[0211] The header can also indicate the nature of the data, such as whether the data represents grayscale or color (RGBA) data, for example, by using other fields 303.

[0212] Then, Figure 4 The corresponding decoding process is illustrated. The decoder receives... Figure 3The data is encoded into the type shown (step 401). The decoder is then able to read the header to determine the rounding mode to be used (and optionally also the encoding scheme, etc.), and then extract the most significant bits of the encoded data elements (step 402). The data elements (samples) can then be reconstructed using the most significant bits of the data elements (samples) (step 403), and then the indicated rounding mode bit pattern is added accordingly (step 404) to give the output sample (step 405).

[0213] Now refer to Figure 5 Description example. In Figure 5 In the diagram, the left column contains the input blocks. The middle column contains the output blocks generated using the existing scheme. The right column contains the output blocks generated using the new scheme. Specifically, Figure 5 A set of input blocks is shown, each block consisting of four samples with associated 8-bit data values. Specifically, Figure 5 This example represents a brightness fade-in, where the input block gradually fades in from 0 to 100 in steps of 20.

[0214] One scheme for encoding this data is to send (only) the two most significant bits, and then set the bit value of the next bit after the least significant bit to "1" and the remaining least significant bits to "0". It should be understood that each bit value in the bit sequence contributes half of the preceding (higher) bit value. Therefore, this scheme has the effect of indiscriminately rounding the encoded data value up by a certain amount.

[0215] Therefore, for Figure 5 In the example shown, for the first input block with a value of "0" (00000000), only the first two digits (00xxxxxx) will be sent, and the rounding scheme will then give an output value of "32" (00...). 10 (0000). For the second, third, and fourth input blocks, the same output value "32" will also be given, with input values ​​of "20", "40", and "60" respectively, because the first two (sent) bits are "0" in each case. In fact, the output only changes when one of the first two most significant bits changes; for example, for the fifth input block with an input value of "80" (01010000), the output will jump to "96" when the above rounding scheme is applied.

[0216] Therefore, using this scheme results in the loss of the desired smooth fade-in effect, and the data values ​​moving along the block sequence only have a single large step.

[0217] This implementation allows for finer control over the available output values, thus enabling the reproduction of smoother variations. For example, in this example, four rounding modes are available, each associated with a single corresponding sequence of rounding bits, as follows:

[0218] Rounding mode Rounding sequence 00 0101 01 0111 10 1001 11 1011

[0219] Therefore, the rounding mode to be used for the block can be represented using two bits from the four available rounding modes. In this example, each rounding mode includes a four-bit sequence to be added immediately after the first two sent bits (e.g., in the case of sending two MSBs, a rounding bit sequence could be added to the next four bits, such as 00xxxxxx). Of course, any suitable set of rounding modes can be used, and each rounding mode can be associated with any suitable set of one or more rounding bit sequences.

[0220] Similarly, while four rounding modes are used in this case (because this allows for a good balance between having finer granularity in the output data without using too many rounding mode bits to indicate the rounding mode), in principle, any number of rounding modes can be used as desired. For example, in the simplest case, two rounding modes can be used, so only one bit can be used to specify the rounding mode. Alternatively, more rounding modes (e.g., octal, hexadecimal, etc.) can be used to provide finer granularity, but with associated bit and encoding costs.

[0221] When applying the rounding scheme of this embodiment, for each data element block, it is determined which rounding mode will give the best reproduction of the initial data value, and then the rounding mode is applied to the decoder when generating the decoded data value.

[0222] For example, for the first input block, whose data value is "0", the smallest round modifier (i.e., mode "00") is selected because it best represents the initial value, so the output is "20" (000101000), where the two bits sent are modified by adding modifier 0.0101 (00xxxxxx).

[0223] For the second input block, whose data value is "20", the same rounding modifier is used again to give an output value of "20".

[0224] For the third input with a data value of “40”, the fourth mode (“11”) was found to give the best data representation because the two bits sent (00xxxxxx) were modified using modifier 0.1011 to give the value “44” (00101100).

[0225] Similarly, the fourth input with a value of "60" was output as "44", the fifth input with a value of "80" was output as "84", and the last input block in the sequence was output as "100".

[0226] Therefore, it can be seen that the output obtained using this embodiment matches the initial data values ​​in the input sequence well and allows for smooth changes in the decoded data values.

[0227] exist Figure 5 In the example, for illustrative purposes, all samples in each input block have the same data (brightness) value. However, this is not necessarily the case (and usually isn't), so the brightness of the entire block can often vary. However, it will be appreciated that a given block only represents a portion of the overall image, and it can be expected that adjacent samples within a single block can have similar (if not identical) brightness levels. Accordingly, the block size can be adjusted such that the above-mentioned situation occurs, for example, so that all samples within the block have similar brightness values. Therefore, applying a rounding modifier at the block level can significantly improve the decoded output, since a "common" representation for that block can typically be computed, which can be considered as an "average" of the data values ​​within the block.

[0228] Generally, as described below, the algorithm for selecting the rounding mode therefore selects the optimal rounding mode by calculating a reference brightness representing the brightness of the initial data array and the corresponding brightness for each rounding mode. Then, the rounding mode with the brightness closest to the reference is selected.

[0229] reference_brightness=sum(pow(sample[i],2))

[0230] rm0_brightness=sum(pow(truncate(sample[i])+rm0[i],2))

[0231] rm1_brightness=sum(pow(truncate(sample[i])+rm1[i],2))

[0232] rm2_brightness=sum(pow(truncate(sample[i])+rm2[i],2))

[0233] rm3_brightness=sum(pow(truncate(sample[i])+rm3[i],2))

[0234] It should be understood that even if the same rounding bit sequence is added for every sample, the rounding bit sequence can be added to different bit positions for different samples, depending on how many most significant bits were stored when the sample was truncated. Therefore, in the above equation, the rounding bit sequence for each rounding mode is also indexed according to the sample.

[0235] Furthermore, it will be understood that the reference brightness does not need to be calculated using all sampling points, but can be determined simply by using, for example, a random subset of these. Of course, various other settings are also possible.

[0236] It should also be noted that in this example, the reference brightness is calculated as the sum of the squared values ​​of the brightness for each sample. This is because brightness does not increase linearly (e.g., from 0 (black) to 256 (white) (but is described by a gamma curve involving powers of 2.2). Therefore, using the sum of squares has been found to provide a better reference value for brightness.

[0237] However, if lower precision is acceptable, the brightness relationship can be approximated by the direct sum of the data values, for example, by determining the brightness value as follows:

[0238] reference_brightness=sum(sample[i])

[0239] rm0_brightness=sum((truncate(sample[i])+rm0[i]))

[0240] rm1_brightness=sum(truncate(sample[i])+rm1[i]))

[0241] rm2_brightness=sum(truncate(sample[i])+rm2[i]))

[0242] rm3_brightness=sum(truncate(sample[i])+rm3[i]))

[0243] Furthermore, while examples of brightness values ​​have been given above, it should be understood that the invention can be used in conjunction with any suitable data value as needed. In such cases, different reference values ​​may be appropriate, for example, depending on what the data value represents. For instance, when the attribute represented by the data value is indeed scaled linearly, a standard linear sum may be more suitable (e.g., as shown in the second example given above).

[0244] Therefore, any appropriate and desired function of the data values ​​within the block can be used to calculate and use the benchmark value. For example, in general, any other suitable benchmark value that typically represents the benchmark or average of the initial data array can be used, depending on the nature of the data values ​​to be encoded.

[0245] Therefore, in general, for a given block of data, the representation value for the reference block and for the possible rounding modes can be determined by performing any appropriate and desired function on the data values ​​in question (and then selecting the rounding mode that most approximates the reference value), as follows:

[0246] reference_value=function(sample[i])

[0247] rm0_value=function((truncate(sample[i])+rm0[i]))

[0248] rm1_value=function(truncate(sample[i])+rm1[i]))

[0249] rm2_value=function(truncate(sample[i])+rm2[i]))

[0250] rm3_value=function(truncate(sample[i])+rm3[i]))

[0251] The foregoing detailed description has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention described herein to the specific forms disclosed. Many modifications and variations can be made in light of the above teachings. The described embodiments were chosen to best explain the principles of the invention described herein and its practical application, thereby enabling others skilled in the art to best utilize the invention described herein in various embodiments and various modifications to the invention suited to the intended particular use. The scope is intended to be defined by the appended claims.

Claims

1. A method of processing an array of data elements, each data element having an associated data value represented by a respective bit sequence, the method comprising the steps of: for a block of data elements in the array of data elements: representing the data value of each of the data elements in the block of data elements as a truncated data value using a subset of one or more most significant bits of the respective bit sequence of the data value of the data element; and selecting a rounding mode to be applied when decoding the block of data elements from a plurality of available rounding modes, wherein each rounding mode is associated with one or more rounding bit sequences, and wherein the rounding mode to be applied when decoding a block of data elements is selected by: determining a first representation value for the block of data elements based on the data values of the data elements in the block of data elements; determining a corresponding representation value for the block of data elements for at least one of the plurality of available rounding modes, wherein the representation value for the rounding mode is determined based on the truncated data values of the data elements in the block of data elements and a respective rounding bit sequence associated with the rounding mode; comparing the first representation value determined based on the data values for the block of data elements with the corresponding representation value determined for the at least one of the plurality of available rounding modes; determining which of the plurality of available rounding modes gives a representation value closest to the first representation value; and selecting the rounding mode of the plurality of available rounding modes determined to give a representation value closest to the first representation value determined for the block of data elements as the rounding mode to be applied when decoding the block of data elements; the method further comprising the steps of: encoding the block of data elements by storing the truncated data values of the data elements as a data packet representing the block of data elements and providing the data packet with data indicating a rounding mode to be used when decoding the data packet; and when it is desired to use the data values of one or more data elements in the block of data elements that have been encoded into the data packet, the method subsequently comprising the steps of: passing the data packet to a decoder circuit with the data indicating the rounding mode to be used when decoding the data packet for decoding; extracting the truncated data values of the one or more data elements; and adding a rounding bit sequence associated with the indicated rounding mode to the truncated data values to obtain decoded data values for the one or more data elements in the block of data elements.

2. The method of claim 1, wherein, the block of data elements is encoded as a fixed size data packet.

3. The method of claim 1 or 2, wherein, the selected rounding mode is included in a header of the data packet.

4. The method of claim 1 or 2, wherein, A plurality of different encoding modes can be used, one of the encoding modes comprising representing data values of individual data elements in the block of data elements as truncated data values using a subset of one or more most significant bits of the respective bit sequence; and encoding the block of data elements by adding the truncated data values of the data elements to a data packet; The method comprises: analysing the array of data elements to determine which of a plurality of available encoding modes should be used to encode the array of data elements; and signalling the determined encoding mode in a header of the data packet.

5. The method of claim 1 or 2, wherein, The array of data elements represents an array of image data.

6. The method of claim 1 or 2, wherein, The step of determining the first representation value comprises summing squares of the data values of at least some of the data elements within the block of data elements, and wherein the step of determining a representation value for a respective rounding mode comprises summing squares of modified data values of at least some of the data elements within the block of data elements, wherein the modified data values of the data elements are obtained by adding a respective rounding bit sequence of the rounding mode to the truncated data values of the data elements.

7. A method of encoding an array of data elements, individual data elements having an associated data value represented by a respective bit sequence, the method comprising the steps of: for a block of data elements in the array of data elements: representing data values of individual ones of the data elements in the block of data elements as truncated data values using a subset of one or more most significant bits of the respective bit sequence of the data value of the data element; and selecting a rounding mode to be applied when decoding the block of data elements from a plurality of available rounding modes, wherein individual rounding modes are associated with one or more rounding bit sequences, and wherein the rounding mode to be applied when decoding a block of data elements is selected by: determining a first representation value for the block of data elements based on data values of the data elements in the block of data elements; determining a corresponding representation value for the block of data elements for at least one of the plurality of available rounding modes, wherein the representation value for the rounding mode is determined based on the truncated data values of the data elements in the block of data elements and a respective rounding bit sequence associated with the rounding mode; comparing the first representation value determined based on the data values for the block of data elements to the corresponding representation value determined for the at least one of the plurality of available rounding modes; determining which of the plurality of available rounding modes gives a representation value closest to the first representation value; and selecting the rounding mode of the plurality of available rounding modes determined to give a representation value closest to the first representation value determined for the block of data elements as the rounding mode to be applied when decoding the block of data elements; the method further comprising the steps of: The data element block is encoded by storing the truncated data values of the data elements as a data packet representing the data element block and providing the data packet with data indicating a rounding mode to be used when decoding the data packet.

8. A method of decoding a data stream comprising one or more data packets, each data packet encoding a block of data elements in an array of data elements, wherein, Each data element in the data element block has an associated encoded data value, and wherein the data packet is provided with data indicating one of a plurality of available rounding modes to be used when decoding the data packet, each rounding mode being associated with one or more rounding bit sequences, the method comprising the steps of: when a data value of one or more data elements in a data element block encoded into a data packet is desired: reading the data packet to extract the encoded data values of one or more data elements in the data element block; identifying, based on the data indicating one of the plurality of available rounding modes, the rounding mode to be used when decoding the data packet; and adding a rounding bit sequence associated with the identified rounding mode to the encoded data values to obtain decoded data values of the one or more data elements in the data element block.

9. A data processing system for processing an array of data elements, each data element having an associated data value represented by a respective bit sequence, the data processing system comprising: an encoder circuit configured to: for a block of data elements in the array of data elements: represent the data values of each data element in the block of data elements as a truncated data value using a subset of one or more most significant bits of the respective bit sequence of the data value of the data element; and select, from a plurality of available rounding modes, a rounding mode to be applied when decoding the block of data elements, each rounding mode being associated with one or more rounding bit sequences, wherein the selection of a rounding mode is made by a rounding mode selection circuit configured to: determine, based on the data values of the data elements in the block of data elements, a first representation value for the block of data elements; determine, for at least one rounding mode of the plurality of available rounding modes, a corresponding representation value for the block of data elements, wherein the representation value for the rounding mode is determined based on the truncated data values of the data elements in the block of data elements and a respective rounding bit sequence associated with the rounding mode; compare the first representation value for the block of data elements determined based on the data values to the corresponding representation values determined for the at least one rounding mode of the plurality of available rounding modes; determine which rounding mode of the plurality of available rounding modes gives a representation value closest to the first representation value; and select, as the rounding mode to be applied when decoding the block of data elements, the rounding mode of the plurality of available rounding modes determined to give a representation value closest to the first representation value determined for the block of data elements; the encoder circuit is further configured to: by storing a truncated data value of the data element as a data packet representing the block of data elements and providing the data packet with data indicating a rounding mode to be used when decoding the data packet; the data processing system further comprises decoder circuitry configured to: when it is desired to use data values of one or more data elements of the block of data elements that have been encoded into a data packet: extract the truncated data values of the one or more data elements of the block of data elements; and add a rounding bit sequence associated with the indicated rounding mode to the truncated data values to obtain decoded data values of one or more data elements of the block of data elements.

10. The data processing system of claim 9, wherein, the encoder circuitry is configured to encode the block of data elements as a fixed size data packet.

11. The data processing system of claim 9 or 10, wherein, the encoder circuitry is configured to include the selected rounding mode in a header of the data packet.

12. The data processing system of claim 9 or 10, wherein, a plurality of different encoding modes are available, one of the encoding modes comprising: representing data values of individual data elements of the block of data elements as truncated data values using a subset of one or more most significant bits of the respective bit sequences; and the encoder circuitry is configured to analyse the array of data elements to determine which of a plurality of available encoding modes should be used to encode the array of data elements; and 13. The data processing system of claim 9 or 10, wherein, signalling the determined encoding mode in a header of the data packet.

14. The data processing system of claim 9 or 10, wherein, the array of data elements represents an array of image data. the step of determining the first representation value comprises summing squares of data values of at least some of the data elements within the block of data elements, and wherein the step of determining a representation value of a respective rounding mode comprises summing squares of modified data values of at least some of the data elements within the block of data elements, wherein the modified data values of the data elements are obtained by adding a respective rounding bit sequence of the rounding mode to the truncated data values of the data elements.

15. An encoder for encoding an array of data elements, individual data elements having an associated data value represented by a respective bit sequence, the encoder comprising: encoder circuitry configured to: for a block of data elements in the array of data elements: represent data values of individual data elements of the block of data elements as truncated data values using a subset of one or more most significant bits of the respective bit sequences of the data values of the data elements; and select a rounding mode to be applied when decoding the block of data elements from a plurality of available rounding modes, individual rounding modes being associated with one or more rounding bit sequences, wherein the selection of the rounding mode is made by rounding mode selection circuitry configured to: determine a first representation value of the block of data elements based on data values of the data elements in the block of data elements; determining a corresponding representation value for the block of data elements for each of the plurality of available rounding modes, wherein the representation value for the rounding mode is determined based on the truncated data values of the data elements in the block of data elements and a respective sequence of rounding bits associated with the rounding mode; comparing the first representation value determined for the block of data elements with the corresponding representation values determined for the at least one of the plurality of available rounding modes; determining which of the plurality of available rounding modes gives a representation value that is closest to the first representation value; and selecting the rounding mode of the plurality of available rounding modes that is determined to give a representation value that is closest to the first representation value determined for the block of data elements as the rounding mode to be applied when decoding the block of data elements; the encoding circuitry is further configured to: encode the block of data elements by storing the truncated data values of the data elements as a data packet representing the block of data elements and providing the data packet with data indicating a rounding mode to be used when decoding the data packet.

16. A decoder for decoding a data stream comprising one or more data packets, each data packet encoding a block of data elements in an array of data elements, wherein, each data element in the block of data elements has an associated encoded data value, and wherein the data packet is provided with data indicating one of a plurality of available rounding modes to be used when decoding the data packet, each rounding mode being associated with one or more sequences of rounding bits, the decoder comprising: when a data value of one or more data elements in a block of data elements encoded into a data packet is desired: reading the data packet to extract the encoded data values of the one or more data elements in the block of data elements; identifying the rounding mode to be used when decoding the data packet based on the data indicating one of the plurality of available rounding modes; and adding a sequence of rounding bits associated with the identified rounding mode to the encoded data values to obtain decoded data values of the one or more data elements in the block of data elements.

17. A computer readable storage medium storing a computer program which, when executed on a data processor, performs the method of claim 1.

Images