Decoding method, decoding unit, and non-transitory computer-readable storage medium

Abstract

The present invention relates to a decoding method, a decoding unit and a non-transitory computer-readable storage medium. A compressed data structure encodes a set of Haar coefficients of a 2×2 pixel quaternary. A set of Haar coefficients includes a plurality of differential coefficients and an average coefficient. A first portion of the compressed data structure encodes the differential coefficients of the 2×2 pixel quaternary, including: a first group of bits indicating the order of magnitudes of the differential coefficients, a second group of bits indicating the sign and exponent of each of one or more of the non-zero differential coefficients. A second portion of the compressed data structure encodes the average coefficient of the 2×2 pixel quaternary. The sign and exponent of the non-zero differential coefficients are determined using the first portion. The representation of the average coefficient is determined using the second portion. The result of a weighted sum of the differential coefficients and the average coefficient is determined using the following operations: the sign and exponent of one or more of the non-zero differential coefficients, the representation of the average coefficient, and the corresponding weights of the differential coefficients. A decoded value is determined using the determined result. The decoded value is output.

Description

Technical Field

[0001] The present disclosure relates to techniques for encoding and / or decoding blocks of pixels (eg, textures or other images) within a computer graphics system. Background Art

[0002] In computer graphics, texturing is often used to add surface detail to objects within a scene to be rendered. Textures are often stored as images that can be accessed to return the color value of the pixel being processed. To obtain the texture color value of a pixel, the values ​​of multiple texels of the texture can be sampled, and the sampled texel values ​​can then be filtered to obtain the final texture value of the pixel.

[0003] Typically, because textures are projected onto 3D geometry within an image, a single pixel of an image being rendered may not be accurately mapped to a single texel of the texture. For example, for texels depicting a relatively near view of the texture, each pixel may be associated with a footprint that maps to a relatively small area of ​​the texture (e.g., covering zero or one texel), while for texels depicting a relatively far view of the texture, each pixel may be associated with a footprint that maps to a relatively larger area of ​​the texture (e.g., covering many (e.g., two or more) texels).

[0004] In some cases, anisotropic texture filtering may be performed. Anisotropic texture filtering recognizes that the sampling rate of the texture is a function of the direction of travel in screen space, and the sampling rate depends on the mapping from screen space (where the image is depicted and pixel coordinates are defined) to texture space (where the texture is depicted and texel coordinates are defined). In other words, the optimal texture sampling rate may be different along different directions of travel or axes in screen space. In this case, the mapping between screen space and texture space can be said to be anisotropic. An example of anisotropic mapping is the 3D projection of a planar textured surface near the horizontal plane, or any other case where a texture is applied to a surface in the scene that is at a significantly large angle to the viewpoint.

[0005] For anisotropic texture mapping, the sampling kernel in texture space that is mapped to a pixel in screen space is elongated along a specific axis in texture space, the direction of which depends on the mapping between screen space and texture space. This is done in Figure 1 As schematically illustrated in Figure 1An image 100 formed by pixels having coordinates defined in image space, and a texture 102 formed by texels having coordinates defined in texel space are shown. Image 100 includes an object 104 whose surface details are specified by texture 102, i.e., texture 102 is mapped to the surface of object 104. For clarity, the remaining objects within image 100 have been omitted. Object 104 is at an oblique viewing angle within image 100. As described above, if a texture is applied to a geometric figure that is oblique to the viewing direction, the isotropic coverage area of ​​the pixel in image space is mapped to an anisotropic coverage area in texture space. Therefore, in the case of mapping to texture space, the shape of the coverage area of ​​the texture mapped pixel within image 100 is not maintained. Number 106 represents a circular pixel coverage area in image space, and number 108 represents the corresponding pixel coverage area in texel space. It can be seen that the coverage area has been extended in texture space to form an ellipse, so that it is anisotropic. In general, a mapping of a pixel with a circular footprint in image space to texture space can be approximated by an ellipse, as long as the texture mapping itself can be approximated by an affine mapping at the pixel origin. Here, it should be noted that the term "footprint" used with respect to a pixel does not necessarily refer to the physical shape of the pixel in a display device, but may refer to the area of ​​the pixel filter used to construct the final pixel value of the pixel. Therefore, it can be said that the footprint of a pixel in texture space is equivalent to the sampling kernel of the pixel in texture space; that is, the footprint in texture space identifies the texels to be sampled and filtered to form the texture color value of the pixel.

[0006] As described above, textures are typically represented as images stored in memory, so that when a graphics processing unit intends to apply the texture to geometry in a scene being rendered, a portion of the texture may be retrieved from memory and filtered in order to determine the texture value on the geometry. In order to reduce the amount of data that needs to be stored in and transferred back and forth from memory (i.e., "memory bandwidth"), the texture data may be encoded (i.e., compressed) before it is stored in memory. In this case, the graphics processing unit may obtain the encoded texture data and then decode (i.e., decompress) the encoded texture data before applying it to the geometry.

[0007] It is generally accepted that the texture encoding and decoding process is a lossy process, i.e., some information may be lost when a texture is encoded and then decoded. By allowing the texture encoding and decoding process to be lossy, textures can be compressed to a greater extent, thereby reducing memory storage and memory bandwidth requirements. The encoding / decoding process typically has three competing goals: (i) it is beneficial to reduce the amount of data used to represent the encoded texture, (ii) it is beneficial to minimize any artifacts introduced by the encoding and decoding process, and (iii) it is beneficial for the encoding and decoding process to be cheap to implement (e.g., in terms of processing latency, power consumption, and / or silicon area). Typically, improving one of these three factors will compromise one or two of the other factors. These costs can be compounded when combined with texture filtering. For example, high-quality texture filtering (such as minimizing sampling artifacts) is typically expensive in terms of memory bandwidth and throughput requirements.

[0008] Another case where encoding and / or decoding a pixel block is useful is for compressing an image, such as an image that has been rendered by a graphics processing unit (GPU). This process may be referred to as frame buffer compression and / or frame buffer decompression. Compressing images reduces the amount of data used to represent these images, which is useful in reducing the amount of storage used to store images and reducing the amount of data transferred from a processing unit (e.g., GPU) to a memory. This can reduce latency, power consumption, and silicon area. For frame buffer compression / decompression techniques, filtering is not applied, and typically the decompressed value is intended to be similar (or identical) to the original value before compression. For example, if the compression / decompression technique is lossless, the decompressed value will be the same as the original value before compression, and if the compression / decompression technique is lossy, the decompressed value may not be identical to the original value before compression (but they are approximate representations of the original value). Summary of the invention

[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0010] A method is provided for decoding a compressed data structure to determine a decoded value, wherein the compressed data structure encodes a set of Haar coefficients for a 2×2 pixel quad of a pixel block, wherein the set of Haar coefficients includes a plurality of differential coefficients and an average coefficient,

[0011] wherein the first portion of the compressed data structure encodes the differential coefficients of the 2×2 pixel quad and includes: (i) a first set of bits indicating an order of magnitudes of the differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero,

[0012] wherein the second portion of the compressed data structure encodes the average coefficient of the 2×2 pixel quad,

[0013] The method comprises:

[0014] determining signs and exponents of the one or more of the differential coefficients that are non-zero using the first portion of the compressed data structure;

[0015] determining a representation of the average coefficient using the second portion of the compressed data structure;

[0016] determining a result of a weighted sum of the differential coefficients and the average coefficients for the 2×2 pixel quadruple using: (i) the determined signs and exponents of the one or more of the differential coefficients that are non-zero, (ii) the determined representation of the average coefficient, and (iii) the corresponding weights of the differential coefficients;

[0017] using the determined result to determine the decoded value; and

[0018] Outputs the determined decoded value.

[0019] The result of determining a weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad may include:

[0020] performing a binary shift of said weights according to said determined exponents of corresponding one or more of said differential coefficients that are non-zero; and

[0021] The determined representation of the average coefficient and the shifted weight are summed according to the determined signs of the corresponding one or more of the differential coefficients that are non-zero.

[0022] The determining of the result of the weighted sum of the differential coefficients and the average coefficients for the 2x2 pixel quad may not involve performing any multiplication or division operations.

[0023] Determining the result of the weighted sum of the differential coefficients and the average coefficients of the 2x2 pixel quadruplet may include adding a gamma correction term.

[0024] The respective weights of the differential coefficients may be predetermined, and wherein the using the determined result to determine the decoded value may include using the determined result as the decoded value representing a value at a pixel position of the 2×2 pixel quad.

[0025] The step of determining the result of the weighted sum of the differential coefficient and the average coefficient of the 2×2 pixel quadruplet and the step of determining the decoded value using the determined result can be performed four times for the corresponding four pixels of the 2×2 pixel quadruplet, wherein the predetermined weight of the differential coefficient can be different each time the steps are performed, thereby determining four decoded values ​​representing the values ​​at the four pixel positions of the 2×2 pixel quadruplet.

[0026] The method may further comprise determining the respective weights of the differential coefficients.

[0027] The weights may be determined based on one or more of the following:

[0028] a position of the decoded value to be determined relative to the 2×2 pixel quad;

[0029] an indication of a level of detail to be provided by the decoded value; and

[0030] An indication of the amount and / or direction of anisotropy to be provided by the decoded value.

[0031] The pixel block may be a 4×4 pixel block comprising four 2×2 pixel quads, wherein for each of the four 2×2 pixel quads, a result of a weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quads may be determined, and wherein the four determined results of the corresponding four 2×2 pixel quads may be combined to determine a decoded value of the pixel block.

[0032] The four determined results for the corresponding four 2x2 pixel quadruplets may be combined by performing bilinear interpolation on the four determined results to determine the decoded value of the pixel block.

[0033] If the location of the decoded value to be determined is within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quads in the pixel block, the decoded value of the pixel block can be equal to the determined result of the single 2×2 pixel quad.

[0034] If the location of the decoded value to be determined is not within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quad in the pixel block, the decoded value of the pixel block can be the result of a weighted sum of two or more of the determined results of the 2×2 pixel quad.

[0035] The determining the corresponding weights of the differential coefficients may include:

[0036] determining initial weights based on an indication of a level of detail, an indication of an amount of anisotropy, and an indication of a direction of anisotropy provided by the decoded values; and

[0037] The determined initial weights are multiplied by corresponding transfer functions to determine the weights of the differential coefficients, wherein each of the transfer functions is based on a position of the decoded value to be determined relative to the 2×2 pixel quad, and wherein each of the transfer functions: (i) is zero at a center of the 2×2 pixel quad, (ii) has a magnitude of one at each of the pixel positions in the 2×2 pixel quad, and (iii) is extrapolated to an edge of the region represented by the 2×2 pixel quad.

[0038] Outside the edge of the region represented by the 2x2 pixel quad, the transfer function may be clamped to the value of the transfer function at the nearest edge of the region.

[0039] Pixels within the 4×4 pixel block may be selectively mirrored about one, none, or both of: (i) a horizontal axis passing through a center of the pixel block, and (ii) a vertical axis passing through the center of the pixel block, so that the location for which the decoded value is to be determined is within a particular quadrant of the pixel block.

[0040] The first set of bits may further indicate how many of the differential coefficients are non-zero.

[0041] The second set of bits may include:

[0042] a first portion indicating a sign of each of the one or more of the differential coefficients that are non-zero; and

[0043] A second portion indicates an exponent of each of the one or more of the differential coefficients that are non-zero.

[0044] If one or more of the differential coefficients is zero, the first part may indicate the mantissa bits of the differential coefficient having the maximum magnitude, wherein the result of determining the weighted sum of the differential coefficients and the average coefficient of the 2×2 pixel quadruplet may include including two terms in the weighted sum of the differential coefficients having the maximum magnitude, the two terms being: (i) the weight of the differential coefficient shifted according to the exponent of the differential coefficient, and (ii) the weight of the differential coefficient shifted according to an amount less than one of the exponent of the differential coefficient.

[0045] The second set of bits may include a single portion indicating a sign and an exponent of each of the one or more of the differential coefficients that are non-zero.

[0046] If the differential coefficients are non-zero and not all equal, then the compressed data structure may not indicate any mantissa bits of the differential coefficients.

[0047] The plurality of differential coefficients of the 2×2 pixel quadruplet may include three directional differential coefficients respectively indicating horizontal differential, vertical differential, and diagonal differential in the 2×2 pixel quadruplet.

[0048] The second portion of the compressed data structure may include bits representing a value of the average coefficient determined for the 2x2 pixel quad.

[0049] Each pixel in the pixel block has values ​​in multiple channels, and wherein the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad in the compressed data structure can be represented as a combination of differential channel coefficients determined for the corresponding channels, and wherein determining the sign and exponent of one or more of the non-zero differential coefficients using the first portion of the compressed data structure can include determining the sign and exponent of one or more differential channel coefficients for each of the channels.

[0050] The number of differential channel coefficients for each of the channels may be the same as the number of differential coefficients of the Haar coefficients determined for the 2×2 pixel quad.

[0051] The third part of the compressed data structure may indicate a combination pattern from a set of predetermined combination patterns, according to which the differential channel coefficients determined for the channels are combined to determine the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad, wherein the method may include using the third part of the compressed data structure to identify the combination pattern, and using the identified combination pattern to determine the differential channel coefficients for each of the channels.

[0052] The plurality of channels may include a red channel, a green channel and a blue channel. Each pixel in the pixel block may have an additional value in an alpha channel, wherein the method may further include decoding the alpha channel into the red channel, the green channel and the blue channel respectively.

[0053] The pixel block may be a 4×4 pixel block comprising four 2×2 pixel quadruplets,

[0054] wherein the compressed data structure may include a first portion of each of the 2×2 pixel quads, the first portion being determined using differential coefficients of the 2×2 pixel quads;

[0055] wherein the second portion of the compressed data structure may encode a set of L1 Haar coefficients of a 2×2 coefficient quadruple represented by the average coefficient of each 2×2 pixel quadruple of the 2×2 pixel quadruple, wherein the set of L1 Haar coefficients may include a plurality of L1 differential coefficients and an L1 average coefficient,

[0056] wherein a first sub-portion of the second portion of the compressed data structure may encode the L1 differential coefficients of the 2×2 coefficient quadruple and may include: (i) a first group of bits indicating an order of magnitudes of the L1 differential coefficients, and (ii) a second group of bits indicating a sign and an exponent of each of one or more of the L1 differential coefficients that are non-zero.

[0057] wherein a second sub-portion of the second portion of the compressed data structure may encode the L1 average coefficients of the 2×2 coefficient quad,

[0058] The method may include:

[0059] determining signs and exponents of one or more of the non-zero differential coefficients for each of the 2x2 pixel quads using the first portion of the compressed data structure for each of the 2x2 pixel quads; and

[0060] Determining a representation of the average coefficient for each of the 2×2 pixel quads using the second portion of the compressed data structure by:

[0061] determining signs and exponents of the one or more of the L1 differential coefficients that are non-zero using the first sub-portion of the second portion of the compressed data structure;

[0062] determining a representation of the L1-averaged coefficients of the 2x2 coefficient quad using the second sub-portion of the second portion of the compressed data structure; and

[0063] determining a result of a weighted sum of the L1 differential coefficients and the L1 average coefficients of the 2×2 coefficient quadruple using: (i) the determined signs and exponents of the one or more of the L1 differential coefficients that are non-zero, (ii) the determined representation of the L1 average coefficient, and (iii) the corresponding weights of the L1 differential coefficients;

[0064] Therein, for each of the 2×2 pixel quads, the respective weights of the L1 differential coefficients may be set such that the determined result of the weighted sum provides a determined representation of the average coefficient of the 2×2 pixel quad.

[0065] The second sub-portion of the second portion of the compressed data structure may include bits representing a value of the L1 average coefficient determined for the 2x2 coefficient quad.

[0066] The using the determined result to determine the decoded value may include:

[0067] using the determined result to determine an increment value; and

[0068] A prediction value is added to the delta value to determine the decoded value of the pixel block, wherein the prediction value is determined based on one or more neighboring pixel blocks.

[0069] A decoding unit is provided, the decoding unit being configured to decode a compressed data structure to determine a decoded value, wherein the compressed data structure encodes a set of Haar coefficients for a 2×2 pixel quad of a pixel block, wherein the set of Haar coefficients comprises a plurality of differential coefficients and an average coefficient,

[0070] wherein the first portion of the compressed data structure encodes the differential coefficients of the 2×2 pixel quad and includes: (i) a first set of bits indicating an order of magnitudes of the differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero,

[0071] wherein the second portion of the compressed data structure encodes the average coefficient of the 2×2 pixel quad,

[0072] The decoding unit is configured as follows:

[0073] determining signs and exponents of the one or more of the differential coefficients that are non-zero using the first portion of the compressed data structure;

[0074] determining a representation of the average coefficient using the second portion of the compressed data structure;

[0075] determining a result of a weighted sum of the differential coefficients and the average coefficients for the 2×2 pixel quadruple using: (i) the determined signs and exponents of the one or more of the differential coefficients that are non-zero, (ii) the determined representation of the average coefficient, and (iii) the corresponding weights of the differential coefficients;

[0076] using the determined result to determine the decoded value; and

[0077] Outputs the determined decoded value.

[0078] The decoding unit may further include a filter parameter setting module configured to determine the corresponding weights of the differential coefficients.

[0079] The pixel block may be a 4×4 pixel block including four 2×2 pixel quadruplets, and the decoding unit may further include:

[0080] four decoder instances, each of the four decoder instances being configured to determine, for a corresponding one of the four 2×2 pixel quads, a result of a weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad; and

[0081] A bilinear interpolation unit is configured to combine the four determined results of the corresponding four 2×2 pixel quadruplets by performing bilinear interpolation on the four determined results to determine the decoded value of the pixel block.

[0082] A method for encoding a pixel block into a compressed data structure is provided, the method comprising:

[0083] determining a set of Haar coefficients for a 2×2 pixel quad of the pixel block, wherein the set of Haar coefficients includes a plurality of differential coefficients and an average coefficient;

[0084] determining a first portion of the compressed data structure using the differential coefficients determined for the 2×2 pixel quad, wherein the first portion comprises:

[0085] a first set of bits indicating an order of magnitudes of the differential coefficients; and

[0086] a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero;

[0087] determining a second portion of the compressed data structure using the average coefficient determined for the 2x2 pixel quad; and

[0088] The compressed data structure is stored.

[0089] A coding unit is provided, the coding unit being configured to encode a pixel block into a compressed data structure, the coding unit comprising:

[0090] a Haar coefficient determination module configured to determine a set of Haar coefficients for a 2×2 pixel quad of the pixel block, wherein the set of Haar coefficients includes a plurality of differential coefficients and an average coefficient; and

[0091] a coefficient encoding module configured to determine a first portion of the compressed data structure using the differential coefficients determined for the 2×2 pixel quad, wherein the first portion comprises:

[0092] a first set of bits indicating an order of magnitudes of the differential coefficients; and

[0093] a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero;

[0094] The encoding unit is configured as follows:

[0095] determining a second portion of the compressed data structure using the average coefficient determined for the 2x2 pixel quad; and

[0096] The compressed data structure is output for storage.

[0097] An encoding unit and a decoding unit configured to perform the method described herein are provided.

[0098] The encoding unit / decoding unit may be implemented in hardware on an integrated circuit. A method of manufacturing the encoding unit / decoding unit at an integrated circuit manufacturing system may be provided. An integrated circuit definition data set may be provided, which, when processed in the integrated circuit manufacturing system, configures the system to manufacture the encoding unit / decoding unit. A non-transitory computer-readable storage medium may be provided, the non-transitory computer-readable storage medium having stored thereon a computer-readable description of the encoding unit / decoding unit, which, when processed in the integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit implementing the encoding unit / decoding unit.

[0099] An integrated circuit manufacturing system can be provided, the integrated circuit manufacturing system comprising: a non-temporary computer-readable storage medium having stored thereon a computer-readable description of a coding unit / decoding unit; a layout processing system configured to process the computer-readable description so as to generate a circuit layout description of an integrated circuit that implements the coding unit / decoding unit; and an integrated circuit generation system configured to manufacture the coding unit / decoding unit according to the circuit layout description.

[0100] A computer program code for performing any of the methods described herein may be provided. In other words, a computer readable code may be provided, the computer readable code being configured to cause any of the methods described herein to be performed when the code is run. A non-transitory computer readable storage medium may be provided, the non-transitory computer readable storage medium having computer readable instructions stored thereon, which, when executed at a computer system, cause the computer system to perform any of the methods described herein.

[0101] As will be apparent to those skilled in the art, the above-described features may be combined as appropriate and may be combined with any of the aspects of the examples described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Examples will now be described in detail with reference to the accompanying drawings, in which:

[0103] Figure 1 A schematic diagram showing the mapping of sampling kernels between screen space and texture space;

[0104] Figure 2 A first exemplary encoding system for encoding a block of pixels is described;

[0105] Figure 3 is a flow chart of a method for encoding a pixel block into a compressed data structure;

[0106] Figure 4 It shows how to determine the Haar coefficients for a 2×2 pixel quad.

[0107] Figure 5 A first exemplary format of data in a compressed data structure is shown;

[0108] Figure 6 A second exemplary encoding system for encoding a block of pixels is described;

[0109] Figure 7 A second exemplary format of data in a compressed data structure is shown;

[0110] Figure 8 Different arrangements corresponding to different combination modes are shown;

[0111] Fig. 9 A third exemplary encoding system for encoding a block of pixels is described;

[0112] Fig.10 A third exemplary format of data in a compressed data structure is shown;

[0113] Fig.11 A first exemplary decoding system for decoding a compressed data structure is described;

[0114] Fig.12 is a flow chart of a first exemplary method of decoding a compressed data structure to determine a decoded value;

[0115] Fig.13 A second exemplary decoding system for decoding a compressed data structure is described;

[0116] Fig.14 is a flow chart of a second exemplary method of decoding a compressed data structure to determine a decoded value;

[0117] Figures 15a to 15c A 4×4 block of pixel values ​​is illustrated, highlighting different regions within the 4×4 block of pixel values;

[0118] Fig.15d illustrates how, for different 2×2 quadruplets within a 4×4 pixel block, the transfer function multiplied by the weights applied to the dx coefficients varies for different u parameters representing different horizontal positions in the 4×4 block;

[0119] Fig.15e illustrates how, for different 2×2 quadruplets within a 4×4 pixel block, the transfer function multiplied by the weights applied to the dy coefficients varies for different v parameters representing different vertical positions in the 4×4 block;

[0120] Fig.16a illustrates a 4×4 block of pixel values ​​that has been mirrored across a vertical axis passing through the center of the 4×4 block;

[0121] Fig.16b It shows how the transfer function multiplied by the weights applied to the dx coefficients for different 2×2 quadruplets is expressed Fig.16a The u parameter at different horizontal positions in the mirrored 4×4 block shown in ;

[0122] Fig.16c It shows how the transfer function multiplied by the weights applied to the dy coefficients for different 2×2 quadruplets is expressed as Fig.16a The v parameter varies with different vertical positions in the mirrored 4×4 block shown in ;

[0123] Fig.17a illustrates a 4×4 block of pixel values ​​that has been mirrored across a horizontal axis passing through the center of the 4×4 block;

[0124] Fig.17b It shows how the transfer function multiplied by the weights applied to the dx coefficients for different 2×2 quadruplets is expressed Fig.17a The u parameter at different horizontal positions in the mirrored 4×4 block shown in ;

[0125] Fig.17c It shows how the transfer function multiplied by the weights applied to the dy coefficients for different 2×2 quadruplets is expressed as Fig.17a The v parameter varies with different vertical positions in the mirrored 4×4 block shown in ;

[0126] Fig.18 A computer system is shown in which the encoding and / or decoding unit is implemented; and

[0127] Fig.19 An integrated circuit manufacturing system for producing an integrated circuit implementing an encoding and / or decoding unit is shown.

[0128] The accompanying drawings illustrate various examples. It will be appreciated by the skilled person that the element boundaries (e.g., boxes, box groups, or other shapes) illustrated in the accompanying drawings represent one example of boundaries. In some examples, it may be the case that one element may be designed as multiple elements, or multiple elements may be designed as one element. Where appropriate, common reference numerals are used throughout the drawings to indicate similar features. DETAILED DESCRIPTION

[0129] The following description is given by way of example to enable one skilled in the art to make and use the invention. The present invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to those skilled in the art.

[0130] Embodiments are now described by way of example only. The examples described herein provide an encoding unit for encoding a block of pixels (e.g., representing a portion of a texture) using Haar coefficients. Similarly, a decoding unit is provided for decoding a compressed data structure to determine a block of pixels using Haar coefficients. The examples described herein can cheaply decompose an image into a reduced image with directional derivatives (e.g., Haar wavelet coefficients). The compression format is designed to reduce the complexity of reconstructing the image, whereby the wavelet coefficients are independently weighted according to an algorithm that is used as an implementation of a parametric texture filter that can automatically provide a range of isotropic and anisotropic filtering as part of the decoding process. In addition, the Haar coefficients are stored using an indication of the sign and exponent of the Haar coefficient, so that performing a weighted sum of the Haar coefficients is reduced from a "multiply and add" operation to a "shift and add" operation, where the weights are shifted by the exponent of the Haar coefficient. The encoding format is simple enough to allow fast online compression of small coverage areas of pixels.

[0131] The examples described herein involve encoding a block of pixels. A block of pixels may be part of a texture, where a "pixel" is a "texel" of the texture. In some other examples, a block of pixels may be part of an image rendered by a graphics processing system, such that the image may be compressed before being written out to memory.

[0132] Examples of encoding pixel blocks (e.g., 2×2 blocks or 4×4 blocks, to name just two examples) into compressed data structures are described herein. Different implementations may achieve different compression ratios, for example, some examples achieve a 50% compression ratio, whereby the amount of data in the compressed data structure is half the amount of data in the pixel block, while some examples achieve a 25% compression ratio, whereby the amount of data in the compressed data structure is one-quarter the amount of data in the pixel block.

[0133] The encoding method described herein can be implemented efficiently (e.g., in terms of processing latency, power consumption, and / or silicon area). In addition, the encoding method encodes the data in the compressed data structure in a format so that decoding of the compressed data structure can be performed efficiently (e.g., in terms of processing latency, power consumption, and / or silicon area) to determine the decoded pixel block. Specifically, the encoded data indicates a signed exponential value (i.e., a value that is a power of 2), and the decoding process involves performing a weighted sum of the signed exponential values. Since the signed exponential value is a power of 2, the multiplication of the signed exponential value with the corresponding weight can be implemented with a corresponding binary shift of the weight, where the direction and magnitude of the shift are represented by the sign and exponent of the signed exponent. Compared to multiplication operations, binary shift operations are relatively cheap to implement (e.g., in terms of processing latency, power consumption, and / or silicon area).

[0134] In addition, the encoding method described herein involves determining the Haar coefficients of a 2×2 pixel quadruplet and encoding these Haar coefficients. As described in more detail below, the Haar coefficients of a 2×2 pixel quadruplet include three directional differential coefficients (dx, dy, and dxy) and an average coefficient (dc). When the compressed data structure is decoded, the directional differential coefficients allow different levels of detail to be represented in different directions. The decoding process includes representing the decoded pixel block as a dc value + a certain differential term, which is determined by performing a weighted sum of the dx, dy, and dxy coefficients. Representing the quadruplet only as a dc value actually blurs the entire quadruplet, causing the details of the spatial frequencies higher than the spatial frequency of the quadruplet to be lost, but the details can then be added back using the differential terms, and by selecting the weights of different differential terms, different levels of detail can be added in different directions. For example, if the horizontal differential coefficient (dx) is used in addition to the average coefficient (dc) to decode the 2×2 pixel quad (by setting its weight to a non-zero value), but the vertical and diagonal differential coefficients (dy and dxy) are not used to decode the 2×2 pixel quad (by setting their weight to zero), then the level of detail in the decoded pixel block in the horizontal direction will be twice that in the vertical direction. In this example, the decoding process implements anisotropic filtering with an anisotropy ratio of 2 and with an anisotropy direction along the vertical direction (because the maximum blur occurs along that axis). By changing the weights applied to the dx, dy, and dxy differential coefficients, the anisotropy ratio and the direction of the anisotropy can be changed. In this way, the encoding / decoding process acts as a parametric texture filter, whereby different parameters (e.g., the desired anisotropy ratio and anisotropy direction) can be provided to the decoding unit to apply different types of filtering when decoding the compressed data structure. As another example, if the magnitude of the weights applied to dx and dy are the same, and the weight applied to dxy is equal to the product of the weights applied to dx and dy, then the decoding unit will operate as an isotropic filter. Specifically, when the weights applied to dx, dy, and dxy all have unit magnitude, and the sign of the weight applied to dxy is the sign of the product of the weights applied to dx and dy, then the decoder will reconstruct one of the original 4 pixels in the 2×2 quadruple, that is, if the weights applied to dx and dy are both -1 (for dxy, the weight is +1), then the upper left pixel will be restored, if the weight applied to dx is +1 but the weight applied to dy is -1 (for dxy, the weight is -1), then the upper right pixel will be restored, if the weight applied to dx is -1 but the weight applied to dy is +1 (for dxy, the weight is -1), then the lower left pixel will be restored, and if the weights applied to dx and dy are both +1 (for dxy, the weight is +1), then the lower right pixel will be restored. These cases correspond to isotropic parametric texture filters with a parameter detail level (LOD) of zero and the sampling position falls on the pixel itself.

[0135] Figure 2 A first exemplary encoding system for encoding a pixel block is illustrated. The encoding system comprises an encoding unit 202 and a memory 204. The encoding unit 202 comprises a Haar coefficient determination module 206, which comprises a transform module 208. The encoding unit 202 also comprises a coefficient encoding module 210, a DC encoding module 212 and a packing unit 214. Figure 2 The illustrated components of encoding unit 202 may be implemented in hardware (eg, fixed function circuitry), software, or a combination thereof.

[0136] Figure 3 is a flow chart of a method for encoding a pixel block into a compressed data structure. The steps of the method may be performed in the encoding unit 202 in hardware (e.g., using dedicated hardware, such as fixed function circuitry) or software, or a combination thereof. We will first describe a simple example of encoding a 2×2 pixel quad, where each pixel includes a value in a single channel, but Figure 3 The same steps shown are applied to other more complex examples as described below. In this example, each pixel value in a channel is represented by 8 bits, but it should be understood that this is just an example and the same principles can be applied to pixel values ​​with different number of bits.

[0137] The pixel value block is received at the Haar coefficient determination module 206 of the encoding unit 202. In step S302, the Haar coefficient determination module 206 (specifically, the transform module 208 in this example) determines a set of Haar coefficients for the 2×2 pixel quad. As described above, the set of Haar coefficients includes a plurality of differential coefficients (dx, dy, and dxy) and an average coefficient (dc).

[0138] Figure 4 It is illustrated how to determine the Haar coefficients for a 2×2 pixel quad 402. The pixel quad 402 includes pixels having values ​​p 00 、p 01 、p 10 and p 11 The horizontal differential coefficient (dx) represents the average change in pixel values ​​across the quad in the horizontal direction and is calculated as The value of dx may be rounded, for example, to the nearest representable number in the format of the pixel value or to the format, extended by one or more bits of precision (for accuracy). Figure 4 The vertical differential coefficient (dy) represents the average change in pixel values ​​across the quadruple in the vertical direction and is calculated as The value of dy may be rounded, for example, to the nearest representable number in the format of the pixel value or to the format, extended by one or more bits of precision (for accuracy). The vertical differential coefficient (dy) is Figure 4 denoted as 406 in . The dx and dy coefficients represent the linear change of the pixel values ​​within the 2×2 quadruplet, i.e., the total derivative. The diagonal differential coefficients (dxy) or cross derivatives represent the nonlinear change of the pixel values ​​within the 2×2 quadruplet and are calculated as where the value of dxy may be rounded, for example, to the nearest representable number in the format of the pixel value or to that format, extended by one or more bits of precision (for accuracy). It can be thought of as a derivative first in x and then in y (and therefore represented as such), or vice versa, and as a second-order term, does not have a unique direction associated with it (but because it still plays a role in the directional filtering described herein, we refer to it as a directional derivative elsewhere in this specification). The dxy differential coefficient (dxy) is Figure 4 The average coefficient (dc) represents the average value of the pixel values ​​in the quadruple and is calculated as The value of dc may be rounded, for example, to the nearest representable number in the format of the pixel value. The average coefficient (dc) is Figure 4 The values ​​of dx, dy and dxy are passed from the Haar coefficient determination module 206 to the coefficient encoding module 210. The value of dc is passed from the Haar coefficient determination module 206 to the DC encoding module 212.

[0139] Figure 5 A first exemplary format of data in a compressed data structure 502 is shown. In this example, the compressed data structure includes a first portion 504 and a second portion 508. The first portion 504 includes a first set of bits 5061 and a second set of bits 5062. In this example, the compressed data structure has 16 bits, and the bit positions of the different fields are as follows: Figure 5 . In this example, the first portion 504 includes 11 bits and the second portion 508 includes 5 bits. In this example, the first group of bits 5061 of the first portion includes 4 bits and the second group of bits 5062 of the first portion includes 7 bits. In this example, the input data values ​​of the pixels in the 2×2 pixel quad are 8-bit values, so the input data representing the quad has 32 bits and the compressed data structure has 16 bits, so the compression ratio in this example is 50%.

[0140] The coefficient encoding module 210 uses the differential coefficients (dx, dy, and dxy) determined for the 2x2 pixel quadruplet to determine (in steps S304 and S306) a first portion 504 of the compressed data structure.

[0141] Specifically, in step S304, the coefficient encoding module 210 determines a first group of bits 5061 of the first portion of the compressed data structure, wherein these bits 5061 (in Figure 5 The order of the magnitudes of the differential coefficients (denoted as "encOrder" in ). As described above, the differential terms (dx, dy, and dxy) are used to represent details within a 2x2 quad in different directions. Since the encoding process is a lossy process, some data is lost, and by determining the order of the magnitudes of the differential coefficients, the methods described herein can ensure that when deciding which of the differential coefficients should hold the most data, the more important differential coefficients (i.e., those with larger magnitudes) take precedence over the less important differential coefficients (i.e., those with smaller magnitudes). In addition, as described below in the various defined encodings, the attributes of the coefficients refer to the approximations made by the encoder in order to select the "best fit" from the available options (e.g., comparisons can be made using rounding coefficients rather than the coefficients themselves). The differential coefficients are sorted according to their magnitudes.

[0142] The first set of bits 5061 further indicates how many of the differential coefficients (dx, dy, and dxy) are non-zero. Figure 5 In the example shown, the first group of bits 5061 has 4 bits, so it can represent 16 different options. Note that in other examples, the first group of bits can have a different number of bits, thereby representing a different number of options.

[0143] If the three differential coefficients are nonzero, and if they do not all have equal magnitude, then there are six permutations of the magnitude ordering of the coefficients, and these are represented by the following encOrder bits:

[0144] <![CDATA[encOrder bit 5061]]> The first (largest) coefficient The second (i.e. middle) coefficient The third (smallest) coefficient 0000 dx dy dxy 0001 dy dx dxy 0010 dxy dx dy 0011 dx dxy dy 0100 dy dxy dx 0101 dxy dy dx

[0145] If two of the three differential coefficients are non-zero and one is zero, there are six permutations of the magnitude ordering of the coefficients, and these permutations are represented by the following encOrder bits:

[0146]

[0147] It should be noted that, while other orderings of the permutations are possible, in this example, the integer "encOrder" describes a cyclic rotation of the coefficients with additional pairwise transpositions of smaller (including zero) elements indicated by the least significant bit, which thereby indicates the parity of the permutation. This structure can simplify the encoder and / or decoder logic.

[0148] If one of the three differential coefficients is nonzero and two are zero, there are three permutations of the magnitude ordering of the coefficients, and these are represented by the following encOrder bits (note that in this example, these encodings extend the cyclic ordering of the coefficients):

[0149]

[0150] If all three differential coefficients have the same magnitude (including the case where they are all zero), then there is a permutation in the magnitude ordering of the coefficients, and this is indicated by the following encOrder bits:

[0151] <![CDATA[encOrder bit 5061]]> The first coefficient The second coefficient The third coefficient 1111 Dx dy dxy

[0152] The differential coefficients are considered to be in floating point format. If the differential coefficients are to be represented without losing data, each differential coefficient will be represented by a sign bit, one or more exponent bits, and one or more mantissa bits. In step S306, the coefficient encoding module 210 determines a second set of bits 5062 of the first portion of the compressed data structure, wherein the bits 5062 indicate the sign and exponent of each of the one or more differential coefficients that are non-zero. Figure 5 As shown, the second group of bits 5062 includes Figure 5 The first part 5071 (denoted as "encSgnMnt") and the second part 5072 (denoted as "encExp") in . The first part 5071 indicates the sign of each of the one or more differential coefficients in the non-zero differential coefficients. Figure 5 In the example shown, this first part 5071 has three bits. The second part 5072 indicates the exponent of each of the one or more non-zero differential coefficients.

[0153] The sign and exponent are the most important information maintained in the compressed data structure, i.e., they are more important than the mantissa bits. As described below, in some cases, for example, if the differential coefficients are non-zero and not all equal, the compressed data structure does not indicate any mantissa bits of the differential coefficients. In other cases, for example, if one or more of the differential coefficients is zero, the first portion 5071 of the second group of bits 5062 indicates one or more mantissa bits of the differential coefficient with the maximum magnitude. With reference to the previous discussion, when the non-zero terms of the weighted sum are performed less than the maximum value, additional precision can be provided without increasing the arithmetic complexity (e.g., a floating-point term with a one-bit mantissa can be written as the sum of two signed exponent terms).

[0154] The encoding of the second set of bits 5062 depends on the number of non-zero differential coefficients (as indicated by the first set of bits 5061 ("encOrder")).

[0155] When there are three non-zero differential coefficients and they are not all equal in magnitude (e.g., if "encOrder" is between 0000 and 0101), the first portion 5071 of the second set of bits 5062 has three bits indicating the signs of the three differential coefficients, respectively, and the second portion 5072 (in this example, having four bits) indicates one possible exponent combination of a set of possible exponent combinations of the three differential coefficients. In the example shown below, in all possible combinations, the first coefficient has an exponent in the range [4, 7] (note that these exponents directly correspond to powers of 2 except 0, which is reserved to encode the special value zero, which cannot be represented by a finite exponent), and the exponents of the other two coefficients are differentially encoded relative to the exponent of the first coefficient so that they are at most 2. In the example shown below, the two least significant bits (LSBs) of the second portion 5072 ("encExp") directly encode the two LSBs of the exponent of the first coefficient. If the exponents of the two other coefficients are different, the most significant bit (MSB) of the second portion 5072 ("encExp") encodes the LSB of the difference between the exponents of the first and second coefficients, and the remaining bits in the second portion 5072 ("encExp") encode the LSB of the difference between the exponents of the first and third coefficients. The ordering of the exponents ensures that these values ​​are unique (e.g., if the LSB of the first and second exponent differences is 1, and the LSB of the first and third exponent differences is 0, then the subsequent exponent difference must be 2 to ensure that the third coefficient is less than the second coefficient). If the exponents of the second and third coefficients match, this is indicated by a value of 1 in each of the two MSBs of the second portion 5072 ("encExp"); note that in this example, it is impossible for the second and third coefficients to have different exponents that are no more than 2 less than the first coefficient to both have an LSB of 1. In this case, the parity of the coefficient ordering is redundant, and the LSB of encOrder (labeled "orderSgn") is reinterpreted as encoding the LSB of the difference between the exponents of the first and second coefficients (which is the same as the LSB of the difference between the exponents of the first and third coefficients). Note that in this example, encodings corresponding to three matching magnitudes are excluded from the list of options, regardless of whether the second and third coefficients have matching indices. This is summarized in the following table (where the symbol "X" means "don't care"):

[0156]

[0157]

[0158] When there are two non-zero differential coefficients (e.g., if encOrder is between 0110 and 1011), the first portion 5071 of the second group of bits 5062 has three bits, two of which indicate the signs of the two non-zero differential coefficients, respectively, and the third bit may indicate the mantissa of the first coefficient. The second portion 5072, which has four bits in this example, indicates one possible exponent combination in a set of possible exponent combinations of the two non-zero differential coefficients. In the example shown below, in all possible combinations, if the two encoded coefficients are different (including the mantissa), the exponent of the first coefficient is in the range [4, 7], and if the two encoded coefficients are equal, the exponent of the first coefficient is in the range [1, 8]. In the example shown below, the exponent of the second coefficient is differentially encoded relative to the exponent of the first coefficient so that the exponent of the second coefficient is in the range [-1, 3]. However, the value -1 is only valid when the mantissa bit of the first coefficient is set (i.e., when it is 1), because otherwise the first coefficient would be smaller than the second coefficient, while the value 3 is instead valid when the mantissa bit of the first coefficient is not set (i.e., when it is 0). Note that this assumes that the coefficients have been sorted before rounding, and even if the second coefficient is rounded to a larger value with lower precision, it is still beneficial to encode the largest value with higher precision. The two LSBs of the second portion 5072 ("encExp") directly encode the two LSBs of the exponent of the first coefficient. The 2 MSBs of the second portion 5072 encode (d1_exp-d0_exp) mod 4, where d0_exp is the exponent of the first coefficient and d1_exp is the exponent of the second coefficient. If this value is 0, the exponents match, and if otherwise the mantissa bit of the first coefficient ("d0_mnt") is not set (i.e., if it is 0), then the parity of the coefficient ordering (as indicated by encOrder) is redundant, and the LSB of the encOrder bits ("orderSgn") are reinterpreted as encoding the MSB of d0_expmod 8 = d1_exp mod 8. This is summarized in the following table:

[0159]

[0160]

[0161] When there is a non-zero differential coefficient (e.g., if encOrder is between 1100 and 1110), the first portion 5071 of the second group of bits 5062 has three bits, one of which indicates the sign of the non-zero differential coefficient, and the second and third of which may indicate two mantissa bits of the first coefficient (i.e., the non-zero coefficient). Optionally, one or more of the second and third bits may alternatively be used to increase the precision of the dc coefficient (e.g., when the exponent is small). The second portion 5072 indicates one of a set of possible exponent combinations for the non-zero differential coefficient. In the example below, the exponent of the first coefficient is in the range [1, 8]. Note that the non-zero coefficient cannot be zero (an exponent of 0 is reserved for the value zero instead of 2). 0 =1). The three LSBs of the second part 5072 ("encExp") encode d0_exp mod 8. The MSB of the second part 5072 is used to increase the precision of the dc coefficient, i.e., the MSB of the second part 5072 is re-planned and used as an extra bit in the second part 508, so that the second part 508 can have 6 bits to represent the average coefficient (dc). The following table summarizes this:

[0162]

[0163] When the three coefficients have equal magnitude (e.g., if encOrder is 1111), the first portion 5071 of the second set of bits 5062 has three bits that indicate the signs of the three differential coefficients, respectively, unless all coefficients are zero, in which case these three bits can be reused and used as extra bits in the second portion 508. The second portion 5072 indicates one possible exponent combination in a set of possible exponent combinations of (equal magnitude) differential coefficients. In this case, when there are 3 matching differential coefficients, the exponent is in the range [0, 7]. As described above, when the three coefficients have the same magnitude, the coefficients may all be zero. The three LSBs of the second portion 5072 ("encExp") directly encode this value. If the coefficient is non-zero, the MSB of the second portion 5072 is used to increase the precision of the dc coefficient, i.e., the MSB of the second portion 5072 is re-planned and used as an extra bit in the second portion 508, so that the second portion 508 can have 6 bits to represent the average coefficient (dc). However, if the coefficient is zero, the three bits of the first part 5071 are used to increase the precision of the dc (so that the second part 508 can have 8 bits to represent the average coefficient), while the MSB of the second part 5072 is reserved for a "bypass" flag used in an optional coefficient prediction feature (described later). This is summarized in the following table:

[0164]

[0165] Thus, when the first portion 504 of the compressed data structure has been determined, the method proceeds to step S308, in which the second portion 508 of the compressed data structure is determined using the average coefficient (dc) determined for the 2×2 pixel quad. For example, the DC encoding module 212 may receive the average coefficient (DC) and may receive an indication of how the differential coefficients have been encoded. The DC encoding module 212 may round the average coefficient to determine the bits that represent the value of the average coefficient determined for the 2×2 pixel quad. For example, the average coefficient (dc) may have 8 bits, and if the second portion 508 has 5 bits (e.g., Figure 5 ), the average coefficient is rounded to the nearest 5-bit number for storage in the second portion 508 of the compressed data structure. In different embodiments, different rounding modes and / or rounding techniques may be used for this purpose. As described above, in some cases (e.g., if there is only one non-zero differential coefficient, or if all differential coefficients are equal), there may be more than 5 bits available for the second portion, so less (or no) rounding may need to be applied in some cases.

[0166] In step S310, the compressed data structure is stored in the memory 204. Specifically, the first portion 504 of the data is provided from the coefficient encoding module 210 to the packing unit 214. The second portion 508 of the data is provided from the DC encoding module 212 to the packing unit 214. The packing unit packs the first portion 504 and the second portion 508 into a compressed data structure and outputs the compressed data structure for storage in the memory 204.

[0167] In the above example, each pixel has a value in a single channel. In another example, each pixel in a pixel block has a value in multiple channels. The multiple channels can be red, green and blue channels.

[0168] Figure 6 A second exemplary encoding system for encoding a pixel block is illustrated. The encoding system includes an encoding unit 602 and a memory 604. The encoding unit 602 includes a Haar coefficient determination module 606, which includes a transform module 608 and a mode encoding module 618. The encoding unit 602 further includes a coefficient encoding module 610, a DC encoding module 612, and a packing unit 614. The transform module 608 includes a red channel 616. R 、Green Channel 616 G and blue channel 616 B In some examples, the transformation module 608 may also include an alpha channel transformation module 616 A . Alpha channel transformation module 616 A exist Figure 6As shown in the dashed box in FIG. 6, since it is not always present, we first describe the conversion module 616 without including the alpha channel. A In some examples, the coefficient encoding module 610 may further include an alpha coefficient encoding logic 622. The alpha coefficient encoding logic 622 may be configured to encode the differential coefficients of the red, green, and blue channels. Figure 6 As it is not always present, we first describe an example that does not include the alpha coefficient encoding logic 622. Figure 6 The illustrated components of encoding unit 602 may be implemented in hardware (eg, fixed function circuitry), software, or a combination thereof.

[0169] Figure 6 The coding system shown can be performed Figure 3 The steps of the flowchart shown are to encode a 2×2 pixel quad, where the pixel has values ​​in the red, green and blue channels. As an example, each pixel value in each channel is represented by 8 bits (so 96 bits are used to represent the three channels of four pixels in the 2×2 quad), but it should be understood that this is just an example and the same principles can be applied to pixel values ​​with different number of bits.

[0170] The pixel value block is received at the Haar coefficient determination module 606 of the encoding unit 602. In step S302, the Haar coefficient determination module 606 determines a set of Haar coefficients for the 2×2 pixel quad. For each channel, the set of Haar coefficients includes a plurality of differential coefficients (dx, dy, and dxy) and an average coefficient (dc). Specifically, in step S302, the Haar coefficient determination module 606 determines the differential channel coefficients as described above for each channel in the channel. Thus, for the red channel, three differential channel coefficients (dx.r, dy.r, and dxy.r) and an average coefficient (dc.r) are determined; for the green channel, three differential channel coefficients (dx.g, dy.g, and dxy.g) and an average coefficient (dc.g) are determined; for the blue channel, three differential channel coefficients (dx.b, dy.b, and dxy.b) and an average coefficient (dc.b) are determined. These nine differential coefficients and three average coefficients are provided to the mode encoding module 618. In other examples, the three average coefficients may not be provided to the mode encoding module 618 because the mode encoding module operates on differential coefficients rather than average coefficients.

[0171] The mode encoding module 618 combines the differential channel coefficients determined for the channels to determine the differential coefficients of the Haar coefficients for the 2×2 pixel quad. Note that in this example, the number of differential channel coefficients for each channel in the channel (i.e., three) is the same as the number of differential coefficients of the Haar coefficients determined for the 2×2 pixel quad. In this example, the mode encoding module 618 combines the nine channel differential coefficients (dx.r, dy.r, dxy.r, dx.g, dy.g, dxy.g, dx.b, dy.b, and dxy.b) according to a combination mode to determine three differential coefficients (dx, dy, and dxy) that can be passed to the coefficient encoding module. An indication of the combination mode ("enc_mode") is passed from the mode encoding module 618 to the DC encoding module 612 and then to the packing unit 614 so that the indication can be included in the compressed data structure. Note that in Figure 6 In the example shown, the combined mode (enc_mode) may be overwritten by the DC encoding module 612 (e.g., so that bits of the DC coefficient may be stolen) after the coefficients have been determined by the mode encoding module 618, so an indication of enc_mode is first passed to the DC encoding module 612 before being passed to the packing unit 614. In other examples, an indication of enc_mode may be passed directly from the mode encoding module 618 to the packing unit 614 (e.g., in examples where the combined mode cannot be overwritten by the DC encoding module 612).

[0172] Figure 7 702 in this example. The compressed data structure 702 includes a first portion 704, a second portion 708, and a third portion 710. The first portion 704 has Figure 5 The first portion 504 of the compressed data structure 502 shown has the same structure and format, such that it has a first set of bits 7061 (in Figure 7 ) and a second group of bits 7062, the second group of bits comprising a first portion 7071 (in Figure 7 denoted as "encSgnMnt") and the second part 7072 (in Figure 7 As discussed above, in this example, the input data value for a pixel in a 2×2 pixel quad is represented by 96 bits, and the compressed data structure 702 has 32 bits, so the compression ratio in this example is 33% (although sometimes the compression ratio is quoted as 25% if the original data is assumed to be 32-bit RGBA values, albeit without an alpha channel).

[0173] The third portion 710 includes an indication of a combination mode ("encMode"). In this example, step S302 includes determining a third portion 710 of the compressed data structure 702 indicating a combination mode from a set of predetermined combination modes, according to which the differential channel coefficients determined for the channels are combined to determine the differential coefficients of the Haar coefficients for the 2×2 pixel quad. The third portion 710 of the compressed data structure 702 has a number of bits n mode , wherein the set of predetermined combination patterns includes Combination mode. For example, mode It can be 3, so that there are 8 combination patterns in the predetermined combination pattern set.

[0174] The purpose of selecting the combination mode is to preserve as much detail as possible when combining the nine channel differential coefficients into three differential coefficients. The following table indicates a set of eight predetermined combination modes that can be used, and for each combination mode and each of the nine channel differential coefficients (dx.r, dy.r, dxy.r, dx.g, dy.g, dxy.g, dx.b, dy.b, and dxy.b), it indicates which of the output differential coefficients (dx, dy, and dxy) the channel differential coefficient contributes to (a value of 0 indicates that it does not contribute). The table also indicates the bits used to indicate the indication (encMode) for each combination mode:

[0175]

[0176] Figure 8 Different arrangements corresponding to the different combination modes shown in the table above are shown. Specifically, block 802 corresponds to encMode 000 and represents the quaternary as a brightness value. In this mode, the output differential coefficients dx, dy and dxy are determined by the following formula:

[0177] dx=[dx.r+dx.g+dx.b] / 3

[0178] dy=[dy.r+dy.g+dy.b] / 3

[0179] dxy=[dxy.r+dxy.g+dxy.b] / 3

[0180] The other seven combination modes represent different options for partitioning the 2×2 quads into two colors. Block 804 corresponds to encMode 001 and represents the quads as two colors with a vertical partition between the two colors. In this mode, the output differential coefficients dx, dy, and dxy are determined by the following equations: dx=dx.r, dy=dx.g, dxy=dx.b.

[0181] Block 806 corresponds to encMode 010 and represents the quad as two colors with a horizontal split between the two colors. In this mode, the output differential coefficients dx, dy and dxy are determined by: dx = dy.r, dy = dy.g, dxy = dy.b.

[0182] Block 808 corresponds to encMode 011 and represents the quad as two colors, the upper left and lower right pixels are the first color, and the upper right and lower left pixels are the second color. In this mode, the output differential coefficients dx, dy and dxy are determined by the following formulas: dx = dxy.r, dy = dxy.g, dxy = dxy.b.

[0183] Block 810 corresponds to encMode 100 and represents the quad as two colors, the lower right pixel is the first color and the other three pixels are the second color. In this mode, the output differential coefficients dx, dy and dxy are determined by:

[0184] dx=[dx.r+dy.r+dxy.r] / 3

[0185] dy=[dx.g+dy.g+dxy.g] / 3

[0186] dxy=[dx.b+dy.b+dxy.b] / 3

[0187] Block 812 corresponds to encMode 101 and represents the quad as two colors, the top left pixel is the first color and the other three pixels are the second color. In this mode, the output differential coefficients dx, dy and dxy are determined by:

[0188] dx=[dx.r+dy.r-dxy.r] / 3

[0189] dy=[dx.g+dy.g-dxy.g] / 3

[0190] dxy=[dx.b+dy.b-dxy.b] / 3

[0191] Block 814 corresponds to encMode 110 and represents the quad as two colors, the lower left pixel is the first color and the other three pixels are the second color. In this mode, the output differential coefficients dx, dy and dxy are determined by:

[0192] dx=[dx.r-dy.r+dxy.r] / 3

[0193] dy=[dx.g-dy.g+dxy.g] / 3

[0194] dxy=[dx.b-dy.b+dxy.b] / 3

[0195] Block 816 corresponds to encMode 111 and represents the quad as two colors, the top right pixel is the first color and the other three pixels are the second color. In this mode, the output differential coefficients dx, dy and dxy are determined by:

[0196] dx=[dx.r-dy.r-dxy.r] / 3

[0197] dy=[dx.g-dy.g-dxy.g] / 3

[0198] dxy=[dx.b-dy.b-dxy.b] / 3

[0199] The values ​​of dx, dy, and dxy determined by the mode encoding module 618 are provided to the first coefficient encoding logic 620. The first coefficient encoding logic 620 operates as described above for the coefficient encoding module 210 to determine a first portion 704 of the compressed data structure 702 that encodes the differential coefficients. The third portion 710 (encMode) is provided from the mode encoding module 618 to the DC encoding module 612. The average coefficients (dc.r, dc.g, and dc.b) are provided to the DC encoding module 612. The DC encoding module 612 determines a second portion 708 of the compressed data structure 708 as described above for the DC encoding module 212. Specifically, the DC encoding module 612 rounds each of the average coefficients to determine an indication of the average coefficient (in Figure 7 Thus, the second portion 708 of the compressed data structure 702 includes bits representing the average channel coefficient values. Figure 7 In the example shown, each average coefficient is encoded with 6 bits, while in Figure 5In the example shown, each average coefficient is encoded with 5 bits. In addition, in contrast to the DC encoding module 212, there are three channels for distributing the repurposed bits (as opposed to one channel). Therefore, the DC encoding module 612 allocates a portion of the available bits to the channels according to some priority (generally green is considered the most important, followed by red, and finally blue). In addition, since the special case of all differential coefficients being zero does not depend on the mode encoding, and the three equal magnitude coefficients encoded in the luma mode can be equally well represented as one of the modes given above by encMode=100, encMode=101, encMode=110, or encMode=111, some examples may choose to retain encOrder=1111 and encMode=000 to encode only zero vectors. In this way, all seven bits of the second group of bits 7062 of the first portion 704 of the compressed data structure can be repurposed so that the DC coefficient of each of the red, green, and blue channels can be encoded with 8 bits of accuracy, and the remaining bits can be used to indicate a bypass mode (described below). This is a useful feature because it allows flat areas of an image to be rendered without loss of precision (avoiding problems like banding artifacts). It is simple for the DC encoding module 612 to convert the matching coefficient luma mode encoding to the palette coefficient encoding using the received encMode (710) and encoding coefficients (704). The mode encoding module can also ensure that the zero coefficient case automatically selects luma mode, so that the zero coefficient case does not need to be converted from a non-luma mode (to achieve the repurposing of the coefficient bits).

[0200] When deciding which combination mode to select, the mode encoding module 618 selects the combination mode from a set of predetermined combination modes that gives the maximum magnitude (or some suitable approximation) to the differential coefficients (dx, dy, and dxy) of the Haar coefficients for the 2×2 pixel quad. By selecting the combination mode that maximizes the magnitude of the differential coefficients, the differential coefficients of the three channels are combined into a set of dx, dy, and dxy coefficients with minimal data loss. In some examples, a slight deviation from brightness encoding is included, because for blocks that are close to monochrome, higher accuracy can generally be achieved in this mode (compared to palette encoding).

[0201] As described above, in some examples, each pixel in the pixel block has an additional value in the alpha channel. In this case, the transform module 608 includes an alpha channel transform module 616 A, the coefficient encoding module includes alpha coefficient encoding logic 622. The DC encoding module is also configured to encode the DC value of the alpha channel. In this example, when an alpha channel is present in addition to the red, green, and blue channels, the encoding unit is configured to encode the value in the alpha channel independently of the values ​​in the red, green, and blue channels.

[0202] Alpha channel conversion module 616 A A set of Haar coefficients is determined for the alpha channel of the 2×2 pixel quad (as described above for transform module 208). The set of Haar coefficients for the alpha channel includes a plurality of differential alpha coefficients (dx.a, dy.a, and dxy.a) and an average alpha coefficient (dc.a). The differential alpha coefficients (dx.a, dy.a, and dxy.a) are obtained from the alpha channel transform module 616. A is provided to the alpha coefficient encoding logic 622, the average alpha coefficient (dc.a) is obtained from the alpha channel transform module 616 A Provided to the DC encoding module 612.

[0203] The alpha coefficient encoding logic 622 operates as described above for the coefficient encoding module 210 to determine a first alpha portion of the compressed data structure using the differential alpha coefficients determined for the 2×2 pixel quad. By analogy with the above description of the coefficient encoding module 210, it will be appreciated that the first alpha portion includes: (i) a first set of alpha bits that indicate the order of magnitudes of the differential alpha coefficients; and a second set of alpha bits that indicate a sign and exponent for one or more of the differential alpha coefficients that are non-zero.

[0204] The DC encoding module 612 uses the average alpha coefficient determined for the 2×2 pixel quad to determine a second alpha portion of the compressed data structure. Specifically, the DC encoding module 612 determines the second alpha portion of the compressed data structure as described above for the DC encoding module 212. That is, the DC encoding module 612 rounds the average alpha coefficient (dc.a) to determine an indication of the average coefficient, e.g., having 5 or 6 bits, and which can be included in the compressed data structure.

[0205] In the above example, the pixel block is a 2x2 pixel block, such that it comprises a single 2x2 pixel quad.The above example may be referred to as a "high quality" or "HQ" encoding scheme.

[0206] However, in reference Fig. 9 and 10 In the described example, the pixel block is a 4x4 pixel block comprising four 2x2 pixel quadruplets. Fig. 9 A third exemplary encoding system for encoding a block of pixels is described. Fig. 9 The coding system shown is very similar to Figure 6The main difference of the coding system shown is that the average coefficient determined by the Haar coefficient determination module 906 for the four 2×2 quadruplets within the 4×4 pixel block is fed back to the input of the Haar coefficient determination module 906 and can be encoded as the coefficients of the 2×2 quadruplet, similar to how the pixels of the 2×2 quadruplet are encoded. Specifically, Fig. 9 The coding system in the embodiment includes a coding unit 902 and a memory 904. The coding unit 902 includes a Haar coefficient determination module 906, and the Haar coefficient determination module includes a transform module 908 and a mode encoding module 918. The coding unit 902 further includes a coefficient encoding module 910, a DC encoding module 912, and a packing unit 914. The transform module 908 includes a red channel 916. R 、Green Channel 916 G and blue channel 916 B In some examples, the transformation module 908 may also include an alpha channel transformation module 916 A . Alpha channel conversion module 916 A exist Fig. 9 As shown in the dashed box, since it is not always present, we first describe the conversion module 916 without including the alpha channel. A In some examples, the coefficient encoding module 910 may further include an alpha coefficient encoding logic 922. The alpha coefficient encoding logic 922 may be configured to encode the differential coefficients of the red, green, and blue channels. Fig. 9 As shown with a dashed box in FIG. 9 , since it is not always present, we first describe an example that does not include the alpha coefficient encoding logic 922 . Fig. 9 The illustrated components of encoding unit 902 may be implemented in hardware (eg, fixed function circuitry), software, or a combination thereof.

[0207] Fig. 9 The coding system shown can be performed Figure 3 The steps of the flowchart shown are to encode a 4×4 pixel block, where the pixels have values ​​in red, green and blue channels. As an example, each pixel value in each channel is represented by 8 bits (so 384 bits are used to represent three channels of sixteen pixels in a 4×4 block), but it should be understood that this is just an example and the same principles can be applied to pixel values ​​with different number of bits.

[0208] The pixel value block is received at the Haar coefficient determination module 906 of the encoding unit 902. In step S302, the Haar coefficient determination module 906 determines a set of Haar coefficients for each 2×2 pixel quadruplet in the 4×4 pixel block. As described above, for each channel, the set of Haar coefficients includes a plurality of differential coefficients (dx, dy, and dxy) and an average coefficient (dc). Specifically, in step S302, the Haar coefficient determination module 906 determines the differential channel coefficients as described above for each channel in the channel. Thus, for each 2×2 quad, for the red channel, three differential channel coefficients (dx.r, dy.r, and dxy.r) and one average coefficient (dc.r) are determined; for the green channel, three differential channel coefficients (dx.g, dy.g, and dxy.g) and one average coefficient (dc.g) are determined, and for the blue channel, three differential channel coefficients (dx.b, dy.b, and dxy.b) and one average coefficient (dc.b) are determined. These nine differential coefficients and three average coefficients are provided to the mode encoding module 918. In other examples, the three average coefficients may not be provided to the mode encoding module 918 because the mode encoding module operates on the differential coefficients rather than the average coefficients.

[0209] Similar to the above reference Figure 6 As described by the mode encoding module 618 shown in , for each 2×2 pixel quad, the mode encoding module 918 combines the differential channel coefficients determined for the channel to determine the differential coefficients of the Haar coefficients of the 2×2 pixel quad, but in this example, only the brightness combination mode is used (i.e., the mode of enc_mode=000 as described above). Therefore, for each 2×2 pixel quad, the mode encoding module 618 can combine nine channel differential coefficients (dx.r, dy.r, dxy.r, dx.g, dy.g, dxy.g, dx.b, dy.b, and dxy.b) to determine three differential coefficients (dx, dy, and dxy):

[0210] dx=[dx.r+dx.g+dx.b] / 3

[0211] dy=[dy.r+dy.g+dy.b] / 3

[0212] dxy=[dxy.r+dxy.g+dxy.b] / 3

[0213] These three differential coefficients for each of the 2×2 pixel quads are passed to the coefficient encoding module 910 .

[0214] Fig.10The data format in the compressed data structure 1002 in this example is shown. The compressed data structure 1002 includes a first portion of each of the 2×2 quads. Each of the first portions includes a first set of bits (denoted as “encOrder”) and a second set of bits (denoted as “encSgnExp”). For the first 2×2 quad in the 2×2 quad, the first portion is Fig.10 The first set of bits is marked as 10061 and the second set of bits is marked as 10062) 1004. The other pixel quads have the same corresponding bit groups, but they are in Fig.10 The compressed data structure 1002 also includes a second portion 1008 having a structure similar to Figure 7 The structure of the compressed data structure 702 shown in FIG. 1002 and as described below, the coefficient quadruplets are encoded using the average coefficients determined for the four 2×2 pixel quadruplets within the 4×4 pixel block. As described above, in this example, the input data values ​​for the pixels in the 4×4 block are represented by 384 bits, and the compressed data structure 1002 has 64 bits, so the compression ratio in this example is 17% (or 12.5% ​​relative to 32-bit redundant packed RGBA data with unused alpha).

[0215] In this example, the coefficient encoding module 910 performs more aggressive compression on the differential coefficients of the 2×2 pixel quad than described above to determine the corresponding first portion 1004 of the compressed data structure 1002 for the 2×2 pixel quad. Specifically, the first group of bits 10061 is the same as the first group of bits 7061 and 5061 described above (although it may have different behavior in some special cases). However, the second group of bits 10062 includes a single portion that indicates the sign and exponent (and sometimes the mantissa bits in the examples described herein) for one or more of the differential coefficients that are non-zero. This is in contrast to the examples described above where the second group of bits (7062 and 5062) includes two portions.

[0216] The second set of bits 10062 has four bits to indicate the signs and exponents of the three differential coefficients. The encoding of the second set of bits 10062 depends on the number of non-zero differential coefficients (as indicated by the first set of bits 10061 (encOrder)).

[0217] When there are three non-zero differential coefficients and they are not all equal in magnitude (for example, if encOrder is between 0000 and 0101), the three LSBs of the second set of bits 10062 are used to encode the signs of the three coefficients. Since there is only one remaining available bit, the two smaller coefficients are forced to have equal magnitudes, making the parity of the coefficient ordering redundant, and the LSB of encOrder (which is marked as "orderSgn") is reinterpreted to encode the second LSB of the exponent of the first coefficient. The MSB of the second set of bits 10062 is used to encode the LSB of the exponent of the first coefficient. In the example shown below, in all possible combinations, the exponent of the first coefficient is within the range [4, 7], and the exponents of the other two coefficients are equal to each other and one less than the exponent of the first coefficient. The following table summarizes the possible exponent combinations of the three differential coefficients in this example:

[0218]

[0219] When there are two non-zero differential coefficients (e.g., if encOrder is between 0110 and 1011), the second and third MSBs of the second set of bits 10062 are used to represent the two signs of the non-zero differential coefficients (the second MSB is used to encode the sign of the first, i.e., larger coefficient, and the third MSB is used to encode the sign of the second, i.e., smaller coefficient), and the MSB and LSB of the second set of bits 10062 indicate a set of four possible exponent combinations of the two non-zero differential coefficients. In the example shown below, in all possible combinations, the exponent of the first coefficient is within the range [4, 7], and the exponents of the other non-zero coefficients are one less than the exponent of the first coefficient. In this example, the MSB of the second set of bits ("encSgnExp") gives the LSB of the maximum coefficient exponent, and the LSB of the second set of bits ("encSgnExp") gives the second bit of the maximum coefficient exponent. The following table summarizes the possible exponent combinations of the two non-zero differential coefficients in this example:

[0220]

[0221] When there is a non-zero differential coefficient (e.g., if encOrder is between 1100 and 1110), the second MSB of the second group of bits 10062 is used to represent the sign of the non-zero differential coefficient, the MSB of the second group of bits 10062 is used to represent the LSB of the exponent of the non-zero coefficient, and the LSB of the second group of bits 10062 is used to represent the second LSB of the exponent of the non-zero coefficient. If both LSBs of the exponent of the non-zero coefficient are zero, the second LSB of the second group of bits 10062 is used to represent the third LSB of the exponent of the non-zero coefficient, otherwise it is used to represent a single mantissa bit of the non-zero coefficient. The following table summarizes the possible exponents of a single non-zero differential coefficient in this example:

[0222]

[0223] This more aggressive encoding may be referred to as "low quality" or "LQ" encoding. In the example given above, this LQ encoding does not provide three equal magnitude coefficients. Instead, the three equal magnitude coefficient encodings (e.g., if encOrder is 1111) are repurposed to provide additional encoding for a non-zero differential coefficient. Since there is no indication in the coefficient ordering which of the three coefficients is the largest (which can be used to determine the non-zero coefficient), the second LSB of the second group of bits 10062 is used to indicate whether the dx coefficient is non-zero (when the second LSB is 0) or whether the dy coefficient is non-zero (when the second LSB is 1); there is no encoding to select the dxy coefficient (as a second-order term, it is generally less important). The third LSB of the second group of bits 10062 represents the sign of the non-zero coefficient, the MSB of the second group of bits 10062 represents the LSB of the exponent of the non-zero coefficient, and the LSB of the second group of bits 10062 represents the second LSB of the exponent of the non-zero coefficient. If both LSBs of the exponent of a non-zero coefficient are zero, the coefficient is considered to be zero (so that the null vector can still be represented with this reinterpreted encoding), and the second and third LSBs, which indicate which coefficient is non-zero and its sign, respectively, are redundant, and the second LSB of the second group of bits is used instead to indicate a prediction bypass flag (described later). In this example, the non-zero exponent is in the range [1, 3]. The following table summarizes the possible exponents of non-zero differential coefficients:

[0224]

[0225] Therefore, for each 2×2 pixel quad in the 2×2 pixel quad, the Haar coefficient determination module 906 determines a set of Haar coefficients for the 2×2 pixel quad, wherein the set of Haar coefficients includes a plurality of differential coefficients and an average coefficient, and then for each 2×2 pixel quad in the 2×2 pixel quad, the coefficient encoding module 910 determines a corresponding first portion 1004 of the compressed data structure 1002 of the 2×2 pixel quad using the differential coefficients determined for the 2×2 pixel quad.

[0226] The second portion 1008 of the compressed data structure 1002 is determined by the encoding unit 902 using the average coefficient (dc) determined for each of the 2×2 pixel quadruplets. Specifically, a 2×2 coefficient quadruplet represented by the average coefficient determined for each of the 2×2 pixel quadruplets is determined. Fig. 9As shown, the average coefficients are not fed forward to the DC encoding module, but are fed back to the input of the Haar coefficient determination module 906. These average coefficients represent a filtered version of the input pixel block, for example, having been filtered with a square box filter with a width of 2 pixels. In other words, the average coefficients are equivalent to a level 1 ("L1") filtered representation of the input pixel block.

[0227] The Haar coefficient determination module 906 operates on the 2×2 coefficient quadruplet in the same manner as the above-described Haar coefficient determination module 606 operates on the 2×2 pixel quadruplet. In this manner, the Haar coefficient determination module 906 determines a set of L1 Haar coefficients for the 2×2 coefficient quadruplet, wherein the set of L1 Haar coefficients includes a plurality of L1 differential coefficients and an L1 average coefficient.

[0228] The coefficient encoding module 910 uses the L1 differential coefficients to determine a first sub-portion of the second portion 1008 of the compressed data structure (including Fig.10 42 in bit positions 32 through 42 of the L1 differential coefficients), the coefficients being determined for the 2×2 coefficient quads using the techniques described above with reference to the coefficient encoding module 610. A first sub-portion of the second portion 1008 includes: (i) a first set of bits (“encOrder”) that indicate the order of the magnitudes of the L1 differential coefficients, and (ii) a second set of bits (“encExp” and “EncSgnMnt”) that indicate the sign and exponent for one or more of the non-zero L1 differential coefficients. As described above, the “encExp” field indicates the exponent of the non-zero one or more L1 differential coefficients, while “EncSgnMnt” indicates the sign (and possibly one or more bits of mantissa) of the non-zero one or more L1 differential coefficients.

[0229] The coefficient encoding module 910 uses the L1 average coefficients to determine a second sub-portion of the second portion 1008 of the compressed data structure (including Fig.10 63 in the "encDC_R", "encDC_G", and "encDC_B" fields), the coefficients being determined for the 2x2 coefficient quadruplets using the techniques described above with reference to the DC encoding module 612. Specifically, the second sub-portion of the second portion 1008 of the compressed data structure 1002 includes bits representing the value of the L1 average coefficient determined for the 2x2 coefficient quadruplets.

[0230] The pattern encoding module 918 uses the techniques described above with reference to the pattern encoding module 618 to determine the third sub-portion of the second portion 1008 of the compressed data structure (including Fig.10 43 to 45 in the "encMode" field).

[0231] In some examples, the values ​​of the differential coefficients (i.e., dx, dy, and dxy coefficients) of a 2×2 pixel quad are predicted based on the average coefficients (i.e., dc coefficients) of one or more adjacent 2×2 pixel quads. The difference (or "incremental value") between the predicted value and the original value of the differential coefficient of the 2×2 pixel quad can be determined, and then used in the encoding of the differential coefficient (i.e., for determining the first part of the compressed data structure) instead of the differential coefficient. The magnitude of the incremental value is often smaller than the differential coefficient itself. In general, the encoding techniques described herein will be more accurate (i.e., introduce fewer or less important artifacts) when the values ​​encoded as differential coefficients have smaller magnitudes. The average coefficient (i.e., dc coefficient) is not modified by the prediction. If the differential coefficient has a zero magnitude, a bypass control signal can be set, and in response to this bypass control signal being set, the encoding unit cannot implement this prediction technique. This avoids (potentially harmful) predictions in flat color areas.

[0232] Sometimes, the average coefficients of adjacent 2×2 quads may not all be available, in which case the dc values ​​of the current 2×2 quad can be used for prediction. In the explanation given below, the control signals "enable_n", "enable_e", "enable_s" and "enable_w" indicate whether the average coefficients are available for quads across each of the quad boundaries aligned across the north, east, south and west axes. When quads are encoded at the boundaries of a block-based scheme, one or more of these signals may be low. The original Haar coefficients before prediction (i.e., "upstream" of the prediction) are denoted as "dx_up", "dy_up", "dxy_up" and "dc_up", and the coefficients after prediction (i.e., "downstream" of the prediction) are denoted as "dx_dn", "dy_dn", "dxy_dn" and "dc_dn". As described above, the dc coefficients are not modified by the prediction, so dc_dn=dc_up.

[0233] The bypass control signal is set (as described above) such that:

[0234]

[0235] The predictor for the dx coefficients is chosen as the available east and west dc coefficients, respectively, for term c e and c w :

[0236]

[0237] Unnormalized predicted δ′ x By δ′ x =c e -c wDefinition. The prediction is normalized by the separation distance of the dc term. If only one neighborhood is available, this distance is half of what it would be if both neighborhoods were available (because the missing neighborhood is replaced by the center value). (Normalized) prediction δ x Given by:

[0238]

[0239] The normalized predicted value δ can be calculated based on the rounding mode (e.g., round to nearest mode) x Note that if neither the east nor the west neighbor is available, the prediction is zero by definition, so the normalization factor is irrelevant.

[0240] The residual dx coefficient (i.e., the incremental value representing the difference between the dx coefficient and the prediction of the dx coefficient) is given by dx_dn = dx_up – δ x The value of dx_dn may be rounded according to a rounding mode (e.g., round-to-nearest mode). The predictor of the dx coefficient selects the available north and south dc coefficients, respectively, for the term c n and c s :

[0241]

[0242] Unnormalized predicted δ′ y By δ′ y =c s -c n Definition. The prediction is normalized by the separation distance of the dc term. If only one neighborhood is available, this distance is half of what it would be if both neighborhoods were available (because the missing neighborhood is replaced by the center value). (Normalized) prediction δ y Given by:

[0243]

[0244] The normalized predicted value δ can be calculated based on the rounding mode (e.g., round to nearest mode) y Note that if both the northern and southern neighbors are not available, the prediction is zero by definition, so the normalization factor is irrelevant.

[0245] The residual dy coefficient (i.e., the incremental value representing the difference between the dy coefficient and the prediction of the dy coefficient) is given by dy_dn = dy_up – δ y The value of dy_dn may be rounded according to a rounding mode (eg, round to nearest mode).

[0246] The predictor for the dxy coefficients is chosen as the available northwest, northeast, southwest, and southeast dc coefficients, respectively, for term c nw 、cne 、c sw and c se Ideally, for each term, data across both relevant boundaries are available. Otherwise, for a given term, the predictor chooses which of the relevant north, east, south, and west coefficients are available (if any), as follows:

[0247]

[0248]

[0249] Unnormalized prediction δ″ xy Depend on Definition. The predictions are normalized by the separation distance of the dc terms along the x and y axes. If only one neighborhood is available on a given axis, this distance is half of what it would be if both neighborhoods were available (since the missing neighborhood is replaced by the center value). (Normalized) prediction δ xy Given by the following formula (based on the intermediate prediction δ′ xy ):

[0250]

[0251]

[0252] The intermediate predicted value δ′ can be adjusted according to the rounding mode (e.g., round to nearest mode). xy and the normalized predicted value δ y Note that if for a given axis both neighbors are unavailable, the prediction is zero by definition, so the normalization factor is irrelevant.

[0253] The residual dxy coefficient (i.e., the incremental value representing the difference between the dxy coefficient and the prediction of the dxy coefficient) is given by dxy_dn = dxy_up – δ xy The value of dxy_dn can be rounded according to the rounding mode, such as round to nearest mode.

[0254] As described above, in some examples, each pixel in the pixel block has an additional value in the alpha channel. In this case, the transform module 908 includes an alpha channel transform module 916 A, the coefficient encoding module includes alpha coefficient encoding logic 922. The DC encoding module is also configured to encode the DC value of the alpha channel. In this example, when an alpha channel is present in addition to the red, green, and blue channels, the encoding unit is configured to encode the value in the alpha channel independently of the values ​​in the red, green, and blue channels. In one example, a low-quality encoding scheme is used to determine the Haar coefficients for each of the 2×2 pixel quads, and the average coefficients of the alpha channel are fed back to the input of the Haar coefficient determination module 906 so that they can be processed as coefficient quads using a high-quality scheme to determine the L1 Haar coefficients for the alpha channel. The alpha channel is then processed independently of the red, green, and blue channels as described above. In another example, only the alpha channel is encoded using a high-quality encoding scheme (such as Figure 5 and 6 ), and packed together with the low-quality RGB data. This scheme achieves an overall 25% compression, with 4 bits per pixel allocated to the RGB data and 4 bits per pixel allocated to the alpha data.

[0255] The coefficient encoding module (210, 610, 910) employs a combination of reasoning and heuristics to determine the appropriate encoding of a pixel block. For example, the following conditions applied to the input coefficients that are sorted and then appropriately rounded (e.g., to an available low-precision floating point format) may be evaluated in descending order of priority:

[0256] 1. If the maximum differential coefficient has zero magnitude or the bypass control signal has been set, then 0 non-zero coefficients are encoded and encoder is set to 1111. The label "PTFC_0D" (which is used to indicate a special case of all zero coefficients as matching coefficients (i.e., "PTFC_MD"); this is useful to distinguish cases where remapping may be required, such as in the RGB dc coefficient encoding described previously) is assigned to this case.

[0257] 2. If the encoding is high quality encoding (so that the "hq" control signal has been set) and the middle of the ordered differential coefficients has a zero magnitude (which of course means that the lowest value of the ordered differential coefficients also has a zero magnitude), or the hq control signal is not set (i.e., so that low quality encoding is used) and the maximum value of the ordered differential coefficients has a rounding exponent of 8, then 1 non-zero coefficient is encoded and encOrder is set to one of 1100, 1101 or 1110, depending on the ordering of the differential coefficients as described above. The label "PTFC_1D" is assigned to this case.

[0258] 3. If the hq control signal has been set and the differential coefficients have matching indices, or the maximum differential coefficient index lies outside the closed interval [4, 7] and the differential coefficient index range is not greater than 1, then 3 non-zero coefficients of equal magnitude are encoded and encOrder is set to 1111. If the hq control signal has not been set and the maximum differential coefficient index is less than 4, then at most 1 non-zero coefficient is encoded (due to coding sparsity in low quality mode, the codes normally reserved for matching coefficients are repurposed for 1 coefficient). However, in both cases, the label "PTFC_MD" is assigned.

[0259] 4. If the hq control signal has been set, and the exponent range of the coefficients is less than 2, then 3 non-zero coefficients of different magnitude are encoded, and encOrder is set to one of 0000 to 0101 according to the ordering of the differential coefficients as described above. If the hq control signal is set, the exponent range of the coefficients is 2, the mantissa of the main coefficient (rounded to three bits of mantissa) is not equal to 1.375, and the exponent of the largest coefficient is in the interval [4, 7], then 3 non-zero coefficients of different magnitude are encoded, and encOrder is set to one of 0000 to 0101 according to the ordering of the differential coefficients as described above. If the hq control signal is not set, and the exponent range of the coefficients is less than or equal to 2, then 3 non-zero coefficients of the same magnitude are encoded, and encOrder is set to one of 0000 to 0101 according to the ordering of the differential coefficients as described above. The label "PTFC_3D" is assigned.

[0260] 5. If the hq control signal has been set, and the difference in exponent between the two largest coefficients is less than 1 when rounding the larger coefficient with a 1-bit mantissa (when available) (due to the potential higher precision representation of the larger coefficient, it can be negative in high quality mode so that rounding does not preserve the ordering of the terms), then the 2 non-zero coefficients are encoded and encOrder is set to one of 0110 to 1011 depending on the ordering of the differential coefficients as described above. If the hq control signal has been set, the difference in exponent of the two larger coefficients is 1 (taking into account rounding when the larger coefficient has a 1-bit mantissa), and the larger exponent is in the closed exponent interval [4, 7], then the 2 non-zero coefficients are encoded and encOrder is set to one of 0110 to 1011 depending on the ordering of the differential coefficients as described above. If the hq control signal has not been set, and the difference in non-zero coefficient exponents is less than 2, then the 2 non-zero coefficients are encoded and encOrder is set to one of 0110 to 1011 depending on the ordering of the differential coefficients as described above. The label "PTFC_2D" is assigned.

[0261] 6. If the hq control signal has been set, but the maximum coefficient exponent (rounded appropriately to a one-bit mantissa for 2 non-zero coefficient encodings) lies outside the closed exponent interval [4, 7], then 1 non-zero coefficient is encoded and encOrder is set to one of 1100 to 1110, depending on the ordering of the differential coefficients as described above. If the hq control signal has not been set, then 1 non-zero coefficient is encoded (so this is the last condition for low-quality encoding, and subsequent conditions only apply if the hq control signal has been set). The label "PTFC_1D" is assigned.

[0262] 7. If the exponent difference between the two largest coefficients is 2 when the largest coefficient is rounded with a 1-bit mantissa, 2 non-zero coefficients are encoded and encOrder is set to one of 0110 to 1011 depending on the order of the differential coefficients as described above. If the 2d exponent range is 3, the rounded mantissa of the 2d main coefficient is 1, and the mantissa of the main coefficient is neither 1.125 nor 1.625, 2 non-zero coefficients are encoded and encOrder is set to one of 0110 to 1011 depending on the order of the differential coefficients as described above. The label "PTFC_2D" is assigned.

[0263] 8. If none of the above conditions are met, 1 non-zero coefficient is encoded, and encOrder is set to one of 1100 to 1110, depending on the order of the differential coefficients as described above. The label "PTFC_1D" is assigned.

[0264] The encoding process described above is efficient to implement (eg, in terms of latency, power consumption, and / or silicon area).

[0265] We now proceed to describe how to decode the compressed data structure representing a block of pixels. As mentioned above, the decoding process involves representing a 2×2 pixel quad as a dc value + some derivative term, which is determined by taking a weighted sum of the dx, dy, and dxy coefficients.

[0266] Fig.11A first exemplary decoding system for decoding a compressed data structure to determine a decoded value is described. The decoded value represents: (i) a value (e.g., a pixel value) at a pixel position of a 2×2 pixel quad, or (ii) a filtered value (e.g., a filtered texture value) at a position within a pixel block. As described above, the compressed data structure has been encoded. As such, the compressed data structure encodes a set of Haar coefficients for the 2×2 pixel quads of the pixel block, wherein the set of Haar coefficients includes a plurality of differential coefficients (dx, dy, and dxy) and an average coefficient (dc). A first portion (e.g., 504) of the compressed data structure encodes the differential coefficients of the 2×2 pixel quads and includes: (i) a first set of bits (e.g., 5061) indicating an order of magnitudes of the differential coefficients, and (ii) a second set of bits (e.g., 5062) indicating a sign and exponent of each of one or more of the differential coefficients that are non-zero. A second portion (e.g., 508) of the compressed data structure encodes the average coefficient of the 2×2 pixel quad. Fig.11 The decoding system shown includes a decoding unit 1102 and a memory 1104 (which can be connected to Figure 2 The memory 204 shown, Figure 6 The memory 604 or Fig. 9 The decoding unit 1102 includes four decoder instances 1105, 1106, 1108, and 1110. As described below, each of the decoder instances operates in parallel to determine a decoded value (p, respectively). 00 、p 01 、p 10 and p 11 ). More generally, the decoding unit 1102 may include one or more decoder instances. Fig.11 The components of the illustrated decoding unit 1102 may be implemented in hardware (eg, fixed function circuitry), software, or a combination thereof.

[0267] Fig.12 1 is a flow chart of a method for decoding a compressed data structure to determine a decoded value in a decoder instance 1105, but it should be understood that each of the decoder instances (1106, 1108 and / or 1110) can implement the same method. The steps of the method can be performed in the decoding unit 1102 in hardware (e.g., using dedicated hardware, such as fixed function circuitry) or software, or a combination thereof. We will first describe a simple example where a compressed data structure is decoded to determine pixel values ​​of a 2×2 pixel quad, where each pixel includes a value in a single channel, but Fig.12The same steps shown in are applied to other more complex examples described below. In this example, each pixel value in the channel is represented by 8 bits, but it should be understood that this is just an example and the same principles can be applied to pixel values ​​with different number of bits.

[0268] At the decoding unit 1102, a compressed data structure is received, for example, from the memory 1104. In step S1202, the decoding unit 1102 (e.g., the decoder instance 1105) uses the first portion (e.g., 504) of the compressed data structure to determine the sign and exponent of one or more of the differential coefficients, which are non-zero for the 2×2 pixel quad. As described above, the first portion (e.g., 504) of the compressed data structure includes: (i) a first group of bits (e.g., 5061) indicating the order of the magnitudes of the differential coefficients, and (ii) a second group of bits (e.g., 5062) indicating the sign and exponent of each of the one or more of the non-zero differential coefficients. Different implementations of how the first portion of the compressed data structure indicates the sign and exponent of the differential coefficients are described above, and in the different implementations described above, it is apparent how the decoder determines the sign and exponent of one or more of the non-zero differential coefficients from the first portion of the compressed data structure. As described above, the first set of bits also indicates whether any of the differential coefficients is zero (if so, which differential coefficient is zero). Therefore, in step S1202, the decoding unit 1102 (e.g., the decoder instance 1105) can use the first part of the compressed data structure (e.g., 504) to determine whether any of the differential coefficients is zero (if so, which differential coefficient is zero). Note that in most cases, the decoding unit 1102 (e.g., the decoder instance 1105) does not determine any mantissa bits of the differential coefficients, but in some cases determines one or more mantissa bits (as described above).

[0269] In step S1204, the decoding unit 1102 (e.g., decoder instance 1105) uses the second portion (e.g., 508) of the compressed data structure to determine a representation of the average coefficient of the 2×2 pixel quad. Different embodiments of how the second portion of the compressed data structure indicates the average coefficient are described above (including examples in which the first portion (e.g., 504) of the compressed data structure also helps determine the average coefficient), and in the above different embodiments, how the decoder determines the average coefficient from the second portion of the compressed data structure will be apparent. For example, the bits of the second portion of the compressed data structure may be a rounded version of the average coefficient.

[0270] In step S1206, the decoding unit 1102 (e.g., the decoder instance 1105) determines the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad. This is done by using: (i) the determined signs and exponents of one or more of the non-zero differential coefficients (dx, dy, and dxy), (ii) the determined representation of the average coefficient (dc), and (iii) the corresponding weights (w x 、w y and w xy ) to complete.

[0271] The weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad may include the addition of a gamma correction term. For example, the weighted sum may be determined as: dc + w x ·dx+w y dy+w xy ·dxy+γ, where γ is an optional pseudo-gamma correction term. The gamma correction term can be determined empirically and can be different in different embodiments. For example, in some embodiments γ=0. As another example, in other embodiments, γ=α[(1-|w x |)|dx|+(1-|w y |)|dy|+(1-|w xy |)|dxy|] (where α is some fixed constant (e.g., or )) is selected to provide the best approximation for the target application while minimizing computational complexity. If α is known at the compression point, the transform module (e.g., 208) can be modified to perform gamma correction on the average coefficient dc according to the exact gamma response of the color format or according to the following expression:

[0272] dc γ = dc + α (|dx| + |dy| + |dxy|), where dc γ "Gamma corrected" average coefficients. With this substitution, the weighted sum simplifies to

[0273] dc γ +w x ·dx+w y dy+w xy ·dxy-α(|w x ·dx|+|w y ·dy|+|w xy dxy|), assuming that α is a simple power of two (or zero), it can be implemented in binary arithmetic as a signed sum of shifted terms involving the differential coefficients dx, dy, and dxy and their corresponding weights w x 、w y and w xy The product of .

[0274] Because the differential coefficients are represented as signed exponents, determining the result of the weighted sum of the differential coefficients and the average coefficients for the 2×2 pixel quad in step S1206 does not need to involve performing any multiplication or division operations. For example, it may involve performing a binary shift of the weights according to the exponents determined for the corresponding one or more of the differential coefficients in step S1202, and summing the representation of the average coefficient determined in step S1202 and the shifted weights according to the signs determined for the corresponding one or more of the differential coefficients in step S1202. If the sign is negative, the true negation of the corresponding shifted weight can be approximated by bit reversal (but an exact reversal may be employed instead without significantly increasing complexity). Specifically, if the weights are provided in a signed magnitude format (such that the value is encoded with a single bit representing its sign and multiple bits representing its magnitude, such as in an unsigned integer format), the product of the differential coefficient represented as a signed exponent and its corresponding weight is given by Given, where s w is the symbol of the weight, s d is the sign of the coefficient, and e d is its exponent, which is particularly efficient to implement using binary arithmetic. For such differential coefficients, the weighted sum above can be written as:

[0275] , we can see in binary arithmetic that if α is a power of two (e.g. or ) or zero, then it is related to w x 、w y and w xy The signed sum (and magnitude) of the shifted terms of . Note that the above valid representation holds regardless of whether gamma correction is applied (e.g., when α=0).

[0276] If there are less than three non-zero coefficients, the above expression is further simplified. In some examples, if no more than two differential coefficients are represented as non-zero, the largest differential coefficient can be represented by a one-bit mantissa. In addition, in some examples, if only one differential coefficient (i.e., the largest) is represented as non-zero, the largest differential coefficient can be represented by a two-bit mantissa. Such examples introduce additional precision without increasing the overall arithmetic complexity. The general form of the above weighted sum is given by the following formula:

[0277]

[0278] , where d0, d1, and d2 are the differential coefficients sorted in descending order of magnitude, w0, w1, and w2 are their corresponding (sorted) weights, and m1 and m2 are the first and second mantissa bits of the largest coefficient d0. Since m2≠0 means d1=d2=0, as long as the largest coefficient has a two-bit mantissa (so the remaining coefficients are zero), the weighted sum can be written as

[0279]

[0280] This expression has the same number of shift-add terms as the pure exponential (i.e., without mantissa) expression, and thus has similar arithmetic complexity. Note that the (single-bit) mantissa coefficients m1 and m2 represent values ​​of one or zero, and thus are used only to include or exclude extra terms (the logic may exist anyway to handle zero coefficients). Otherwise, since m1≠0 implies d2=0, the weighted sum can be written as long as the largest coefficient has a one-bit mantissa (rather than two bits)

[0281] This expression has the same number of shift-and-add terms as the pure exponential expression, and therefore has similar arithmetic complexity. Thus, we see that by replacing terms involving zero coefficients with partial mantissa products (using relatively cheap selection logic), additional precision can be introduced without increasing complexity.

[0282] In step S1208, the decoding unit 1102 (e.g., decoder instance 1105) uses the result determined in step S1206 to determine a decoded value. In step S1210, the decoding unit 1102 (e.g., decoder instance 1105) outputs the determined decoded value. The output decoded value can be used in any suitable manner, for example, processed by a processing unit, sent to another device, or stored in a memory (e.g., memory 1304), to give a few examples of how to use the decoded value.

[0283] exist Fig.11 In the example shown, the decoding unit 1102 is configured to output pixel values ​​that represent the original pixel values ​​of the 2×2 pixel quad prior to encoding. Thus, in this example, the decoding unit 1102 is not intended to apply filtering to the values, but rather to provide representations of the original pixel values. These representations may be exactly the same as the original pixel values ​​(in a lossless scheme), or they may be approximations of the original pixel values ​​(in a lossy scheme). This approach is useful for the frame buffer decompression process. Fig.11In the example shown, four decoder instances 1105, 1106, 1108, and 1110 each output a different original pixel value of the original pixel value of the encoded 2×2 pixel value. The four decoder instances operate independently and / or in parallel to output four pixel values. Different decoder instances use different weights (w x 、w y and w xy ), in order to determine different decoded values ​​representing different pixels of the 2×2 pixel quad. In this example, the weights do not need to be changed during runtime, so the weights are pre-determined for use by different decoder instances. For example, the weights can be pre-stored in the decoder instance and / or configured into the hardware (e.g., fixed function circuitry) of the decoder instance.

[0284] exist Fig.11 In the example shown, the result of the weighted sum determined in step S1206 is the decoded value, which is then output from the decoder instance, i.e., no further processing is performed on the result of the weighted sum to determine the decoded value. Thus, in this example, step S1208 of determining the decoded value using the determined result includes using the determined result as the decoded value representing the value at the pixel position of the 2×2 pixel quad.

[0285] As described above, steps S1206 and S1208 are performed four times by the corresponding four decoder instances for the corresponding four pixels of the 2×2 pixel quad. Steps S1202 and S1204 may be performed once to determine the sign and exponent of the average coefficient and the differential coefficient of the 2×2 pixel quad, and these values ​​are provided to each decoder instance in the decoder instance. Alternatively, each decoder instance in the decoder instance may implement steps S1202 and S1204. As described above, when w x 、w y and w xy The weights all have unit magnitude and w xy The symbol of weight is w x and w y The decoder will reconstruct one pixel out of the original 4 pixels in the 2×2 quad. x 、w y and w xy When the weights all have unit magnitude, then the pseudo-gamma correction term described above is zero.

[0286] As above reference Figure 4 As stated, and The predetermined weights (w x 、w y and w xy ) is w x=-1, w y = -1 and w xy = +1. Therefore, the result of the weighted sum determined in step S1206 is given by Given.

[0287] The predetermined weights (w x 、w y and w xy ) is w x =-1, w y = +1 and w xy = -1. Therefore, the result of the weighted sum determined in step S1206 is given by Given.

[0288] The predetermined weights (w x 、w y and w xy ) is w x =+1, w y = -1 and w xy = -1. Therefore, the result of the weighted sum determined in step S1206 is given by Given.

[0289] The predetermined weights (w x 、w y and w xy ) is w x =+1, w y = +1 and w xy = +1. Therefore, the result of the weighted sum determined in step S1206 is equivalent to:

[0290] The decoder instance moves the weights by the exponents of the corresponding differential coefficients and uses the signs of the differential coefficients to correctly sum them. Binary shifts and addition / subtraction are easily implemented in hardware, for example. Therefore, the processing power, silicon area and / or latency of the decoding unit 1102 can be reduced compared to conventional decoding units. In addition, the compression ratio that can be achieved using the scheme described herein can be improved compared to traditional encoding / decoding schemes without adversely affecting data quality.

[0291] In some of the above examples, the values ​​of the differential coefficients (i.e., dx, dy, and dxy coefficients) of the 2×2 pixel quad are predicted based on the average coefficients (i.e., dc coefficients) of one or more adjacent 2×2 pixel quads. The difference (or "delta value") between the predicted value and the original value of the differential coefficient of the 2×2 pixel quad can be determined and then used in the encoding of the differential coefficient (i.e., for determining the first part of the compressed data structure) instead of the differential coefficient. In these examples, the step S1208 of determining the decoded value using the determined result includes: (i) determining the delta value using the determined result; and (ii) adding the predicted value to the delta value to determine the decoded value of the pixel block, wherein the predicted value is determined based on one or more adjacent pixel blocks. Since these predicted values ​​are supplementary to the coefficients encoded in the compressed data structure (and may not be suitable for signed exponential representation), they are generally more suitable for examples such as those described above where filtering is not required and therefore variable weights do not need to be applied to the prediction items.

[0292] Fig.13 A second exemplary decoding system for decoding a compressed data structure to determine a decoded value is described. As described above, the compressed data structure has been encoded. Fig.13 The decoding system shown includes a decoding unit 1302 and a memory 1304 (which can be connected to Figure 2 The memory 204 shown, Figure 6 The memory 604 shown, Fig. 9 The memory 904 or Fig.11 1304). The decoding unit 1302 includes a set of decoder instances 1303, the set of decoder instances including four decoder instances 1305, 1306, 1308 and 1310. The decoding unit 1302 also includes a bilinear interpolation unit 1314, which is configured to receive outputs from the decoder instances and output decoded values. The decoding unit 1302 also includes a filter parameter setting module 1312, which is configured to determine the weights of the decoder instances 1303 and the u and v coordinates of the bilinear interpolation unit 1314. Fig.13 The illustrated components of decoding unit 1302 may be implemented in hardware (eg, fixed function circuitry), software, or a combination thereof.

[0293] Fig.14 is a flow chart of a method of decoding a compressed data structure to determine a decoded value in the decoding unit 1302. The steps of the method may be performed in the decoding unit 1302 in hardware (eg, using dedicated hardware such as fixed function circuitry) or software, or a combination thereof.

[0294] This example is particularly useful for texture filtering. In this example, the weights are not predetermined. This allows the weights to be changed at runtime to provide different types of filtering, such as isotropic filtering or anisotropic filtering with different detail levels, different anisotropy ratios, and different anisotropy directions. Specifically, the filter parameter setting module 1312 of the decoding unit 1302 receives an indication of the detail level along the short axis of the filter (p λ ), an indication of the level of detail along the long axis of the filter (p μ ) and anisotropy direction and u and v coordinates indicating the position of the filter sample to be determined. In step S1401, the decoding unit 1302 (specifically, the filter parameter setting module 1312) determines the corresponding weights of the differential coefficients. As described in more detail below, the weights may be determined based on one or more of the following: (i) the position of the filter sample to be determined relative to the 2×2 pixel quad (e.g., as indicated by the u and v coordinates); (ii) an indication of the level of detail provided by the decoded value (e.g., as indicated by the p λ parameter indication); and (iii) an indication of the amount and / or direction of anisotropy to be provided by the decoded value (e.g., as indicated by p λ 、p μ and parameter indication).

[0295] In this example, the compressed data structure encodes 4×4 pixel blocks. Fig.15a As shown, the pixel block includes four 2×2 pixel quads (1502, 1504, 1506, and 1508). In other examples, the four 2×2 pixel quads can be encoded by four compressed data structures, respectively. Fig.15a For each 2×2 pixel quad shown, the upper left pixel is represented as "00", the lower left pixel is represented as "01", the upper right pixel is represented as "10", and the lower right pixel is represented as "11". This compressed data structure can be used to determine the decoded value at a location within the shaded area 1510, which represents the coverage area of ​​the four pixels in the center of the 4×4 pixel block. For example, the representation Fig.15a The compressed data structure for the illustrated 4×4 pixel block identifies a decoded value at location 1511. Location 1511 is identified using u and v coordinates that include horizontal and vertical coordinates of location 1511 within region 1510.

[0296] For each of the four 2×2 pixel quads, a corresponding one of the decoder instances (1305, 1306, 1308, and 1310) sets the weights based on the corresponding set of weights (w x 、w y and w xy), the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quadruplets is determined as described above. Specifically, steps S1402, S1404 and S1406 are performed by a corresponding one of the decoder instances (1305, 1306, 1308 and 1310) for each 2×2 quadruplet in the corresponding 2×2 quadruplets (1502, 1504, 1506 and 1508). Steps S1402 to S1406 are now described with reference to the processing of quadruplet 1502 by decoder instance 1305, but it should be understood that the other decoder instances (1306, 1308 and 13010) process their corresponding quadruplets (1504, 1506 and 1508) in the same manner. In step S1402, the decoder instance 1305 uses the first portion of the compressed data structure (e.g., 504) to determine the sign and exponent of one or more of the differential coefficients that are non-zero for the 2×2 pixel quad. As described above, the first portion of the compressed data structure (e.g., 504) includes: (i) a first group of bits (e.g., 5061) that indicates the order of the magnitudes of the differential coefficients, and (ii) a second group of bits (e.g., 5062) that indicates the sign and exponent of each of the one or more of the non-zero differential coefficients. Different implementations of how the first portion of the compressed data structure indicates the sign and exponent of the differential coefficients are described above, and in the different implementations described above, it is apparent how the decoder determines the sign and exponent of one or more of the non-zero differential coefficients from the first portion of the compressed data structure. As described above, the first group of bits also indicates whether any of the differential coefficients is zero (if so, which differential coefficient is zero). Thus, in step S1402, the decoding unit 1302 (e.g., the decoder instance 1305) can use the first portion of the compressed data structure (e.g., 504) to determine whether any of the differential coefficients is zero (and if so, which differential coefficient is zero). Note that in most cases, the decoding unit 1302 (e.g., the decoder instance 1305) does not determine any mantissa bits of the differential coefficients, but in some cases determines one or more mantissa bits (as described above).

[0297] In step S1404, the decoding unit 1302 (e.g., decoder instance 1305) uses the second portion (e.g., 508) of the compressed data structure to determine a representation of the average coefficient of the 2×2 pixel quad. Different embodiments of how the second portion of the compressed data structure indicates the average coefficient are described above (including examples in which the first portion (e.g., 504) of the compressed data structure also helps determine the average coefficient), and in the above different embodiments, it will be apparent how the decoder determines the average coefficient from the second portion of the compressed data structure. For example, the bits of the second portion of the compressed data structure may be a rounded version of the average coefficient.

[0298] In step S1406, the decoding unit 1302 (e.g., the decoder instance 1305) determines the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad. This is done by using: (i) the determined sign and exponent of one or more of the non-zero differential coefficients (dx, dy, and dxy), (ii) the determined representation of the average coefficient (dc), and (iii) the corresponding weights (w of the differential coefficients received from the filter parameter setting module. x 、w y and w xy ) to complete.

[0299] Each of the decoder instances (1305, 1306, 1308, and 1310) outputs the determined results of their weighted sums to the bilinear interpolation unit 1314. Note that different decoder instances will determine different results because the weights (provided by the filter parameter setting module 1312) used in the weighted sums are different. In step S1408, the bilinear interpolation unit 1314 combines the four determined results of the corresponding four 2×2 pixel quadruplets to determine the decoded value of the pixel block. Specifically, the bilinear interpolation unit 1314 performs bilinear interpolation on the four determined results of the corresponding four 2×2 pixel quadruplets to determine the decoded value of the pixel block. In step S1410, the decoding unit 1302 (e.g., the bilinear interpolation unit 1314) outputs the determined decoded value. The output decoded value can be used in any suitable manner, such as being processed by a processing unit, sent to another device, or stored in a memory (e.g., in the memory 1304), to give a few examples of how to use the decoded value.

[0300] For some locations within the region 1510, the decoded value can be determined from a single 2×2 pixel quad. In these cases, the bilinear interpolation unit 1314 applies a weight of 1 to the determined result of the single 2×2 pixel quad, and it applies a weight of 0 to the other three determined results of the other 2×2 pixel quads. In other words, if the location of the decoded value to be determined is within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quads in the pixel block, the decoded value of the pixel block is equal to the determined result of the single 2×2 pixel quad. Fig.15b Shown with Fig.15a The same four pixel quadruplets (1502, 1504, 1506 and 1508) are shown, and Fig.15bThe pixel positions with black dots are illustrated (e.g., 1512 is the position of the upper left pixel in the 4×4 pixel block). In some cases, if the decoded value falls within a square region defined by the pixel positions of a 2×2 quad, the decoded value can be represented by considering a single pixel in the 2×2 quad. For example, if the position of the decoded value to be determined is within region 1514, the decoded value can be determined using the determined result of the upper left quad (and not using the determined results of the other three quads, e.g., by setting the weights of the other three quads to zero in the bilinear interpolation unit 1314). As another example, if the position of the decoded value to be determined is within region 1516, the decoded value can be determined using the determined result of the upper right quad (and not using the determined results of the other three quads, e.g., by setting the weights of the other three quads to zero in the bilinear interpolation unit 1314). As another example, if the location of the decoded value to be determined is within region 1518, the determined result of the lower left quad may be used to determine the decoded value (and the determined results of the other three quadrants are not used, e.g., by setting the weights of the other three quadrants to zero in bilinear interpolation unit 1314). As another example, if the location of the decoded value to be determined is within region 1520, the determined result of the lower right quad may be used to determine the decoded value (and the determined results of the other three quadrants are not used, e.g., by setting the weights of the other three quadrants to zero in bilinear interpolation unit 1314). In some cases, regions 1514, 1516, 1518, and 1520 may not have the values ​​that they are in. Fig.15b For example, if the decoding unit 1302 were to apply blur or anisotropy, the determined results of a single quad could be used to determine that the region of the decoded value would be smaller than Fig.15b Regions 1514, 1516, 1518, and 1520 are shown because more information from nearby quads will be used to apply blur or anisotropy.

[0301] For some locations within region 1510, the decoded value cannot be determined from a single 2×2 pixel quad. In these cases, the bilinear interpolation unit 1314 mixes the results determined for the two or more 2×2 pixel quads by applying a non-zero weight to the results determined for the two or more 2×2 pixel quads in their weighted sum. In other words, if the location for which the decoded value is to be determined is not within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quads in the pixel block, the decoded value of the pixel block is the result of a weighted sum of two or more of the determined results for the 2×2 pixel quad. Fig.15c Shown with Fig.15a and Fig.15bThe same four pixel quadruplets (1502, 1504, 1506 and 1508) are shown, and Fig.15c The pixel position with the black dot is illustrated (for example, 1512 is the position of the upper left pixel in the 4×4 pixel block). If the position of the decoded value to be determined (for example, position 1511) falls into Fig.15c 1530, the decoded value is determined by mixing the determined results of all four quadruplets, that is, the bilinear interpolation unit 1314 uses non-zero weights in its weighted sum for all four determined results of the four 2×2 pixel quadruplets. If the position of the decoded value to be determined falls into an area that does not fall into any area 1514, 1516, 1518, 1520 or 1530, the decoded value is determined by mixing the determined results of two (but only two) quadruplets in the quadruplets, that is, the bilinear interpolation unit 1314 uses non-zero weights in its weighted sum for the determined results of two 2×2 pixel quadruplets in the 2×2 pixel quadruplets, and uses zero weights in its weighted sum for the determined results of the other two 2×2 pixel quadruplets in the 2×2 pixel quadruplets. For example, if the position of the decoded value to be determined falls within the rectangular area 1522 shown with diagonal hatching, the decoded value is determined by mixing the determined results of the quaternary groups 1502 and 1504 (instead of the quaternary groups 1506 and 1508). As another example, if the position of the decoded value to be determined falls within the rectangular area 1524 shown with diagonal hatching, the decoded value is determined by mixing the determined results of the quaternary groups 1502 and 1506 (instead of the quaternary groups 1504 and 1508). As another example, if the position of the decoded value to be determined falls within the rectangular area 1526 shown with diagonal hatching, the decoded value is determined by mixing the determined results of the quaternary groups 1504 and 1508 (instead of the quaternary groups 1502 and 1506). As another example, if the position of the decoded value to be determined falls within the rectangular area 1528 shown shaded with diagonal lines, the decoded value is determined by mixing the determined results of the quads 1506 and 1508 (rather than the quads 1502 and 1504 ).

[0302] The weights determined by the filter parameter setting module 1312 for each of the 2×2 quadruplets vary according to the u and v parameters defining the locations where the decoded values ​​are to be determined. For example, the weights of the differential coefficients may be determined by: (i) determining the weights of the differential coefficients based on the LOD indication (p λ ), an indication of the amount of anisotropy (which can be represented by p λ and p μ or it may be indicated by an anisotropy ratio η) and an indication of the anisotropy direction to determine initial weights, as described in more detail below; and multiplying the determined initial weights by the corresponding transfer functions to determine the weights of the differential coefficients. The transfer functions can be considered as functions that model the outputs of modules for different inputs. As described in more detail below, each of the transfer functions is based on the position of the decoded value to be determined relative to the 2×2 pixel quadruplet, and each of the transfer functions: (i) is zero at the center of the 2×2 pixel quadruplet, (ii) has a magnitude of one at each pixel position in the 2×2 pixel quadruplet, and (iii) is extrapolated to the edge of the area represented by the 2×2 pixel quadruplet. For example, the transfer function t x and t y Each transfer function in (which is multiplied by the initial weights of the dx and dy coefficients, respectively, to determine the final weights of the dx and dy coefficients) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2]. The transfer function t xy (which is multiplied by the initial weights of the dx and dy coefficients and then added to the initial weight of the dxy coefficient to determine the final weight of the dxy coefficient) is t x and t y The product of the transfer function, so t xy Extrapolation to the edge of the region represented by a 2×2 pixel quad is not necessarily linear, and t xy The value of lies in the range [-4, +4].

[0303] Fig.15d shows the transfer function value t x (which is multiplied by the initial weight of the dx coefficient to determine the final weight of the dx coefficient) varies over 4×4 blocks of pixel values ​​for different quadruplets, and Fig.15e shows the transfer function value t y How does the initial weight of the dy coefficient (multiplied by it to determine the final weight of the dy coefficient) vary over 4×4 blocks of pixel values ​​for different quadruplets. Figures 15a to 15c The region 1510 shown in is the region where the u and v parameters are valid, and within this region: (i) the u parameter changes linearly from 0 at the left edge to 2 at the right edge so that it has a value of 1 at the center of the 4×4 pixel block, and (ii) the v parameter changes linearly from 0 at the upper edge to 2 at the lower edge so that it has a value of 1 at the center of the 4×4 pixel block.

[0304] Each of the transfer functions is determined based on the position of the decoded value to be determined relative to the 2×2 pixel quad. Fig.15d , solid line 1532 shows the transfer function t x, the transfer function is multiplied by the initial weights of the dx coefficients of the quaternary groups on the left side of the 4×4 block (ie, quaternary groups 1502 and 1506); while the dashed line 1534 shows the transfer function t x , the transfer function is multiplied by the initial weights of the dx coefficients of the right quadruplets of the 4×4 block (i.e., quadruplets 1504 and 1508). Fig.15d As shown, the transfer function t of a 2×2 pixel quad x : (i) The center of the 2×2 pixel quad is zero (e.g., line 1532 passes through t at u=0 x = 0, and for example line 1534 passes through t at u = 2 x =0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1532 passes through t at u=-0.5). x = -1 and passes through t at u = -0.5 x = +1, and for example line 1534 passes through t at u = +1.5 x = -1 and crosses t at u = +2.5 x = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1532 has t at the left-hand edge of the 4×4 block x = -2, and has t at the center of the 4×4 block x = +2, and for example line 1534 has t at the right hand edge of the 4×4 block. x = +2, and has t at the center of the 4×4 block x = -2). Outside the left and right edges of the region represented by the 2×2 pixel quad, the transfer function t x is clamped to the value of the nearest edge of the region. For example, line 1532 is clamped to the right hand side of the center of the 4×4 pixel block. x = +2, while line 1534 is clamped at t on the left hand side of the center of the 4×4 pixel block. x =-2 places.

[0305] exist Fig.15e , solid line 1536 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dy coefficients of the top quads of the 4×4 block (i.e., quads 1502 and 1504); while the dashed line 1538 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dx coefficients of the bottom quad of the 4×4 block (i.e., quads 1506 and 1508). Fig.15e As shown, the transfer function t of a 2×2 pixel quad y: (i) The center of the 2×2 pixel quad is zero (e.g., line 1536 passes through t at v=0 y = 0, and for example line 1538 passes through t at v = 2 y =0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1536 passes through t at v=-0.5). y = -1 and crosses t at v = -0.5 y = +1, and for example line 1538 passes through t at v = +1.5 y = -1 and crosses t at v = +2.5 y = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1536 has t at the top edge of the 4×4 block y = -2, and has t at the center of the 4×4 block y = +2, and for example line 1538 has t at the bottom edge of the 4×4 block. y = +2, and has t at the center of the 4×4 block y = -2). Outside the top and bottom edges of the region represented by the 2×2 pixel quad, the transfer function t y is clamped to the value on the nearest edge of the region. For example, line 1536 is clamped at t below the center of the 4×4 pixel block. y = +2, while line 1538 is clamped at t above the center of the 4×4 pixel block. y =-2 places.

[0306] The final weight of the dxy coefficients is given by the transfer function t x and t y The product of gives, as mentioned above, the transfer function t x and t y Each of them has been weighted by the initial weights for dx and dy, respectively, and the initial dxy weights are added (which do not have any spatial (i.e., u and v) dependence). Note that as long as the filter response is isotropic, the initial dxy weights are zero, so in these cases the weight of the dxy coefficient is always the product of the weights of the dx and dy coefficients (with reference to Fig.111534 and 1536, to which the initial weights of the dxy coefficients are added. In another example, the final weights of the dxy coefficients of the upper right quadruplet 1504 are given by the product of the (weighted) transfer functions given by the dashed line 1534 and the solid line 1536, to which the initial weights of the dxy coefficients are added. In another example, the final weights of the dxy coefficients of the lower left quadruplet 1506 are given by the product of the (weighted) transfer functions given by the solid line 1532 and the dashed line 1538, to which the initial weights of the dxy coefficients are added. In another example, the final weight of the dxy coefficients of the lower right quad 1508 is given by the product of the (weighted) transfer functions given by dashed line 1534 and dashed line 1538, where the initial weights of the dxy coefficients are added. As described above, the transfer function t xy Yes x and t y The product of the transfer function, so t xy Extrapolation to the edge of the region represented by the 2×2 pixel quad is not necessarily linear, and can be described as "bilinear". xy The value of is in the range [-4, +4].

[0307] In some examples, in order to apply full weighting to any individual texel within a quad, the texel is mirrored so that the interpolation point falls within a particular quadrant (e.g., the upper left quadrant). After mirroring, and depending on the LOD setting of the filter, the input interpolation weights can be modified to apply the correct amount of blending to the samples (or weighted combinations thereof) generated by the decoding unit for each pixel quad. In general, pixels within a 4×4 pixel block are selectively mirrored about one, one, or both of: (i) a horizontal axis passing through the center of the pixel block, and (ii) a vertical axis passing through the center of the pixel block, so that the location of the decoded value to be determined is within a particular quadrant of the pixel block (e.g., the upper left quadrant).

[0308] Fig.16a 1604, 1606, and 1608. Prior to the mirroring operation, the quadruplets 1602, 1604, 1606, and 1608 correspond to Fig.15a 1502, 1504, 1506 and 1508 for the 4×4 pixel block shown. However, after the mirroring operation, as shown in FIG. Fig.16aAs shown, quads 1604 and 1608 are to the left of the 4×4 block (not to the right, as in 1504 and 1508), and quads 1602 and 1606 are to the right of the 4×4 block (not to the left, as in 1502 and 1506). In addition, in each 2×2 quad, the pixels are mirrored across the vertical axis such that the upper left pixel is represented as "10", the upper right pixel is represented as "00", the lower left pixel is represented as "11", and the lower right pixel is represented as "01". Fig.15a The region 1510 shown (before the mirroring operation) corresponds to Fig.16a If the position of the decoded value to be determined is within the footprint of the "01" pixels in the 2×2 quad 1504, then this mirroring operation can be applied so that after the mirroring operation, the position of the decoded value to be determined is within the footprint of the "01" pixels in the 2×2 quad 1604, which is now in the upper left quadrant of the 4×4 pixel block.

[0309] Fig.16b It shows that for different 2×2 quadruples, the transfer function t x How to target representation Fig.16a The u parameter varies for different horizontal positions in the mirrored 4×4 block shown in . Fig.16b , the solid line 1632 shows the transfer function t x , the transfer function is multiplied by the initial weights of the dx coefficients of the quaternary groups to the right of the mirrored 4×4 block (i.e., quaternary groups 1602 and 1606); while the dashed line 1634 shows the transfer function t x , the transfer function is multiplied by the initial weights of the dx coefficients of the quads to the left of the mirrored 4×4 block (i.e., quads 1604 and 1608). Fig.16b As shown, the transfer function t of a 2×2 pixel quad x : (i) The center of the 2×2 pixel quad is zero (e.g., line 1632 passes through t at u=2 x = 0, and for example line 1634 passes through t at u = 0 x = 0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1632 passes through t at u=2.5). x = -1 and passes through t at u = 1.5 x = +1, and for example line 1634 passes through t at u = +0.5 x = -1 and passes through t at u = -0.5 x = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1632 has t at the right-hand edge of the 4×4 block x= -2, and has t at the center of the 4×4 block x = +2, and for example line 1634 has t at the left hand edge of the 4×4 block x = +2, and has t at the center of the 4×4 block x = -2). Outside the left and right edges of the region represented by the 2×2 pixel quad, the transfer function t x is clamped to the value of the nearest edge of the region. For example, line 1632 is clamped to the left hand side of the center of the 4×4 pixel block. x = +2, while line 1634 is clamped at t on the right hand side of the center of the 4×4 pixel block. x =-2 places.

[0310] Fig.16c It shows that for different 2×2 quadruples, the transfer function t y How to target representation Fig.16a The v parameter varies with different vertical positions in the mirrored 4×4 block shown in . Fig.16c , solid line 1636 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dy coefficients of the top quads of the mirrored 4×4 block (i.e., quads 1604 and 1602); while the dashed line 1638 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dx coefficients of the bottom quad of the mirrored 4×4 block (i.e., quads 1608 and 1606). Fig.16c As shown, the transfer function t of a 2×2 pixel quad y : (i) The center of the 2×2 pixel quad is zero (e.g., line 1636 passes through t at v=0 y = 0, and for example line 1638 passes through t at v = 2 y =0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1636 passes through t at v=-0.5). y = -1 and crosses t at v = -0.5 y = +1, and for example line 1638 passes through t at v = 1.5 y = -1 and crosses t at v = 2.5 y = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1636 has t at the top edge of the 4×4 block y = -2, and has t at the center of the 4×4 block y = +2, and for example line 1638 has t at the bottom edge of the 4×4 block. y= +2, and has t at the center of the 4×4 block y = -2). Outside the top and bottom edges of the region represented by the 2×2 pixel quad, the transfer function t y is clamped to the value on the nearest edge of the region. For example, line 1636 is clamped at t below the center of the 4×4 pixel block. y = +2, while line 1638 is clamped at t above the center of the 4×4 pixel block. y =-2 places.

[0311] Fig.17a 1704, 1706, and 1708. Prior to the mirroring operation, the quadruplets 1702, 1704, 1706, and 1708 correspond to Fig.15a 1502, 1504, 1506 and 1508 for the 4×4 pixel block shown. However, after the mirroring operation, as shown in FIG. Fig.17a As shown, quads 1702 and 1704 are at the bottom of the 4×4 block (not the top, as in 1502 and 1504), and quads 1706 and 1708 are at the top of the 4×4 block (not the bottom, as in 1506 and 1508). In addition, in each 2×2 quad, the pixels are mirrored across the horizontal axis such that the upper left pixel is represented as "01", the upper right pixel is represented as "11", the lower left pixel is represented as "00", and the lower right pixel is represented as "10". Fig.15a The region 1510 shown (before the mirroring operation) corresponds to Fig.17a If the location of the decoded value to be determined is within the footprint of the "10" pixels in the 2x2 quad 1506, then this mirroring operation can be applied so that after the mirroring operation, the location of the decoded value to be determined is within the footprint of the "10" pixels in the 2x2 quad 1706, which is now in the upper left quadrant of the 4x4 pixel block.

[0312] Fig.17b It shows that for different 2×2 quadruples, the transfer function t x How to target representation Fig.17a The u parameter varies for different horizontal positions in the mirrored 4×4 block shown in . Fig.17b , solid line 1732 shows the transfer function t x , the transfer function is multiplied by the initial weights of the dx coefficients of the quaternary groups to the left of the mirrored 4×4 block (i.e., quaternary groups 1702 and 1706); while the dashed line 1734 shows the transfer function t x, the transfer function is multiplied by the initial weights of the dx coefficients of the quadruplets to the right of the mirrored 4×4 block (i.e., quadruplets 1704 and 1708). Fig.17b As shown, the transfer function t of a 2×2 pixel quad x : (i) The center of the 2×2 pixel quad is zero (e.g., line 1732 passes through t at u=0 x = 0, and for example line 1734 passes through t at u = 2 x =0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1732 passes through t at u=-0.5). x = -1 and passes through t at u = -0.5 x = +1, and for example line 1734 passes through t at u = 1.5 x = -1 and passes through t at u = 2.5 x = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1732 has t at the left-hand edge of the 4×4 block x = -2, and has t at the center of the 4×4 block x = +2, and for example line 1734 has t at the right hand edge of the 4×4 block. x = +2, and has t at the center of the 4×4 block x = -2). Outside the left and right edges of the region represented by the 2×2 pixel quad, the transfer function t x is clamped to the value of the nearest edge of the region. For example, line 1732 is clamped to the right hand side of the center of the 4×4 pixel block. x = +2, while line 1734 is clamped at t on the left hand side of the center of the 4×4 pixel block. x =-2 places.

[0313] Fig.17c It shows that for different 2×2 quadruples, the transfer function t y How to target representation Fig.17a The v parameter varies with different vertical positions in the mirrored 4×4 block shown in . Fig.17c , solid line 1736 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dy coefficients of the bottom quads of the mirrored 4×4 block (i.e., quads 1702 and 1704); while the dashed line 1738 shows the transfer function t y , the transfer function is multiplied by the initial weights of the dx coefficients of the top quads of the mirrored 4×4 block (i.e., quads 1706 and 1708). Fig.17cAs shown, the transfer function t of a 2×2 pixel quad y : (i) The center of the 2×2 pixel quad is zero (e.g., line 1736 passes through t at v=2 y = 0, and for example line 1738 passes through t at v = 0 y = 0), (ii) having a magnitude of one at each pixel position in the 2×2 pixel quad (e.g., line 1736 passes through t at v = 2.5). y = -1 and crosses t at v = -0.5 y = +1, and for example line 1738 passes through t at v = +0.5 y = -1 and crosses t at v = -0.5 y = +1), and (iii) is linearly extrapolated to the edge of the region represented by the 2×2 pixel quad so that it lies within the range [-2, +2] (e.g. line 1736 has t at the bottom edge of the 4×4 block y = -2, and has t at the center of the 4×4 block y = +2, and for example line 1738 has t at the top edge of the 4×4 block. y = +2, and has t at the center of the 4×4 block y = -2). Outside the top and bottom edges of the region represented by the 2×2 pixel quad, the transfer function t y is clamped to the value on the nearest edge of the region. For example, line 1736 is clamped at t above the center of the 4×4 pixel block. y = +2, while line 1738 is clamped at t below the center of the 4×4 pixel block. y =-2 places.

[0314] If the location of the decoded value to be determined is within the coverage area of ​​the "00" pixel in the 2×2 quad 1508, two mirroring operations may be performed: (i) a mirroring operation along a horizontal axis passing through the center of the 4×4 block (e.g., as described above with reference to Figures 16a to 16c ), and (ii) by a mirroring operation about a vertical axis passing through the center of the 4×4 block (e.g., as described above with reference to Figures 17a to 17c After two mirroring operations, the location of the decoded value to be determined is within the footprint of the "00" pixels in the 2x2 quad, which is in the upper left quadrant of the 4x4 pixel block.

[0315] We now provide a detailed example of how the filter parameter setting module 1312 can determine the weights. The secondary LOD (p λ ) and the main LOD (p μ) parameters are shifted to determine the shifted secondary LOD parameters (p' λ ) and the main LOD parameters of the shift (p' μ ), so that and where p′ λ and p′ μ The value of may be rounded according to the rounding mode (e.g., round to nearest mode). The above shift operations have been designed to better map the filter response to the expected sampling characteristics of the filter texture (such as a mipmap), but may be skipped in some examples. In particular, since the exact reconstruction of the source pixel is only possible if p′ λ =p′ μ = 0 and p μ ≥p λ ≥0, so the "interpolation" filter can skip the above shift operation.

[0316] p′ λ and p′ μ The value of the logarithmic anisotropy ratio p α , so p α =p′ μ -p′ λ The direction of the anisotropy is determined by the angle with the texture's x-axis. The sine and cosine of are given by:

[0317] and

[0318] From the logarithmic anisotropy ratio p α Weighted sine squared s 2 Determine the xy-axis variance correction p δ , so that p δ =s 2 ·p α , where p δ The value of may be rounded according to a rounding mode (eg, round to nearest mode).

[0319] Thus, the weights of the xy axes closest to the minor axis and the major axis are w λ and w μ Determined as w λ =1-(p λ +p δ ) and w μ =1-(p μ -p δ ).

[0320] The sign of the cosine determines the axis coefficient weight w λ and w μWith w x and w y The corresponding relationship between them:

[0321]

[0322] From the logarithmic anisotropy ratio p α Weighted twice the angle The sine of determines the (scaled) xy covariance weight w xy , so that w xy =2·s·c·p α

[0323] These three weights w x 、w y and w xy Can be used as weights for the dx, dy, and dxy coefficients, respectively. As described above, in some examples, the weights are multiplied by the transfer function before being used as weights for the dx, dy, and dxy coefficients. An example of how this is accomplished by the filter parameter setting module 1312 is now described to determine a set of weights for the dx, dy, and dxy coefficients for each pixel quad in a 4×4 pixel block, and to modify the input interpolation weights (u, v) to feed to the bilinear interpolation unit 1314.

[0324] Since the interpolation blocks (e.g. Fig.15a The region 1510 shown has a corner defined at the centroid of four consecutive pixel quadruplets, so the interpolated coordinates are in the half-open interval [0, 2]. The bilinear interpolation unit 1314, which is provided with parameters in fractional fixed points (non unorm), can only apply integral weighting to at most one corner sample. Therefore, the coordinates are mirrored (as described above) so that the coordinates of the interpolated point are in the closed interval [0, 1]. The value s can be determined x and y To indicate whether the pixel block is to be mirrored. Specifically:

[0325] and

[0326] s x = -1 means that the pixel block is mirrored on the vertical axis (e.g. Fig.16a As shown), and s x A value of = +1 means that the pixel block is not mirrored on the vertical axis. y = -1 means that the pixel block is mirrored on the horizontal axis (e.g. Fig.17a As shown), and s y = +1 means that the pixel block is not mirrored on the horizontal axis. Since the pixel quad must also be mirrored, these operations are recorded and output as mirror_x and mirror_y respectively:

[0327] and

[0328] Among them, the Boolean function sgn() returns true for negative arguments and returns false for positive arguments.

[0329] Mirror coordinate u mirror and v mirror It can be given by the following formula (in pixels):

[0330] and

[0331] Global coefficient weight w x 、w y and w xy Used to control the strength of the dx, dy, and dxy coefficients associated with each pixel quad in the 4×4 patch. As described above, the transfer function is used to linearly extrapolate the dx and dy coefficients over the patch, saturating at the vertical and horizontal mirror lines, respectively:

[0332] (w x0 , w x1 )=(2s x ·w x ·u mirror’ -2s x ·w x ) and (w y0 , w y1 )=(2s y ·w y ·v mirror’ -2s y ·w y )

[0333] Note that x and y The term ensures that the correct signs are applied to the dx and dy coefficients when decoded.

[0334] (Mirrored if mirror_x) The common dx weight of the top-left and bottom-left pixel quads is output as w x0 .

[0335] (Mirror if mirror_x) The common dx weight of the upper right and lower right pixel quads is output as w x1 .

[0336] (Mirror if mirror_y) The common dy weight of the top left and top right pixel quads is output as w y0 .

[0337] (Mirror if mirror_y) The common dy weight of the lower left and lower right pixel quads is output as w y1 .

[0338] The dxy coefficients are extrapolated quadratically, to which a constant contribution is added, given by w xy The coefficients are weighted so that: w xy00 =w xy +w x0 ·w y0 、w xy01 =w xy +w x0 ·w y1 、w xy10 =w xy +w x1 ·w y0 And w xy11 =w xy +w x1 ·w y1

[0339] (Mirror if mirror_x / mirror_y) The dxy weights of the upper left pixel quad, upper right pixel quad, lower left pixel quad, and lower right pixel quad are output as w respectively. xy00 、w xy10 、w xy01 and w xy11 .

[0340] Modified bilinear interpolation factor u mirror and v mirror Ensure that when dx, dy, and dxy are zeroed, the pixel quads are independently reconstructed at full resolution (by disabling interpolation) by applying an outward bias to the coordinates proportional to the dx and dy weights (clamping to zero), and their dc coefficients use ordinary bilinear interpolation:

[0341] The interpolation factor is used as the output signal u out and v out is output for consumption by the bilinear filter unit, so that and By replacing the earlier expression with |w x0 |, we can write From this we see that when w x =0 o'clock (as in conventional (bi)linear interpolation), when Time out = 0 (indicating a region where only the (mirrored) left half of the four quads is interpolated), and whenever u mirror =1 hour (so that a 50:50 blend is always applied to vertical mirror lines.) Similarly, by replacing the earlier expression with |w y0 |, we can write From this we see that when w y =0 o'clock (as in conventional (bi)linear interpolation), when Time out = 0 (indicating a region where only the (mirrored) top halves of the four quads are interpolated), and whenever u mirror =1 hour (so that a 50:50 blend is always applied to the horizontal mirror line).

[0342] A number of exemplary encoding formats are described above, and the manner in which a compressed data structure is decoded to determine the sign and exponent of the differential coefficients from a first portion of the compressed data structure and to determine a representation of the average coefficients from a second portion of the compressed data structure will be apparent to a technician who has read the description of the manner in which data is encoded into a compressed data structure in the different examples above.

[0343] For example, the first group of bits (e.g., 5061) of the first part (e.g., 504) of the compressed data structure indicates the order of the magnitudes of the differential coefficients and can indicate how many of the differential coefficients are non-zero, so the decoding unit can use the first group of bits to determine how many of the differential coefficients are non-zero and determine the order of the non-zero differential coefficients encoded in the compressed data structure based on the encoding format used (as described above).

[0344] As described above, the second group of bits may include: (i) a first portion (e.g., 5071) indicating the sign of each of the non-zero differential coefficients, and (ii) a second portion (e.g., 5072) indicating the exponent of the non-zero differential coefficient. Thus, the decoding unit may use the second group of bits to determine the sign and exponent of each of the non-zero differential coefficients according to the encoding format used (as described above). The first portion is composed of a first bit series, the first bit series indicating the sign of each of the one or more differential coefficients of the non-zero differential coefficients, and the second portion is composed of a second bit series, the second bit series indicating the exponent of each of the one or more differential coefficients of the non-zero differential coefficients of the second group of bits.

[0345] As described above, if one or more of the differential coefficients is zero, the first part (e.g., 5071) may indicate the mantissa bits of the differential coefficient having the largest magnitude. In this case, the determination of the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quaternion performed in step S1406 includes including two terms in the weighted sum of the differential coefficients having the largest magnitude, the two terms being: (i) the weight of the differential coefficient shifted according to the exponent of the differential coefficient, and (ii) the weight of the differential coefficient shifted by an amount less than one of the exponents of the differential coefficient. In this way, the weighted sum is still performed as the sum of (at most three) shifted weights, which is simple to implement in the decoding unit and, in particular, does not include complex operations such as multiplication or division operations.

[0346] In some of the examples described above (e.g., reference Fig.10 ), the second group of bits (10062) includes a single portion that indicates the sign and exponent of each of the one or more of the non-zero differential coefficients. In these examples, the decoding unit can use the second group of bits to determine the sign and exponent of each of the non-zero differential coefficients based on the encoding format used (as described above). The single bit portion is composed of a single bit series that indicates the sign and exponent of each of the one or more of the non-zero differential coefficients of the second group of bits.

[0347] In some of the examples described above (e.g., reference Figures 6 to 10 ), each pixel in the pixel block has values ​​in multiple channels (e.g., red, green, blue channels), and the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad in the compressed data structure are represented as combinations of differential channel coefficients determined for the corresponding channels. In these examples, according to the encoding format used (as described above), the first part of the compressed data structure is used to determine the sign and exponent of one or more of the non-zero differential coefficients for each of the channels (in step S1402).

[0348] In some of the examples described above (e.g., reference Figure 7 ), a third portion (e.g., 710) of the compressed data structure indicates a combination pattern from a set of predetermined combination patterns according to which the differential channel coefficients determined for the channels are combined to determine the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad. In these examples, the decoding unit 1302 can use the third portion of the compressed data structure to identify the combination pattern according to the encoding format used (as described above). The decoding unit 1302 can then use the identified combination pattern to determine the differential channel coefficients for each of the channels according to the encoding format used (as described above).

[0349] In some of the examples described above (e.g., reference Figure 6 and Fig. 9 ), the multiple channels include red, green and blue channels, and each pixel in the pixel block has an additional value in the alpha channel. In these examples, the decoding unit 1302 can decode the alpha channel into red, green and blue channels respectively according to the encoding format used (as described above).

[0350] In some of the examples described above (e.g., reference Fig. 9 and Fig.10), the pixel block is a 4×4 pixel block including four 2×2 pixel quads, wherein the compressed data structure includes a first portion (e.g., 1004) of each 2×2 pixel quad in the 2×2 pixel quad, the first portion being determined using differential coefficients of the 2×2 pixel quad. A second portion (e.g., 1008) of the compressed data structure encodes a set of L1 Haar coefficients of the 2×2 coefficient quad represented by an average coefficient of each 2×2 pixel quad in the 2×2 pixel quad, wherein the set of L1 Haar coefficients includes a plurality of L1 differential coefficients and an L1 average coefficient. A first sub-portion of the second portion (e.g., 1008) of the compressed data structure encodes the L1 differential coefficients of the 2×2 coefficient quad and includes: (i) a first set of bits indicating an order of magnitudes of the L1 differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the non-zero L1 differential coefficients. The second sub-portion of the second portion of the compressed data structure (e.g., 1008) encodes the L1 average coefficients of the 2×2 coefficient quadruplets. In this example, the decoding unit 1302 uses the first portion of the compressed data structure (e.g., 1004) for each 2×2 pixel quadruplet in the 2×2 pixel quadruplet to determine the sign and exponent of one or more of the non-zero differential coefficients according to the encoding format used (as described above). The decoding unit 1302 uses the second part of the compressed data structure to determine the representation of the average coefficient of each 2×2 pixel quad in the 2×2 pixel quad by: (i) using the first sub-part of the second part of the compressed data structure to determine the sign and exponent of the one or more L1 differential coefficients among the non-zero L1 differential coefficients according to the encoding format used (as described above), (ii) using the second sub-part of the second part of the compressed data structure to determine the representation of the L1 average coefficient of the 2×2 coefficient quad according to the encoding format used (as described above), and (iii) using the following to determine the result of the weighted sum of the L1 differential coefficients and the L1 average coefficient of the 2×2 coefficient quad: (i) the determined signs and exponents of the one or more L1 differential coefficients among the non-zero L1 differential coefficients, (ii) the determined representation of the L1 average coefficient, and (iii) the corresponding weights of the L1 differential coefficients. For each of the 2x2 pixel quads, the respective weights of the L1 differential coefficients are set such that the determined result of the weighted sum provides the determined representation of the average coefficient of the 2x2 pixel quad.

[0351] Fig.18A computer system in which the encoding and / or decoding unit described herein may be implemented is shown. The computer system includes a CPU 1802, a GPU 1804, a memory 1806, and other devices 1814, such as a display 1816, a speaker 1818, and a camera 1822. The encoding and / or decoding unit 1810 (corresponding to the encoding unit 202, 602 or 902 or the decoding unit 1102 or 1302) and a neural network accelerator (NNA) 1811 are implemented on the GPU 1804. In other examples, the encoding and / or decoding unit 1810 may be implemented on the CPU 1802 or within the NNA 1811, or as a separate processing unit in the computer system. The components of the computer system may communicate with each other via a communication bus 1820. The storage repository 1812 (corresponding to the memory 204, 604, 904, 1104, and 1304) is implemented as part of the memory 1806.

[0352] The coding and / or decoding unit (202, 602, 902, 1102 and / or 1302) is shown as including a plurality of functional blocks. This is merely illustrative and is not intended to limit the strict division between the different logical elements of such an entity. Each functional block may be provided in any suitable manner. It should be understood that the intermediate values ​​described herein as being formed by a coding unit or a decoding unit do not need to be physically generated by the coding and / or decoding unit at any point, and may simply represent a logical value that conveniently describes the processing performed by the coding and / or decoding unit between its input and output.

[0353] The coding and / or decoding unit described herein can be implemented in hardware on an integrated circuit. The coding and / or decoding unit described herein can be configured to perform any of the methods described herein. In general, any of the functions, methods, techniques or components described above can be implemented in software, firmware, hardware (e.g., fixed logic circuitry) or any combination thereof. The terms "module", "functionality", "component", "element", "unit", "block" and "logic" can be used herein to generally represent software, firmware, hardware or any combination thereof. In the case of software implementation, a module, functionality, component, element, unit, block or logic represents a program code that performs a specified task when executed on a processor. The algorithms and methods described herein can be performed by one or more processors that execute code, and the code causes the processor to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disk, flash memory, hard disk storage, and other memory devices that can use magnetic, optical and other technologies to store instructions or other data and can be accessed by a machine.

[0354] As used herein, the terms computer program code and computer readable instructions refer to any kind of executable code for execution by a processor, including code expressed in machine language, interpreted language, or scripting language. Executable code includes binary code, machine code, byte code, code that defines an integrated circuit (e.g., hardware description language or netlist), and code expressed in programming language code such as C, Java, or OpenCL. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, run in a virtual machine or other software environment, causes a processor of a computer system supporting the executable code to perform the tasks specified by the code.

[0355] A processor, computer or computer system may be any kind of device, machine or special purpose circuit, or a collection or part thereof, which has processing capabilities so that instructions can be executed. A processor may be or include any kind of general or special purpose processor, such as a CPU, GPU, NNA, system on chip, state machine, media processor, application specific integrated circuit (ASIC), programmable logic array, field programmable gate array (FPGA), etc. A computer or computer system may include one or more processors.

[0356] It is also intended to cover software that defines the configuration of hardware as described herein, such as HDL (Hardware Description Language) software, such as for designing integrated circuits or for configuring programmable chips to achieve desired functions. That is, a computer-readable storage medium having computer-readable program code in the form of an integrated circuit definition data set encoded thereon may be provided, which, when processed (i.e., run) in an integrated circuit manufacturing system, configures the system to manufacture encoding and / or decoding units configured to perform any of the methods described herein, or to manufacture encoding and / or decoding units including any of the devices described herein. The integrated circuit definition data set may be, for example, an integrated circuit description.

[0357] Thus, a method of manufacturing the encoding and / or decoding unit described herein at an integrated circuit manufacturing system may be provided. In addition, an integrated circuit definition data set may be provided, which, when processed in an integrated circuit manufacturing system, enables the method of manufacturing the encoding and / or decoding unit to be performed.

[0358] The integrated circuit definition data set may be in the form of computer code, for example as a netlist, code for configuring a programmable chip, as a hardware description language that defines hardware suitable for fabrication at any level in an integrated circuit, including as register transfer level (RTL) code, as a high-level circuit representation such as Verilog or VHDL, and as a low-level circuit representation such as OASIS (RTM) and GDSII. Higher-level representations that logically define hardware suitable for fabrication in an integrated circuit (such as RTL) may be processed on a computer system configured to generate a manufacturing definition of the integrated circuit in the context of a software environment that includes definitions of circuit elements and rules for combining those elements to generate a manufacturing definition of the integrated circuit defined by the representation. As is typically the case with software executed at a computer system to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to configure the computer system for generating a manufacturing definition of the integrated circuit to execute code that defines the integrated circuit to generate the manufacturing definition of the integrated circuit.

[0359] Now about Fig.19 An example of processing an integrated circuit definition data set at an integrated circuit manufacturing system to configure the system to manufacture encoding and / or decoding units is described.

[0360] Fig.19 An example of an integrated circuit (IC) manufacturing system 1902 configured to manufacture an encoding and / or decoding unit as described in any of the examples herein is shown. In particular, the IC manufacturing system 1902 includes a layout processing system 1904 and an integrated circuit generation system 1906. The IC manufacturing system 1902 is configured to receive an IC definition data set (e.g., defining an encoding and / or decoding unit as described in any of the examples herein), process the IC definition data set, and generate an IC (e.g., which embodies an encoding and / or decoding unit as described in any of the examples herein) based on the IC definition data set. The processing of the IC definition data set configures the IC manufacturing system 1902 to manufacture an integrated circuit embodying an encoding and / or decoding unit as described in any of the examples herein.

[0361] The layout processing system 1904 is configured to receive and process an IC definition data set to determine a circuit layout. Methods for determining a circuit layout based on an IC definition data set are known in the art and may, for example, involve synthesizing RTL code to determine a gate-level representation of a circuit to be generated, such as in terms of logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining location information for the logic components, the circuit layout may be determined based on the gate-level representation of the circuit. This may be done automatically or with user involvement in order to optimize the circuit layout. When the layout processing system 1904 has determined the circuit layout, it may output the circuit layout definition to the IC generation system 1906. The circuit layout definition may be, for example, a circuit layout description.

[0362] As is known in the art, IC generation system 1906 generates an IC according to a circuit layout definition. For example, IC generation system 1906 may implement a semiconductor device manufacturing process to generate an IC, which may involve a multi-step sequence of photolithography and chemical processing steps during which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition may be in the form of a mask that can be used in a photolithography process to generate an IC according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1906 may be in the form of a computer readable code that IC generation system 1906 may use to form a suitable mask for generating an IC.

[0363] The different processes performed by the IC manufacturing system 1902 may all be implemented at one location, such as by one party. Alternatively, the IC manufacturing system 1902 may be a distributed system such that some processes may be performed at different locations and may be performed by different parties. For example, some of the following stages may be performed at different locations and / or by different parties: (i) synthesizing RTL code representing an IC definition data set to form a gate-level representation of a circuit to be generated; (ii) generating a circuit layout based on the gate-level representation; (iii) forming a mask based on the circuit layout; and (iv) using the mask to manufacture the integrated circuit.

[0364] In other examples, processing of the integrated circuit definition data set at the integrated circuit manufacturing system may configure the system to manufacture encoding and / or decoding units in which the IC definition data set is not processed to determine the circuit layout. For example, the integrated circuit definition data set may define a configuration of a reconfigurable processor, such as an FPGA, and processing of the data set may configure the IC manufacturing system to generate (e.g., by loading the configuration data into the FPGA) a reconfigurable processor having the defined configuration.

[0365] In some embodiments, when processed in an integrated circuit manufacturing system, the integrated circuit manufacturing definition data set can enable the integrated circuit manufacturing system to generate a device as described herein. Fig.19 An integrated circuit manufacturing system configured in the manner described may manufacture the apparatus described herein.

[0366] In some examples, the integrated circuit definition data set may include software that runs on, or in combination with, hardware defined at the data set. Fig.19 In the example shown in , the IC generation system can be further configured by the integrated circuit definition dataset to load firmware onto the integrated circuit according to the program code defined at the integrated circuit definition dataset when manufacturing the integrated circuit, or otherwise provide the integrated circuit with program code for use with the integrated circuit.

[0367] Compared to known implementations, the implementation of the concepts set forth in this application in devices, apparatuses, modules and / or systems (and in the methods implemented herein) can cause performance improvements. Performance improvements can include one or more of improved computing performance, reduced latency, increased throughput and / or reduced power consumption. During the manufacture of such devices, devices, modules and systems (e.g., in integrated circuits), a trade-off can be made between performance improvements and physical implementations to improve manufacturing methods. For example, a trade-off can be made between performance improvements and layout area to match the performance of known implementations, but using less silicon. For example, this can be accomplished by reusing functional blocks in a serial manner or sharing functional blocks between elements of devices, devices, modules and / or systems. On the contrary, the concepts of improvements (such as reduced silicon area) that cause physical implementations of devices, devices, modules and systems set forth in this application can be weighed against performance improvements. This can be accomplished, for example, by manufacturing multiple instances of a module within a predefined area budget.

[0368] Applicants hereby independently disclose each individual feature described herein and any combination of two or more such features, to the extent that such feature or combination can be implemented according to the common knowledge of a person skilled in the art based on the specification as a whole, regardless of whether such feature or combination of features solves any problem disclosed herein. In view of the foregoing description, it will be clear to those skilled in the art that various modifications can be made within the scope of the invention.

Claims

1. A method of decoding a compressed data structure to determine a decoded value, wherein the compressed data structure encodes a set of Haar coefficients for a 2×2 pixel quad of a pixel block, wherein the set of Haar coefficients includes a plurality of differential coefficients and an average coefficient, wherein the first part of the compressed data structure encodes the differential coefficients of the 2×2 pixel quad and comprises: (i) a first set of bits indicating an order of magnitudes of the differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero, wherein the second portion of the compressed data structure encodes the average coefficient of the 2×2 pixel quad, The method comprises: determining signs and exponents of the one or more of the differential coefficients that are non-zero using the first portion of the compressed data structure; determining a representation of the average coefficient using the second portion of the compressed data structure; determining a result of a weighted sum of the differential coefficients and the average coefficients for the 2×2 pixel quadruplet using: (i) the determined signs and the determined exponents of the one or more of the differential coefficients that are non-zero, (ii) the determined representation of the average coefficient, and (iii) the corresponding weights of the differential coefficients; using the determined result to determine the decoded value; and Outputs the determined decoded value.

2. The method of claim 1, wherein said determining the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad comprises: performing a binary shift of said weights according to said determined exponents of corresponding one or more of said differential coefficients that are non-zero; and The determined representation of the average coefficient and the shifted weight are summed according to the determined signs of the corresponding one or more of the differential coefficients that are non-zero.

3. A method as claimed in claim 1 or 2, wherein said determining the result of the weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad does not involve performing any multiplication or division operations.

4. The method of claim 1 or 2, wherein determining the result of the weighted sum of the differential coefficients and the average coefficients of the 2x2 pixel quad comprises adding a gamma correction term.

5. A method as claimed in claim 1 or 2, wherein the corresponding weights of the differential coefficients are predetermined, and wherein the use of the determined result to determine the decoded value includes using the determined result as the decoded value representing the value at the pixel position of the 2×2 pixel quad.

6. A method as claimed in claim 5, wherein the step of determining the result of the weighted sum of the differential coefficient and the average coefficient of the 2×2 pixel quadruplet and the step of determining the decoded value using the determined result are performed four times for the corresponding four pixels of the 2×2 pixel quadruplet, wherein the predetermined weight of the differential coefficient is different each time the steps are performed, thereby determining four decoded values ​​representing the values ​​at four pixel positions of the 2×2 pixel quadruplet.

7. The method according to claim 1 or 2, further comprising: The corresponding weights of the differential coefficients are determined using a transfer function based on positions relative to the 2x2 pixel quad to model outputs for different inputs.

8. The method according to claim 7, The weights are determined based on one or more of the following: a position of the decoded value to be determined relative to the 2×2 pixel quad; an indication of a level of detail to be provided by the decoded value; and An indication of the amount and / or direction of anisotropy to be provided by the decoded value.

9. A method as claimed in claim 7, wherein the pixel block is a 4×4 pixel block including four 2×2 pixel quads, wherein for each of the four 2×2 pixel quads, a result of a weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad is determined, and wherein the four determined results of the corresponding four 2×2 pixel quads are combined to determine a decoded value of the pixel block.

10. The method of claim 9, wherein the four determined results for the corresponding four 2x2 pixel quadruplets are combined by performing bilinear interpolation on the four determined results to determine the decoded value of the pixel block.

11. A method as claimed in claim 9, wherein if the position of the decoded value to be determined is within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quad in the pixel block, then the decoded value of the pixel block is equal to the determined result of the single 2×2 pixel quad.

12. A method as claimed in claim 9, wherein if the position of the decoded value to be determined is not within an area that can be represented by considering the pixels of a single 2×2 pixel quad in the 2×2 pixel quad in the pixel block, then the decoded value of the pixel block is the result of a weighted sum of two or more of the determined results of the 2×2 pixel quad.

13. The method of claim 11, wherein said determining said corresponding weights of said differential coefficients comprises: determining initial weights based on an indication of a level of detail, an indication of an amount of anisotropy, and an indication of a direction of anisotropy provided by the decoded values; and The determined initial weights are multiplied by corresponding transfer functions to determine the weights of the differential coefficients, wherein each of the transfer functions is based on a position of the decoded value to be determined relative to the 2×2 pixel quad, and wherein each of the transfer functions: (i) is zero at a center of the 2×2 pixel quad, (ii) has a magnitude of one at each pixel position of the 2×2 pixel quad, and (iii) is extrapolated to an edge of the region represented by the 2×2 pixel quad.

14. The method of claim 13, wherein outside the edge of the region represented by the 2x2 pixel quad, the transfer function is clamped to the value of the transfer function at the nearest edge of the region.

15. The method of claim 9, wherein pixels within the 4×4 pixel block are selectively mirrored about one or both of: (i) a horizontal axis passing through a center of the pixel block, and (ii) a vertical axis passing through the center of the pixel block, such that the location for which the decoded value is to be determined is within a particular quadrant of the pixel block, or Wherein pixels within the 4×4 pixel block are selectively unmirrored about both: (i) a horizontal axis passing through a center of the pixel block, and (ii) a vertical axis passing through the center of the pixel block, so that the location where the decoded value is to be determined is within a particular quadrant of the pixel block.

16. The method of claim 1 or 2, wherein the second set of bits comprises: a first portion indicating a sign of each of the one or more of the differential coefficients that are non-zero; and A second portion indicates an exponent of each of the one or more of the differential coefficients that are non-zero.

17. A method as claimed in claim 16, wherein if one or more of the differential coefficients is zero, the first part indicates the mantissa bits of the differential coefficient having the maximum magnitude, wherein the result of determining the weighted sum of the differential coefficients and the average coefficient of the 2×2 pixel quadruplet includes two terms in the weighted sum of the differential coefficients having the maximum magnitude, the two terms being: (i) the weight of the differential coefficient shifted according to the exponent of the differential coefficient, and (ii) the weight of the differential coefficient shifted according to an amount less than one of the exponents of the differential coefficient.

18. The method of claim 1 or 2, wherein the second set of bits comprises a single portion indicating a sign and an exponent of each of the one or more of the differential coefficients that are non-zero.

19. The method of claim 1 or 2, wherein the plurality of differential coefficients of the 2×2 pixel quadruplet include three directional differential coefficients respectively indicating horizontal differential, vertical differential, and diagonal differential in the 2×2 pixel quadruplet.

20. The method of claim 1 or 2, wherein each pixel in the pixel block has values ​​in multiple channels, and the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad in the compressed data structure are represented as a combination of differential channel coefficients determined for the corresponding channels, Wherein said determining the sign and exponent of said one or more of said differential coefficients that are non-zero using said first portion of said compressed data structure comprises determining the sign and exponent of one or more differential channel coefficients for each of said channels.

21. A method as claimed in claim 20, wherein the third part of the compressed data structure indicates a combination pattern from a set of predetermined combination patterns, according to which the differential channel coefficients determined for the channels are combined to determine the encoded differential coefficients of the Haar coefficients of the 2×2 pixel quad, and wherein the method includes using the third part of the compressed data structure to identify the combination pattern, and using the identified combination pattern to determine the differential channel coefficients for each of the channels.

22. The method of claim 20, wherein the plurality of channels comprises a red channel, a green channel, and a blue channel, and wherein each pixel in the pixel block has an additional value in an alpha channel, wherein the method further comprises decoding the alpha channel into the red channel, the green channel, and the blue channel, respectively.

23. The method of claim 1 or 2, wherein the pixel block is a 4×4 pixel block comprising four 2×2 pixel quadruplets, wherein the compressed data structure includes a first portion of each of the 2×2 pixel quads, the first portion being determined using differential coefficients of the 2×2 pixel quads; wherein the second portion of the compressed data structure encodes a set of L1 Haar coefficients of a 2×2 coefficient quadruple represented by the average coefficient of each 2×2 pixel quadruple of the 2×2 pixel quadruple, wherein the set of L1 Haar coefficients includes a plurality of L1 differential coefficients and an L1 average coefficient, wherein a first sub-portion of the second portion of the compressed data structure encodes the L1 differential coefficients of the 2×2 coefficient quad and comprises: (i) a first set of bits indicating an order of magnitudes of the L1 differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the L1 differential coefficients that are non-zero, wherein a second sub-portion of the second portion of the compressed data structure encodes the L1-averaged coefficients of the 2×2 coefficient quad, The method comprises: determining signs and exponents of one or more of the non-zero differential coefficients for each of the 2x2 pixel quads using the first portion of the compressed data structure for each of the 2x2 pixel quads; and Determining a representation of the average coefficient for each of the 2×2 pixel quads using the second portion of the compressed data structure by: determining signs and exponents of the one or more of the L1 differential coefficients that are non-zero using the first sub-portion of the second portion of the compressed data structure; determining a representation of the L1-averaged coefficients of the 2x2 coefficient quad using the second sub-portion of the second portion of the compressed data structure; and determining a result of a weighted sum of the L1 differential coefficients and the L1 average coefficients of the 2×2 coefficient quadruple using: (i) the determined signs and exponents of the one or more of the L1 differential coefficients that are non-zero, (ii) the determined representation of the L1 average coefficient, and (iii) the corresponding weights of the L1 differential coefficients; wherein, for each of the 2×2 pixel quads, the respective weights of the L1 differential coefficients are set such that the determined result of the weighted sum provides the determined representation of the average coefficient of the 2×2 pixel quad.

24. The method of claim 1 or 2, wherein the using the determined result to determine the decoded value comprises: determining an increment value using the determined result; and A prediction value is added to the delta value to determine the decoded value of the pixel block, wherein the prediction value is determined based on one or more neighboring pixel blocks.

25. A decoding unit, the decoding unit being configured to decode a compressed data structure to determine a decoded value, wherein the compressed data structure encodes a set of Haar coefficients for a 2×2 pixel quad of a pixel block, wherein the set of Haar coefficients comprises a plurality of differential coefficients and an average coefficient, wherein the first part of the compressed data structure encodes the differential coefficients of the 2×2 pixel quad and comprises: (i) a first set of bits indicating an order of magnitudes of the differential coefficients, and (ii) a second set of bits indicating a sign and an exponent of each of one or more of the differential coefficients that are non-zero, wherein the second portion of the compressed data structure encodes the average coefficient of the 2×2 pixel quad, The decoding unit comprises: means for determining signs and exponents of said one or more of said differential coefficients that are non-zero using said first portion of said compressed data structure; means for determining a representation of said average coefficient using said second portion of said compressed data structure; means for determining a result of a weighted sum of the differential coefficients and the average coefficients for the 2×2 pixel quadruplet using: (i) the determined signs and exponents of the one or more of the differential coefficients that are non-zero, (ii) the determined representation of the average coefficient, and (iii) the corresponding weights of the differential coefficients; means for determining said decoded value using the determined result; and A device for outputting the determined decoded value.

26. The decoding unit of claim 25, further comprising a filter parameter setting module configured to determine the corresponding weights of the differential coefficients using a transfer function based on the position relative to the 2×2 pixel quadruplet to model the outputs of different inputs.

27. The decoding unit of claim 26, wherein the pixel block is a 4×4 pixel block comprising four 2×2 pixel quadruplets, and the decoding unit further comprises: four decoder instances, each of the four decoder instances being configured to determine, for a corresponding one of the four 2×2 pixel quads, a result of a weighted sum of the differential coefficients and the average coefficients of the 2×2 pixel quad; and A bilinear interpolation unit is configured to determine the decoded value of the pixel block by combining the four determined results of the corresponding four 2×2 pixel quadruplets by performing bilinear interpolation on the four determined results.

28. A non-transitory computer-readable storage medium having a computer-readable code stored thereon, the computer-readable code being configured to cause the method of claim 1 or 2 to be executed when the code is executed.

Images