SYSTEM AND / OR METHOD IMPLEMENTING A STATISTICAL APPROACH TO TEXTURE COMPRESSION - Patent application

Abstract

To provide a texture compression method, system, and storage medium that produce lossy encoded textures suitable for use with subsequent lossless compression in accordance with Rate Distortion Optimization.SOLUTION: A method includes: encoding a plurality of blocks obtained by dividing a texture into a plurality of block bitstreams; selecting the block bitstreams having the lowest distortion values such that a predetermined number of block bitstreams for each block are selected; acquiring, for each selected block bitstream, data chunks such that each data chunk is a continuous region of the block bitstream; detecting collisions among the acquired data chunks; forming, on the basis of the detected collisions, match chains with redundant data chunks such that the redundant data chunk therein represent a common value of the block bitstream; selecting the best match chain, and acquiring a lossy encoding of the texture from the selected best match chains.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of 63 / 465,672, filed May 11, 2023, the entire contents of which are incorporated herein by reference. Certain exemplary embodiments described herein relate to texture compression that may be used in three-dimensional virtual environments, etc. More particularly, certain exemplary embodiments described herein relate to statistical techniques for texture compression that result in lossy encoded textures suitable for use in subsequent lossless compression according to rate-distortion optimization (RDO). [Background technology]

[0002] Textures are one of the most important assets in a virtual environment, such as those used in video games. A texture is a material that acts as a layer covering the surface of a three-dimensional (3D) object to give it a desired appearance in the virtual environment (e.g., a desired in-game appearance). Most textures are stored as files containing a two-dimensional (2D) array of pixels, sometimes called texels. The process of mapping these pixels onto an object is called texture mapping. Summary of the Invention

[0003] Figure 1 illustrates wrapping a three-dimensional object with a two-dimensional texture to produce a texture-mapped object. More specifically, Figure 1 illustrates a three-dimensional spherical object 102 and a two-dimensional texture 104 that is a world map. Texture mapping causes the texture-mapped object 106 to appear as a sphere because the two-dimensional texture 104 is wrapped around the three-dimensional spherical object 102. For example, the example of Figure 1 takes into account lighting effects included in the three-dimensional object 102, as can be seen in the generated texture-mapped object 106.

[0004] Textures differ in the type of data they store. A "normal map," for example, stores the X, Y, and Z coordinates of a surface normal (a vector perpendicular to the object's local surface). These components can be used to calculate how light reflects off different areas of an object, providing a sense of greater detail and therefore increased "realism." "Realism" in this sense can reflect a more immersive experience of a real-world environment, or, for example, a distinctly unrealistic setting (such as some virtual game environments). Specular maps, on the other hand, typically store grayscale values ​​that can be used to control the shininess of an object. In such cases, the higher the value, the more reflective the area. Other common types of textures include diffuse maps, albedo maps, displacement maps, light maps, ambient occlusion, etc.

[0005] Texture data is generally not stored "as is." In other words, textures used in a virtual environment, such as a game world, are typically not stored in a format that is directly usable in such an environment. Instead, texture data is typically stored in a lossy compressed format. Storing texture data in a compressed format provides several technical advantages, such as, for example, a reduced amount of required GPU memory or other memory and reduced memory bandwidth usage, compared to, for example, "raw" texture data.

[0006] Compression formats used for textures differ from those used in image compression formats (such as JPEG) and video formats (such as H.264). One difference is that compression formats used for textures typically encode blocks of pixels independently, producing a fixed-size output for each block. For example, the BC1 format used to compress textures encodes 4x4 blocks of pixels and outputs an 8-byte bitstream per block. In this way, each pixel is encoded with a fixed number of bits (or bits per pixel (BPP)), which is well-suited for random GPU access. That is, to access a particular pixel, the GPU calculates the address of the block to which it belongs. The GPU then decodes the block on the fly to obtain the pixel data. Because the bitrate is fixed, such formats generally result in lower compression ratios compared to variable bitrate formats. Common and widely used texture compression formats include block compression (BC) formats such as BC1, BC3, BC4, BC5, BC7, and Adaptive Scalable Texture Compression (ASTC), with typical bit rates for such formats ranging from 8 BPP to 1 BPP.

[0007] Texture-related assets typically make up a large portion of a game's total storage size. In many situations, limiting their size is crucial. For example, some games are stored on physical media such as cartridges. Smaller games can use smaller media (e.g., smaller cartridges), which can result in lower manufacturing costs and more flexibility in the types of media that can be used. As another example, downloading a digital game can use less server bandwidth, require fewer computing resources on both the server and client side, and potentially reduce maintenance requirements on the server side.

[0008] There are various ways to control texture size. One approach involves creating smaller textures when possible. For example, a texture with a resolution of 2048x2048 pixels may actually only have a "useful area" of 1024x1024 because, for example, data outside this area may not be fetched by the GPU (e.g., a game or other application may not use it). In this case, data outside the useful area is never rendered on the screen and is not used in any rendering operations. In such cases, there may be no need to store such data. In the same vein, some channels (e.g., color channels, coordinate channels, etc.) may be redundant with other channels, or again, may not be accessed at all by the GPU.

[0009] Another common technique that can be combined with the techniques mentioned above is to store (compressed) texture files in a lossless compression format. In other words, a lossy compressed texture file can be further compressed using lossless compression. This is sometimes referred to as "ultra-compression." Lossless formats are often quickly decompressed, which typically adds little or negligible overhead to load time (in this case, the time it takes for the texture to migrate from storage memory (e.g., cartridge, SSD, or other location) to GPU memory (e.g., VRAM) or other memory location). Lossless compression formats (e.g., LZ4 and ZStandard) offer a technically "interesting" tradeoff between compression ratio and decompression speed.

[0010] While various compression techniques (including ultra-compression and other techniques) are widely used for virtual environments (including virtual environments provided in connection with games), further improvements are still possible. For example, it would be desirable to better manage the tradeoff between the amount of compression and quality while potentially allowing for reasonable compression and decompression times.

[0011] Certain exemplary embodiments improve texture compression techniques, for example, by providing improvements to texture compression tools. For example, certain exemplary embodiments provide a statistical approach for texture compression that results in lossy encoded textures suitable for use in subsequent lossless compression according to rate-distortion optimization (RDO).

[0012] One aspect of an exemplary embodiment relates to providing a "generic" encoding technique (including a "generic" RDO algorithm) for compression (including texture compression). The encoding technique includes an RDO algorithm that is generic in the sense that the bitstream details of the format used to encode the texture do not need to be known in advance, meaning that it is almost completely independent of the underlying encoder (e.g., the underlying texture encoder). The encoding technique includes the following: First, each block in the texture is encoded in a number of different ways (e.g., using an optimal compression format, such as BC1, ASTC, etc.). The generic RDO algorithm looks for collisions, post-encoding, among all encoding configurations of all blocks, recognizing that from a statistical perspective, such collisions are likely to occur quite frequently and can be exploited in the overall compression technique.

[0013] Another aspect of certain example embodiments relates to decoupling the generation of block encoding configurations from downstream processing, so that the algorithm of certain example embodiments is parallelizable and can rapidly handle many different tradeoffs between quality and size (as controlled by hyperparameters).

[0014] Another aspect of certain exemplary embodiments relates to the ability to advantageously use a wide variety of lossy compression formats. Indeed, due to the comprehensiveness described herein, the algorithms of certain exemplary embodiments can perform RDO using compression standards such as ASTC, where the bitstream is significantly more complex.

[0015] In one exemplary embodiment, a method for encoding a texture is provided. The texture is read from a data store. The texture is divided into a plurality of blocks. Each of the blocks is encoded into a plurality of block bitstreams, each of which corresponds to a respective encoding configuration and has an associated distortion value. For each block, a block bitstream with the smallest associated distortion value is selected, such that up to a predetermined number of distinct block bitstreams are selected. For each selected block bitstream, a plurality of data chunks are obtained, such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained. The data chunks for each selected block bitstream begin at a respective position in the respective selected block bitstream and extend to an end of the respective selected block bitstream, such that each data chunk has a minimum data chunk size. Data chunks of different sizes can be obtained for the selected bitstreams. Collisions between the obtained data chunks are detected. Based on the detected collisions, a match chain is formed for each match chain with redundant data chunks, such that the redundant data chunks represent a common truncated value of the associated block bitstreams. A best match chain is iteratively selected from the formed match chains, the best match chain is determined using hyperparameters, and a lossy encoding of the texture is obtained from the iteratively selected best match chain.

[0016] In one example embodiment, the lossy encoded texture may be encoded using a lossless compressor.

[0017] In one exemplary embodiment, the data chunks may start at different byte locations.

[0018] In one example embodiment, for each block, as many distinct block bitstreams as possible may be selected, up to a predetermined number. An indicator may be provided for each block for which the number of selected block bitstreams is less than the predetermined number.

[0019] In an example embodiment, an input may be received that identifies a maximum distortion value for one or more identified blocks of the plurality of blocks, and for each of the one or more identified blocks, only block bitstreams having a distortion value better than the maximum distortion value may be selectable for the respective identified block.

[0020] In one example embodiment, the minimum data chunk size may be at least as large as the minimum size available for a lossless compressor to apply lossy encoding of the texture.

[0021] In an example embodiment, obtaining the data chunks may comprise extracting a subset of valid data chunks for a given selected block bitstream, and generating residual data chunks for the given selected block bitstream from the extracted subset.

[0022] In one example embodiment, the hyperparameters may trade off rate or distortion for the other and may be part of a defined function that determines which match chain is best.

[0023] In one example embodiment, the texture may be divided into slices, each of which contains a predetermined number of blocks, and the slices may be treated as individual textures that are separately divided into blocks.

[0024] In an example embodiment, match chains determined to be unselectable in the iterative selection may not be considered for forming match chains. For example, a given match chain may be discarded in response to a determination that another match chain has been formed or will be formed that is equal to or better than the given match chain. Application of a function according to hyperparameters to the match chains may allow different match chains to be compared to each other, and / or iterative selection may select the best match chain.

[0025] In one exemplary embodiment, a method for encoding a texture is provided. Each of multiple blocks obtained by dividing the texture is encoded into multiple block bitstreams, each block bitstream corresponding to a respective encoding configuration and having an associated distortion value. For each block, a block bitstream with the smallest associated distortion value is selected, such that up to a predetermined number of distinct block bitstreams are selected. For each selected block bitstream, multiple data chunks are obtained, such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained. Data chunks of different sizes can be obtained for the selected bitstreams. Collisions between the obtained data chunks are detected. Based on the detected collisions, a match chain is formed for each match chain with redundant data chunks, such that the redundant data chunks therein represent common values ​​of the associated block bitstreams. A best match chain is selected from the formed match chains. The best match chain is determined using a hyperparameter. A lossy encoding of the texture is obtained from the selected best match chain.

[0026] In one example embodiment, a texture encoding system is provided, where a texture is stored in a data store, a memory is provided, and at least one processor is configured to perform operations including: reading the texture from the data store; dividing the texture into a plurality of blocks; and encoding each of the blocks into a plurality of block bitstreams, each block bitstream corresponding to a respective encoding configuration and having a distortion value associated with each of the block bitstreams; selecting, for each block, a block bitstream having a lowest associated distortion value, such that, for each block, up to a predetermined number of distinct block bitstreams are selected; and, for each selected block bitstream, obtaining a plurality of data chunks, each data chunk being a contiguous section of the respective selected block bitstream from which it was obtained, each selected block bitstream the data chunks for the texture start from a respective position in a respective selected block bitstream and extend to an end of the respective selected block bitstream such that each data chunk has a minimum data chunk size, data chunks of different sizes are obtainable for the selected bitstreams, the operations further comprising: detecting collisions between the obtained data chunks; forming, for each match chain, match chains having redundant data chunks based on the detected collisions, such that the redundant data chunks therein represent common truncated values ​​of the associated block bitstreams; and iteratively selecting a best match chain from the formed match chains, the best match chain being determined using a hyperparameter; and the operations further comprising obtaining a lossy encoding of the texture from the iteratively selected best match chain. In one example embodiment, the lossy encoded texture may be encoded using a lossless compressor.

[0027] In one exemplary embodiment, the data chunks may start at different byte locations.

[0028] In one example embodiment, for each block, as many distinct block bitstreams as possible may be selected, up to a predetermined number. An indicator may be provided for each block for which the number of selected block bitstreams is less than the predetermined number.

[0029] In an example embodiment, an input may be received that identifies a maximum distortion value for one or more identified blocks of the plurality of blocks, and for each of the one or more identified blocks, only block bitstreams having a distortion value better than the maximum distortion value may be selectable for the respective identified block.

[0030] In an example embodiment, obtaining the data chunks may comprise extracting a subset of valid data chunks for a given selected block bitstream, and generating residual data chunks for the given selected block bitstream from the extracted subset.

[0031] In one exemplary embodiment, a method for encoding a texture is provided. Each of a plurality of blocks obtained by dividing the texture is encoded into a plurality of block bitstreams. Up to a predetermined number of separate block bitstreams are selected for each block. For each selected block bitstream, a plurality of data chunks are obtained, such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained. Collisions between the obtained data chunks are detected. Based on the detected collisions, a match chain having redundant data chunks is formed for each match chain, such that the redundant data chunks therein represent common values ​​of the associated block bitstreams. A set of match chains is iteratively selected from the formed match chains based on hyperparameters. At least some potential match chains are determined to be unselectable in the iterative selection and are not considered when forming the match chains. A lossy encoding of the texture is obtained from the selected set of match chains.

[0032] In an example embodiment, a non-transitory computer-readable storage medium is provided that stores instructions that, when executed by a processor, cause a computer to perform operations including: encoding each of a plurality of blocks obtained by dividing a texture into a plurality of block bitstreams; selecting up to a predetermined number of distinct block bitstreams for each block; obtaining, for each selected block bitstream, a plurality of data chunks such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained; detecting collisions among the obtained data chunks; forming, for each match chain based on the detected collisions, match chains having redundant data chunks therein such that the redundant data chunks represent common values ​​of associated block bitstreams; iteratively selecting a set of match chains from the formed match chains based on hyperparameters, wherein at least some potential match chains are determined to be unselectable in the iterative selection and are not considered in forming the match chains; and obtaining a lossy encoding of the texture from the selected set of match chains.

[0033] In one example embodiment, a texture encoding system is provided. A texture is stored in a data store. A memory is provided. At least one processor is configured to perform operations including: selecting up to a predetermined number of distinct block bitstreams for each block; for each selected block bitstream, obtaining a plurality of data chunks, such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained; detecting collisions between the obtained data chunks; and, based on the detected collisions, forming match chains with redundant data chunks for each match chain, such that the redundant data chunks therein represent common values ​​of the associated block bitstreams; iteratively selecting a set of match chains from the formed match chains based on hyperparameters, wherein at least some potential match chains are determined to be unselectable in the iterative selection and are not considered in forming the match chains; and obtaining a lossy encoding of the texture from the selected set of match chains.

[0034] In an example embodiment, a given match chain may not be considered in response to a determination that another match chain has been or will be formed that is equal to or better than the given match chain.

[0035] In one example embodiment, application of a function of hyperparameters to match chains may allow different match chains to be compared to each other.

[0036] In an example embodiment, iterative selection may select the best match chain. In one example embodiment, a lossless compressor may be used to encode the lossy encoded texture.

[0037] In one example embodiment, the block bitstream with the smallest associated distortion value may be selected for each block such that up to a predetermined number of distinct block bitstreams are selected.

[0038] In an example embodiment, it may be possible to obtain data chunks of different sizes for a selected bitstream.

[0039] In one example embodiment, a method for providing a virtual environment in connection with a computing system is provided, wherein an encoded texture is read from a non-transitory computer-readable storage medium, the texture being encoded using techniques (e.g., methods) described herein, and the encoded texture is provided to at least one processor of the computing system for decoding the texture and using the decoded texture in the virtual environment.

[0040] In an example embodiment, a non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause a computer to perform operations corresponding to the techniques (eg, methods) described herein.

[0041] In one example embodiment, a computing system is provided through which a virtual environment can be displayed. The system includes a memory coupled to one or more processors configured to perform operations comprising reading an encoded texture from a non-transitory computer-readable storage medium, the texture being encoded using a technique (e.g., a method) disclosed herein, and providing the encoded texture to at least one of the at least one processor of the computing system for decoding the texture and using the decoded texture in the virtual environment.

[0042] The features, aspects, advantages, and exemplary embodiments described herein may be used separately and / or applied in various combinations to achieve still further embodiments of the present invention.

[0043] These and other features and advantages may be more fully and completely understood by reference to the following detailed description of exemplary illustrative embodiments taken in conjunction with the drawings. [Brief explanation of the drawings]

[0044] [Figure 1] Shows a 3D object wrapped with a 2D texture to produce the object after texture mapping. [Figure 2] Three encodings of the same texture are shown, along with the size and peak signal-to-noise ratio (PSNR) for each. [Figure 3] 10 is an example illustrating extraction of different encoding configurations according to an example embodiment. [Figure 4] 10 illustrates an extraction of chunks from an exemplary encoding configuration, according to an exemplary embodiment. [Figure 5] 10 illustrates a numerical example of match chain identification, according to an example embodiment. [Figure 6] 10 helps illustrate how optimal match chains are created in one exemplary embodiment. [Figure 7] 10 helps illustrate how optimal match chains are created in one exemplary embodiment. [Figure 8] 10 helps illustrate how optimal match chains are created in one exemplary embodiment. [Figure 9] 10 illustrates one iteration of a match chain resolution procedure of an example embodiment. [Figure 10] 10A and 10B illustrate a schematic representation of a texture divided into four slices according to the texture slicing technique of an example embodiment. [Figure 11]In connection with one exemplary embodiment, it helps illustrate when a match chain can be broken. [Figure 12] 1 illustrates an example of an algorithmic factorization in accordance with an example embodiment. [Figure 13] 10 is a flowchart illustrating a process for generating an encoded texture according to an example embodiment. [Figure 14] 14 is a block diagram illustrating a computing architecture that may be used in connection with the example flowchart of FIG. 13, according to an example embodiment. [Figure 15] 1 is a flowchart illustrating a process for decoding texture encoded via the techniques described herein, in accordance with an example embodiment. [Figure 16] 16 is a block diagram illustrating a computing device that may be used in connection with the example flowchart of FIG. 15, according to an example embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0045] Certain exemplary embodiments relate to texture compression techniques, and more particularly to systems and / or methods for texture compression suitable for use in virtual worlds such as those used in video games and the like.

[0046] Texture encoding is the process of applying a texture encoder to a texture. The texture encoder encodes the texture by processing the pixels into discrete blocks (typically 4x4 square windows) and outputs a bitstream for each of these blocks (typically 8 or 16 bytes, depending on the format). The bitstream is the binary sequence that the texture encoder produces as a result of encoding the texture. It is the binary sequence that you get after encoding a single block of pixels from the texture, or after encoding the entire texture.

[0047] Compression and encoding are related and may be used interchangeably in some situations. Encoding data generally refers to the process of changing one data representation into another. For example, the value "15" can be encoded in hexadecimal as "0xF." Compressing data, on the other hand, means effectively reducing the size of the data. Encoding data often makes it smaller. Therefore, from this perspective, encoding can be considered a form of compression, but it does not have to be. Encoding and compression techniques can be lossy or lossless. Lossless compression (encoding) means that the compressed (encoded) data can be decompressed (decoded) to allow perfect reconstruction of the original data. Common lossless compressors include LZMA, ZStandard, and LZ4. Texture encoders are generally not lossless. The PNG format is an example of lossless encoding followed by lossless compression.

[0048] The inventors recognized that texture encoders typically have multiple ways to encode a given block of pixels, often with very little additional cost in terms of quality. This is because such texture compression formats use a fixed bit rate per block, which is often unnecessarily high given the complexity of the block being encoded. Second, the inventors recognized that lossless compressors generally achieve better compression when more redundancy appears in the data. Thus, by carefully choosing the encoding configuration of texture blocks, it is possible to maintain high texture quality while achieving a good compression ratio. Certain exemplary embodiments provide the advantageous technical effects of high quality and good compression for texture compression techniques.

[0049] One exemplary embodiment uses a rate-distortion optimization (RDO) technique. RDO is a technique used in lossy image / video compression to jointly optimize two criteria in the encoding process: rate (size of data) and distortion (quality of the asset). Distortion is generally measured using a metric. A smaller rate generally means higher distortion and therefore a desire for optimization.

[0050] As applied to texture compression, RDO encodes textures not by optimizing quality alone, but rather by optimizing both the texture's size (which should be understood as the size after compression by a lossless compressor) and its quality. These two "axes" are the rate and distortion mentioned earlier. The optimization process can be represented by hyperparameters, which represent a tradeoff between quality and memory. Changing the hyperparameter value favors one of the two axes over the other, resulting in a texture with better quality at the expense of more memory, or a texture with lower quality but smaller size. For example, depending on the cost function, a larger value of the hyperparameter can mean that rate is prioritized over distortion. In the context of texture compression, where the encoder produces a fixed-size bitstream (fixed rate), the rate can be influenced by utilizing a lossless compressor.

[0051] In the context of RDO, "rate" refers to the size of the data after encoding / compressing an asset. In contrast, in the context of texture encoding using RDO, an attempt is made to estimate the rate at the level of the texture block. Thus, the rate implies how much storage size savings are expected when choosing a given encoding configuration for a given block after applying lossless compression to the texture. As a specific example, selecting encoding configuration number 3 for block number 24 can be estimated to reduce the texture size by two units (e.g., bytes), so the rate is 2. In the context of RDO, "distortion" refers to the quality of the asset after encoding / compression, with the idea being that the greater the distortion, the worse the asset quality. In the context of texture encoding using RDO, distortion is measured at the level of the texture block. Thus, the distortion implies how much quality loss occurs as a result of choosing a given encoding configuration for a given block. As a specific example, selecting encoding configuration number 3 for block number 24 can be estimated to reduce the block quality by two units (e.g., decibels), so the distortion is 2. Further details regarding rate and distortion in the context of RDO of an example embodiment are provided below.

[0052] Figure 2 shows three encodings of the same texture, along with their size and peak signal-to-noise ratio (PSNR). PSNR is a widely used metric for evaluating image quality. The unit of PSNR is decibels (dB). The higher the PSNR value, the higher the overall quality of the texture. In Figure 2, the top encoding is a standard block compression using BC1 encoding (a lossy format) without RDO, while the next two encodings are BC1 encoding with RDO using different values ​​for the hyperparameter (called lambda). As the PSNR indicates, the quality of the textures after RDO encoding is lower. However, their size is smaller after compression using the lossless ZStandard format (denoted as Zstd in Figure 2). It is acknowledged that the images, formats, and figures are provided for illustrative purposes only. These and / or other lossy and / or lossless encoders may be used in conjunction with these and / or other images / textures, with these and / or other lambda values, which may result in different metrics (as measured by PSNR or some other metric) and / or different size reduction.

[0053] As described in more detail below, an exemplary embodiment provides a statistical approach to RDO in conjunction with a texture collision algorithm. The texture collision algorithm of an exemplary embodiment is advantageous in several ways. First, the approach of an exemplary embodiment uses a statistical principle known as the Birthday Paradox. This is a statistical result that repeatedly generating values ​​from variables leads to "collisions" (previously obtained values) much faster than common sense would suggest would happen, hence the paradox. For texture compression, an exemplary embodiment extracts a large number of encoding configurations for a given block and exploits the high collision frequency obtained with surrounding blocks that themselves have a large number of possible encoding configurations. As alluded to above, an encoding configuration is a possible valid way to encode a block of pixels (i.e., to produce a valid bitstream for this block), given a texture encoder. The algorithmic approach is almost completely independent of the underlying texture encoder. That is, once the algorithm has a "catalog" of encoding configurations for each block, it can work with only very basic information about the encoder, such as block bitstream size and block dimensions. More precisely, an exemplary embodiment does not need to know details about block bitstream structure, such as headers, colors, weights, and delimiters, whereas a traditional RDO approach would require precise description of this and / or other information. Thus, the approach of an exemplary embodiment paves the way for compatibility with a wide range of lossy encoders / texture formats, including (for example) ASTC, which has been a hurdle for many encoders for many years. Furthermore, the algorithmic approach of an exemplary embodiment can essentially find matches with completely unrelated data. A "match" in the context of traditional lossless compression is defined as a collision (repetition) of some chunk of data in the source being compressed.A lossless compressor exploits such redundancy, for example, by replacing one or more of the duplicated chunks with a reference to another chunk, resulting in smaller data. Unlike some conventional approaches, an exemplary embodiment, as noted, can find matches with completely unrelated data. For example, it can match the color of one block with the weight of another block, provided it determines that doing so is "interesting" or useful.

[0054] A second advantage relates to decoupling the texture encoder. The texture encoder of an example embodiment is not "RDO-aware" in the sense that it extracts multiple encoding configurations for each texture block. This is in contrast to conventional encoders, which typically select only the best-quality configuration. That is, an example embodiment allows for decoupling the generation of block encoding configurations from other downstream operations in the encoding pipeline. In this way, the approach of an example embodiment is easily parallelizable and can quickly handle several different tradeoffs between quality and size, i.e., several different values ​​of the hyperparameters.

[0055] The third advantage below relates to (lambda) factorization. The decoupling between texture encoding and RDO makes it possible to "factorize" the texture encoding, resulting in multiple RDO-encoded versions of a texture, each with different hyperparameter values ​​(and therefore different quality-compression tradeoffs). As will be shown in more detail below, it becomes possible to factorize most of the core steps of the texture collision algorithm of an example embodiment, thereby minimizing the lambda-dependent parts. In other words, it becomes possible to output multiple versions of an encoded texture with very little additional time compared to outputting a single texture.

[0056] The birthday paradox in texture compression This section helps illustrate the birthday paradox as it applies to texture compression. Suppose K random numbers between 1 and N are chosen (with equal probability and with replacement, meaning that the same number will be chosen several times). Then the expected collision frequency E (the average number of times a duplicate number will be chosen) is given by:

[0057]

number

[0058] Assume a 256x256 pixel texture is encoded in ASTC format with a block size of 4x4 pixels. This corresponds to 4,096 blocks. In ASTC format, each block is encoded with 16 bytes. For this example, we count the expected number of 4-byte matches that will be present in the encoded texture. Since the encoded size is 65,536 bytes (4,096 x 16 bytes), this represents a total of 65,533 consecutive 4-byte chunks. Applying the above formula, we determine the expected collision frequency of these chunks, or how many chunks are expected to have the same value as at least one other chunk. In particular, applying the above formula with N = 2^32 (the range of integer values ​​that a 4-byte chunk can take) and K = 65,533 (the number of 4-byte chunks) shows that E = 0.9988 ≈ 1.

[0059] Now, suppose the texture is "enlarged" to 1,024 times its original size. This corresponds to a texture of 8,192 x 8,192 pixels, or 4,194,304 blocks of 4 x 4 pixels. When E is calculated on the encoded texture, N still equals 2^32, but now K is 6,7108,861, which gives E = 1,040,426.

[0060] In summary, by multiplying the texture size by 1,024, the expected collision frequency is now multiplied by over 1 million.

[0061] As a general matter, actually increasing the texture size / texture dimensions is not really useful. However, a similar effect can be achieved in a different way: if we can extract multiple encoding configurations for each block, it will be possible to recreate conditions similar to the illustrated scenario while still working with the original texture dimensions. Essentially, this is how one exemplary embodiment proceeds.

[0062] Note that this example has been simplified in many ways. In reality, not all 4-byte values ​​are achievable possibilities, since some would correspond to unrelated (and sometimes invalid) bitstreams. Furthermore, textures typically exhibit strong spatial correlation patterns in some areas, which would result in a significant increase in the probability of a match in such regions. This example and the formula for E assume uncorrelated values. Typical supercompression ratios for ASTC-encoded textures (defined as the ratio between the size of the ASTC-encoded texture and the size of the ASTC-encoded and losslessly compressed texture) range from 1 to 1.3, which in practice tends to indicate a higher collision frequency than calculated. Thus, practical reality may enable the techniques disclosed herein to perform even better than a strict mathematical interpretation of the Birthday Paradox would suggest.

[0063] Texture Collision Algorithm of an Example Embodiment The texture collision algorithm of an example embodiment can be decomposed into different (conceptual) stages, each of which is described in more detail below.

[0064] Stage 1: Multi-component texture encoding As this first stage, the texture encoder encodes the texture and returns, for each block, not only the best encoding configuration (i.e., the block bitstream), but rather a list of the top N distinct best encoding configurations. In one exemplary embodiment, "best" should be understood to mean least distortion. Typical values ​​of N can range, for example, from 128 to 4,096, depending on the texture encoder at hand as well as the memory consumption required to store these configurations. The "Texture Slices" section below provides some considerations regarding memory consumption. While these values ​​of N have been found to work well, different implementations may use larger or smaller values.

[0065] In addition to the bitstream, a distortion value is stored for each of these encoding configurations. The "Rate and Distortion" section below provides a discussion of examples of possible metrics that may be used in an exemplary implementation.

[0066] It may not be possible to completely fill the list for some blocks. One example where this can happen is when some regions in a texture need to be treated with special care. For example, textures often have completely flat regions, and any visual artifacts introduced in such regions are generally very noticeable and therefore rarely desired. In such cases, it may be desirable to simply maintain configurations with the least distortion. More generally, for some blocks, it may be desirable to explicitly control the distortion value and therefore the number of encoding configurations. In any case, if the encoder would not return the maximum number of configurations it is allowed to obtain, an indicator may be provided in the storage buffer immediately after the last valid configuration to indicate the end of the list.

[0067] Figure 3 is an example illustrating extraction of different encoding configurations according to an exemplary embodiment. In Figure 3, area 302 is a texture divided into M texture blocks, and the encoding configurations for these blocks are represented by gray boxes with bent corners. Each block has N possible encoding configurations. i,j is the block number i(b i ) refers to the encoding configuration number j for d i,j is the distortion value associated with this encoding structure. "END" is a marker that indicates the end of the encoding structure if the list is not completely filled. For a given block b i For , the encoding configurations are sorted by increasing distortion (d i,1 ≦d i,2 ≦ ≦ d i,N ).

[0068] Stage 2: Chunk Extraction This stage extracts consecutive groups of bytes (also called chunks) from the bitstream collected through the previous stage. For a given bitstream, chunks are extracted starting from every byte position and extending to the end of the bitstream. In one exemplary embodiment, the size of each chunk is greater than a given value specified as a modifiable parameter, MIN_MATCH_SIZE.

[0069] To help understand the operation of this stage, consider the following example where the texture encoder is BC1. BC1 produces a bitstream of 8 bytes for each block. In this example, MIN_MATCH_SIZE is set to 4. Continuing with this illustration, FIG. 4 shows the extraction of chunks from an exemplary encoding configuration, according to an exemplary embodiment. As can be seen from FIG. 4, five chunks ranging in size from 4 bytes to 8 bytes are extracted from the BC1 bitstream. For each of these extracted chunks, additional information elements that may be useful for subsequent stages are also stored. These additional information elements include the encoding configuration index (j in FIG. 4), the block index (i in FIG. 4), and the starting position of the chunk in the bitstream (0 means that the chunk starts at the first byte).

[0070] There is no need to extract more chunks, because from these five chunks it is possible to obtain all values ​​in the bitstream by removing some of the terminating bytes from these chunks (e.g., via a masking process). For example, by taking a 7-byte chunk (with values ​​v2v3v4v5v6v7v8) and masking its last 3 bytes (v6, v7, v8), one can obtain the value v2v3v4v5.

[0071] The value of MIN_MATCH_SIZE may vary in different exemplary embodiments. For example, it is recognized that MIN_MATCH_SIZE may depend on the lossless compressor used. Typically, LZ4 does not consider matches truly smaller than 4 bytes, so extracting chunks of this size would be counterproductive. It is desirable to create redundancy in the final texture bitstream so that the lossless compressor can compress the texture more efficiently. There is little point in looking for a 3-byte match if it is known or expected that the compressor will not see 3 bytes. ZStandard can consider 3-byte matches, but efficiency is not guaranteed. In fact, most lossless compressors actually begin to utilize matches starting at 4-byte size, so that may be the default value used in one exemplary embodiment.

[0072] MAX_MATCH_SIZE is defined to be the maximum match size that the texture collision algorithm will consider. By default, you may set this to be equal to the bitstream size, since you will not be able to extract chunks larger than this value. You can choose a smaller value. There may be some cases where you have specific reasons to avoid such a match size, so you can lower the value from this default.

[0073] Stage 3: Creating a Matching Chain This stage contains several sub-stages, including sub-stages for collision detection and for creating optimal match chains.

[0074] Stage 3.1: Collision Detection Once the bitstream chunks are extracted for all encoding configurations of all blocks, we can process the chunks by increasing the match size from MIN_MATCH_SIZE to MAX_MATCH_SIZE to detect collisions between them and identify matches.

[0075] For the smallest match size (MIN_MATCH_SIZE), all chunks are sorted by value truncated to MIN_MATCH_SIZE bytes. For example, if MIN_MATCH_SIZE=4, then in the example in Figure 4, the 8-byte chunks would be truncated to v1v2v3v4, the 7-byte chunks would be truncated to v2v3v4v5, and so on. Since these values ​​are integers (32-bit values ​​for 4-byte chunks), an efficient implementation of radix sort may be used to sort the chunk buffer in linear time. This immediately gives duplicate values, and therefore matches.

[0076] It is recognized that other sorting algorithms may be used in different exemplary embodiments. Furthermore, additional optimizations may be implemented to help speed collision detection. For example, to find matches of higher sizes, the following and / or other optimizations are possible: First, only chunks known to have overlaps at the previous chunk size need to be sorted. For example, to sort chunks at a match size of 5 bytes, only chunks that had overlaps at a match size of 4 bytes can be considered, because if a given chunk does not match at 4 bytes, it cannot match at 5 bytes. In practice, this approach has been found to dramatically reduce the number of chunks to be sorted.

[0077] Second, chunks that are not large enough to handle the current match size can be discarded. Referring back to Figure 4 for illustration, a 4-byte chunk cannot be considered for a 5-byte match size because it cannot be "extended" since doing so would put it outside the bitstream data. The start position is used to detect and discard these "unextendable" chunks.

[0078] Third, because all chunks are already sorted by the previous match size, an incremental sorting algorithm can be used to sort the chunks using only additional bytes. In practice, this approach has been shown to also dramatically reduce processing time. For example, assuming the chunk buffer is already sorted by the first four bytes (and assuming the sort is stable), sorting a chunk by five bytes would only involve sorting according to the fifth byte. For example, an efficient radix sort using a single pass can perform this procedure again in linear time.

[0079] A group of overlapping values ​​may be thought of as a match chain. Figure 5 shows a numerical example of match chain identification, according to an example embodiment. The top of Figure 5 shows an extraction of collisions of 4-byte chunks. When moving to higher match sizes (as in the bottom of Figure 5), we see that some chunks still collide at 5 bytes. However, some chunks cannot extend over 5 bytes (chunks marked with a cross on the left) or no longer collide (chunks marked with a cross on the right).

[0080] It is recognized that some conflicting chunks may refer to the same block but use different encoding configurations. An exemplary embodiment operates such that only one encoding configuration per block is ultimately chosen, so that such types of matches are not actually made. When such cases occur, only the chunks from the encoding configuration with the least distortion are kept in the match chain, while the other chunks are removed.

[0081] Stage 3.2: Creating an optimal match chain 6 through 8 help illustrate how optimal match chains are created in an example embodiment. In FIG. 6, the match chain contains all redundant chunks for a given value X, where X is the associated encoding configuration e of index j for block i. i,jThe starting position of the chunk is not needed here, so it is omitted from this representation. However, the distortion (d i,j ) is added. Note that this is the associated encoding configuration for this block (e i,j ) distortion that occurs in the block as a result of choosing

[0082] All these redundant chunks form a chain of length N. However, from this chain we can create smaller match chains with fewer chunks. For example, we can group the first three chunks together, which would also form another chain, as shown in Figure 7.

[0083] As can be seen from Figure 7, there are many ways to form chains. 2. Form a new chain from a chain of length N (assuming that the chain must contain at least two duplicate values). N There are N-1 ways. These newly formed chains all have their own rate and distortion, which makes each of them potentially interesting to consider, but keeping them all may not be feasible, since N can potentially be any value. For example, if N=16, there would be 65,519 chains to remember.

[0084] This number can be reduced due to suboptimality. In one exemplary embodiment, a match chain is treated as suboptimal if another chain exists that exhibits a better tradeoff between rate and distortion. However, if no such chain exists, one exemplary embodiment considers the chain to be the optimal chain.

[0085] To understand this, in Figure 7, we assume that chunks are sorted from left to right with increasing skewness, i.e., d i1,j1 ≦d i2,j2 ≦ ≦ d iN,jN Assume that:

[0086] Now, suppose the chain consisting of the first two chunks in Figure 7 is "applied." Applying the chain in this context means setting encoding configuration j1 ​​to block i1 and encoding configuration j2 to block i2. A lossless compressor will see a value (X) in the texture bitstream that repeats twice, and such redundancy will then result in a certain storage gain R (rate). The quality loss (distortion) caused by choosing these two encoding configurations is the sum of the two distortions, D=d i1,j1 +d i2,j2 This becomes:

[0087] Now consider instead the chain consisting of the second and third chunks in Figure 7. After setting the two encoding configurations j2 and j3 to blocks i2 and i3 respectively, the situation for the lossless compressor is the same as before: it will see the same overlap value (X) and in turn will result in the same rate R. However, the distortion is now D=d i2,j2 +d i3,j3 ≧d i1,j1 +d i2,j2 is.

[0088] Thus, an exemplary embodiment prefers the first chain over the second chain because it has less distortion while maintaining the same rate. In other words, the first chain "outperforms" the second chain. The second chain is therefore considered a suboptimal chain. Based on the above, it can be inferred that in practice there are only N-1 optimal chains: one consisting of the first two chunks, a second consisting of the first three chunks, and so on, up to one consisting of all chunks.

[0089] Note that the use of terms such as "optimal," "optimized," and the like, here and elsewhere, does not necessarily mean "most optimal," "fully optimized," or the like. For example, it is not strictly accurate to call these chains optimal here, since it is not yet known whether they are in fact most optimal. In fact, it may be that some of all the chains that will be formed (derived from all chunks of all encoding configurations of all blocks) will be better than these. For example, there may be another chain that proves to have a better rate-distortion tradeoff than any of the N-1 chains considered here. However, at this point, it is known that the discarded chain is suboptimal because at least one of the N-1 chains considered outperforms them all, and therefore, these N-1 chains can be treated as optimal, at least at this point in the process.

[0090] As can be seen from Figure 8, the number of chains is 2 NThe number of chains dramatically decreases from -N-1 to N-1. An exemplary embodiment does not lose anything by considering only these chains and discarding the others. As will be seen below in the description of the next stage, chains are applied according to a specific order (controlled by the hyperparameter lambda). Each time a chain is applied, some blocks are set to a specific encoding configuration, and once the blocks are set, they can no longer be changed. Returning to the example above, we can see that the chain formed by the first and second chunks and the chain formed by the second and third chunks conflict with each other. That is, they both reference block i2, which means that applying one would make the other no longer applicable. However, due to optimality, applying the second chain instead of the first chain would be suboptimal. This makes the second chain unnecessary and can be discarded. If block i1 is already set, the chain consisting of the first and second chunks is no longer applicable (because it references block i1), and logically, the chain consisting of the second and third chunks would be the optimal chain. However, there is a problem. Because that chain is not kept and therefore not applicable. In this case, once block i1 is set, this chain becomes "naturally" available. Indeed, if we consider the optimal chain consisting of the first three chunks (length 3), once block i1 is set, it will consist only of the second and third chunks, which is the chain of interest. Thus, keeping only the optimal chains is indeed sufficient, since these chains will be decomposed into smaller chains that will be optimal in the current context, and therefore there is no need to keep the other discarded chains.

[0091] Note that the hyperparameter lambda controls the tradeoff between rate and distortion. Without hyperparameters, an exemplary embodiment would be unable to determine which of the chains represents a better reciprocity than the other chains, and therefore which chain is truly optimal compared to the other chains. For example, the chain consisting of the first two chunks has lower distortion than the chain consisting of the first three chunks. However, the former has a worse rate than the latter (because rate increases with chain length). As will be seen below, using hyperparameters and introducing the concept of cost makes it possible to answer the question of which chain represents the best reciprocity.

[0092] Stage 4: Match Resolution This stage helps to efficiently apply the chains retained during the previous stage. Since there can be multiple (possibly optimal) chains, all of which reference different (and potentially common) blocks, a decision is made as to the order in which they should be applied. The hyperparameter λ (lambda) helps in this regard. It controls the trade-off between rate and distortion. One can think of this as a kind of "market price" between exchanging one unit of rate for one unit of distortion. The concept of cost is also introduced. Cost (C) is defined as: C=D-λR where D represents the distortion and R represents the rate. The distortion is calculated as the sum of all the distortions that make up the chain (of length N).

[0093]

number

[0094] Note that C = R - λD is another formula that can be used to define the cost. However, in the following paragraphs in the remainder of this section, we will assume that the cost function is equal to D - λR.

[0095] As discussed above, the rate R corresponds to the storage gain achieved by applying chaining. This can be difficult to calculate, since it depends on the behavior of the lossless compressor. Thus, one exemplary embodiment provides an estimator of how many bytes the lossless compressor will "save," taking into account the matches introduced into the texture bitstream as a result of applying chaining. The Rate and Distortion section below provides further details on this point.

[0096] While the formula for C given above correctly models the trade-off between rate and distortion, improvements are possible. That is, each time a chain is applied, the encoding configuration of the blocks is set once for all. This means that these blocks can no longer be configured. However, it may be desirable to avoid prioritizing longer chains over smaller chains. Otherwise, one exemplary embodiment may "burn out" blocks very quickly. Thus, one exemplary embodiment may reduce the cost per block (C block ) can be considered.

[0097]

number

[0098] In one example embodiment, the procedure works as follows: First, each block is marked as "unset". Each time, all chains are processed and their cost C blockis calculated. Positive costs imply an invalid rate-distortion trade-off, so these chains are skipped. The chain with the smallest (negative) cost is taken and applied. As discussed above, applying a chain involves updating the texture and setting the encoding configuration that the chain references for each associated block. Each of these referenced blocks is then marked as "set". From each block, it is possible to reference all the chains to which it belongs. Thus, once a block is "set", each chain that references it is updated by "removing" that block from it. Removing a block from a chain involves decreasing its length by 1 while also subtracting the distortion of this block.

[0099] This procedure is repeated until one of the following conditions is met: (1) all blocks are marked as "set"; (2) all chains have been applied; or (3) the remaining chains are either inapplicable (length strictly less than 2) or have a positive cost. It is acknowledged that a chain with a positive cost does not necessarily mean that it will never remain in that state forever. As discussed above, a chain can be decomposed into smaller chains, and one of these smaller chains may have a negative cost (and thus represent an interesting rate-versus-distortion tradeoff), which can then be selected by the procedure above.

[0100] Figure 9 illustrates one iteration of the match chain resolution procedure of an example embodiment. Figure 9 begins with choosing a chain to apply in the leftmost portion of the diagram. It will be appreciated that the numerical values ​​in the example of Figure 9 are given by way of example and not limitation.

[0101] To maximize cache efficiency, chains are stored as very lightweight (taking up very little space in memory) objects, containing three pieces of information: length, match size, and sum of skew (which is the sum of the skews of all blocks referenced by the chain). Therefore, from this "lightweight chain" alone, an exemplary embodiment cannot identify the blocks it references. In an exemplary embodiment, such information is kept in a separate buffer. Indexing allows the data to be referenced. For example, an exemplary embodiment may implement the following indexing procedure:

[0102] Given the index position of a chain in the chain buffer, you can access the list of paired information it references (e.g., in the form of (block index, encoding configuration index)).

[0103] Given an encoding configuration index, its distortion value can be accessed.

[0104] Given a block index, you can access the block status ('set' or 'not set').

[0105] Given a block index, you can access the list of chains it participates in.

[0106] These four links enable the procedure of FIG. 9 to be implemented in an exemplary embodiment.

[0107] Finally, if the procedure terminates without all blocks being in the "set" state, an example embodiment applies an encoding configuration with zero distortion to each of these remaining blocks. As detailed in the next section, such an encoding configuration always exists and corresponds to the best quality configuration that exists for a block. Note that doing so can also be viewed as applying a hypothetical chain consisting of only one block that has zero distortion but also zero rate (because a chain must reference at least two blocks for its rate to be non-zero). Therefore, such a hypothetical chain would have a cost of zero, the highest cost that RDO can consider.

[0108] Exemplary Rate and Distortion Considerations Rate and distortion have been discussed above. To measure quality, metrics are used. Possible metrics that can be used in connection with an exemplary embodiment include, for example, the sum of squared differences (SSD) in a given color space (RGB, YUV, CieLab, OkLab, etc.), and some variant of SSD aimed at reducing blockiness artifacts, which may include (1) a weighted SSD that gives more weight to pixels on block boundaries, or (2) SSD normalized by pixel variance (noisy blocks can often be altered more aggressively than other blocks because this is less noticeable), and further metrics include angular SSD of normal maps, structural similarity (SSIM), etc.

[0109] An example embodiment measures quality at the texture block level, thereby obtaining a value that measures the quality of a particular texture block. In an example embodiment, the quality metric includes more than pixels for a given block. For example, it can also include pixels surrounding the block (e.g., using the spatial context of the block). The quality metric is additive, in the sense that a quality measure for a group of texture blocks is equal to the sum of the quality measures for each block in the group, or can at least be approximated in this way.

[0110] Since distortion is a measure of quality loss, it actually measures the quality of a texture block relative to a reference. The reference in one exemplary embodiment is, for a given block, the best possible quality that can be obtained for that block among all available encoding configurations. In other words, Q is the quality metric used, and q i,j is the encoding configuration j (or e i,j ), the distortion is given by d i,j =q i,j -Min j (q i,j ) This formula implies that for a given block, the distortion is a positive value and the best encoding configuration has zero distortion, which is desirable.

[0111] Rate, which is how much storage size is expected to be saved, can be more difficult to obtain because (unlike distortion) it cannot be calculated directly and therefore is estimated in some cases. In fact, predicting how a lossless compressor will react to changes in the texture bitstream can be difficult because this often depends on many things internal to the compressor itself, so it may be unknown how much a match chain will actually save. Exemplary techniques for estimating storage gain include, for example, closed-form and model-based approaches, as discussed in more detail below.

[0112] Exemplary Closed-Form Approach to Rate Estimation One exemplary embodiment attempts to model the storage gain using a function, which has the advantage of being very fast to evaluate. One function that can be used is R(Length,MatchSize)=Max(0,(Length-1)*(MatchSize-MIN_MATCH_SIZE+1) is.

[0113] As discussed above, length and match size are attributes of a match chain. If there are 6 overlapping chunks of 5 bytes in a texture, there is a match chain with length 6 and (match) size 5. Thus, assuming MIN_MATCH_SIZE is 4, the rate of this match chain given by the above formula is R=10.

[0114] This function also has the property of being predictable in terms of rate. For MatchSizes strictly smaller than MIN_MATCH_SIZE or lengths < 2, R=0 ·R is an increasing function of length, and ·R is an increasing function of MatchSize Includes:

[0115] Exemplary Model-Based Approach to Rate Estimation Instead of using an explicit formula to calculate the rate, an exemplary embodiment can attempt to estimate the rate from a simplified model of the lossless compressor. There are many ways to approximate the performance of a compressor. One approach that an exemplary embodiment can use is to attempt to independently estimate the rate of each possible match chain that the algorithm can consider. For example, suppose there is a desire to have an estimate of how much a 4-byte match chain of length 3 (i.e., a 4-byte chunk repeating itself 3 times) will save. It is known that matches occur within slices of a given size (see the Texture Slices section above). It is possible to "simulate" how much this particular match chain will save by running the following experiment many times:

[0116] 1. Generate a random buffer of data the size of the texture slice size you are considering (e.g. a slice of 4,096 ASTC blocks means a buffer of 65,536 bytes).

[0117] 2. Put the slice into a given "compressibility state." The goal is to modify random data in the buffer to create matches that will cause the buffer to reach a certain compression level (when compressed with a lossless compressor). That is, matches are randomly created in the buffer until the required compression level (e.g., 60% of the uncompressed buffer size) is reached.

[0118] 3. (In our example, this means a 4-byte match of length 3) Do another match in the buffer using the target match chain, making sure not to overwrite any existing matches made in previous steps.

[0119] 4. Compress the buffer using a lossless compressor. Calculate the difference between the resulting compressed size and the uncompressed size from 2. This gives an estimate of how much storage size the match chain saved.

[0120] 5. The "experiment" is repeated a number of times (say 1 to 4), and in each iteration the initial buffer contents (1), the compression level and the type of match inserted to reach that level (2), and the location in the buffer of the match chain from which the rate must be estimated (3) are randomized. The rate estimate is then given by the average of all estimates from 4.

[0121] The above procedure is performed for all types of chains that the algorithm can possibly run. For example, if MIN_MATCH_SIZE is set to 4 and MAX_MATCH_SIZE is set to 8, then with a texture slice of 40,96 blocks, there are (8-5+1)*(4,096-1)=20,475 types of chains for which rates should potentially be estimated. This procedure can be done "offline" (e.g., outside of the algorithm run), and the results of all rate estimates can be stored (e.g., in an array) that is accessible during the algorithm run.

[0122] Exemplary Techniques for Controlling Memory Usage Exemplary Texture Slicing Technique As discussed above, stage 2 of the algorithm of an example embodiment extracts chunks from the encoding configuration. For large texture sizes, this may result in excessive memory consumption and therefore be impractical. For example, employing a 1,024 x 1,024 texture with ASTC format and 1,024 encoding configurations per block to compute stage 2 combinations results in nearly 1 billion extracted chunks.

[0123] To help address this issue, an example embodiment divides the texture into slices, each containing a given number of blocks. Each slice is processed independently by an algorithm of an example embodiment. Figure 10 schematically illustrates a texture divided into four slices according to the texture slicing approach of an example embodiment.

[0124] In one exemplary embodiment, typical slice sizes range from 1,024 to 4,096 blocks. Choosing the right slice size correlates with choosing the right number of encoding configurations per block, since both strongly affect memory usage and processing time, which you may want to keep at controlled levels. You may want to favor larger slice sizes and fewer encoding configurations, or conversely, more but smaller slices. Such parameters may be determined empirically, depending not only on constraints (e.g., memory and processing time budgets) but also on the behavior of the algorithm itself (including compression level and texture quality to be achieved) that use these parameters.

[0125] Breaking a Match Chain in an Example Embodiment In some cases, there may be early chain rejection, such as when it is known in advance that some chains can be discarded (and therefore not stored) because Stage 4 of an example embodiment's algorithm will not consider them. This means that such a chain is always suboptimal, regardless of the smaller chains it can decompose into. In other words, there will always be better match chains than this one. Stage 3 described one such situation: some chains are suboptimal because they were identical in rate to another while having worse distortion; the discussion above showed that these chains do not need to be stored, because the optimal chains considered would naturally decompose into them if they were optimal.

[0126] Similarly, a chain can have the same distortion as another while having a poorer rate.

[0127] FIG. 11 helps illustrate when a match chain may be discarded in the context of an example embodiment. FIG. 11 represents a supposedly optimal chain of length 3. An example distortion of each referenced block is provided for illustrative purposes. Moving to a higher match size (5 bytes), all of the chunks that make up the chain also collide at 5 bytes. Therefore, even if a 4-byte chain is identified as a potentially optimal chain in stage 3 of an example embodiment's algorithm, it does not need to be stored because it is not considered in stage 4 of an example embodiment's algorithm. In fact, no matter what smaller chains it decomposes into, it will always be a suboptimal chain because higher-sized chains also decompose into the same smaller chains, with the same distortion but a larger match size (here, 5 bytes) and therefore a better rate. That is, in FIG. 11, the second match chain with a match size of 5 and a length of size 3 "beats" the first match chain with a match size of 4 and a length of size 3.

[0128] More details on breaking match chains There are further optimizations that may allow discarding multiple chains. For example, after stage 1 of an exemplary embodiment algorithm, there are multiple encoding configurations for each block. It is recognized that some distortion values ​​are such that it can already be determined that any chains referencing these encoding configurations will not be considered by stage 4 of an exemplary embodiment algorithm.

[0129] To see this, consider a chain of length N. Its cost per block is

[0130]

number

[0131] In the formula, d iis the distortion of the i-th block referenced in the chain, and R N is the rate. The subscript N is used here to indicate that the rate depends, among other things, on the chain length. The strain increases with increasing value d1≦d2≦ ≦d N Assume that the data is sorted by

[0132] Now, the new cost value C block,N+1 Distortion d N+1 Consider adding another block to this chain using . Considering the numerator of this cost,

[0133]

number

[0134] is. R N+1 -R N is the increment rate, which corresponds to the additional storage gain by extending a chain of length N to a chain of length N+1. This value ΔR max Suppose we know the dominant series for . Then,

[0135]

number

[0136] is. Therefore, d N+1 >λΔR max If so,

[0137]

number

[0138] and two cases can occur. First,

[0139]

number

[0140] This means that a chain of length N has a positive cost and is therefore uninteresting to apply. But then, by following the inequality, a chain of length N+1 also has a positive cost and is therefore also uninteresting. Second,

[0141]

number

[0142] A chain of length N is applicable. For a chain of length N+1 to be applicable,

[0143]

number

[0144] Dividing the above inequality by N gives

[0145]

number

[0146] However,

[0147]

number

[0148] Therefore,

[0149]

number

[0150] In other words, 0>C block,N+1 >C block,N is. In the first case, a chain of length N+1 has a positive cost, and in the second case, a chain of length N is superior because it has a higher cost, so there are suboptimal chains. i >λΔR max If , then any chain that references such an encoding configuration will be a suboptimal chain. However, suboptimality is not a sufficient criterion for rejection in an exemplary embodiment. An exemplary embodiment ensures that such a chain can never be selected in stage 4 of the algorithm so that it can be discarded. In other words, an exemplary embodiment ensures that it always remains a suboptimal chain. Two conditions, 0 > C, must be satisfied: block,N+1 >C block,N and d N+1 >λΔR max Therefore, the chain of length N+1 (or what remains of it) will never be optimal and therefore will never be applied. The reason is that as long as it remains, there are at least two blocks in the chain of length N, 0>C. block,N+1 >C block,N always holds, so a length N chain would be preferable compared to an N+1 chain. If only one block remains in the length N chain (and therefore the length N chain becomes inapplicable), then the length N+1 chain (now a length 2 chain) turns out to have a positive cost. For example, suppose only block 1 remains in the length N chain. Then the cost of the length N+1 chain is

[0151]

number

[0152] (because λR2=λ(R2-R1)≦λΔR max , since R1 is equal to 0).

[0153] Therefore, chains of length N+1 are not "interesting" and can be discarded. Exemplary Lambda Factorization Related Techniques As discussed above, the algorithm of an exemplary embodiment can be factorized. Furthermore, the use of hyperparameters was mentioned in Stage 4. Thus, by performing Stages 1 through 3 only once, it is possible to output different versions of the encoded texture, corresponding to different tradeoffs between rate and distortion, for example. In such a case, only Stage 4 would be executed several times. FIG. 12 shows an example of algorithm factorization according to an exemplary embodiment. In 10 steps, 11 hyperparameter values ​​ranging from 0 to 100 are fed into the algorithm. As can be seen, Stages 1 through 3 are executed only once per slice, in the same manner as described above. Only Stage 4 is executed as many times as there is input to the process. The input is a particular slice with a specific lambda value, giving a count of 11*4=44 executions of Stage 4. Because each of these executions can be fully parallelized, such a structure enables efficient multithreading in an exemplary embodiment.

[0154] As discussed in the previous subsection, an optimization that can be applied after stage 1 of the algorithm of one exemplary embodiment to avoid inserting a large number of chains is to reduce the number of chains by a certain threshold λΔR max This threshold depends on the value of lambda, which may contradict the notion that stages 1 to 3 of the algorithm are lambda independent. However, by modifying the threshold to make it lambda independent, e.g., by adding a new threshold λ max ΔR max This can be handled by adopting it as

[0155] Example Development and Runtime Pipeline From a developer's perspective (e.g., from the perspective of someone who may be designing or implementing a portion of a virtual environment, for example, for use in a game world), the algorithmic techniques disclosed herein may be provided in association with a development tool. The tool may be a software program running on a computing system including at least one hardware processor operably coupled to a memory. The tool may include a user interface through which a user interacts with the software program. A data store local to or remote from the computing system that accesses the software program stores a library of textures to be compressed.

[0156] Using the tool, a developer selects one or more textures to compress and specifies one or more target lossy formats from a list of possible lossy formats, such as those identified above (e.g., BC1, BC3, BC4, BC5, BC7, and ASTC). The developer decides whether to apply RDO, and if so, one or more RDO lambda values ​​are specified.

[0157] An encoding is generated for one or more selected textures using the specified parameters. It is recognized that multiple encodings can be performed efficiently, for example, as discussed above. Once the encodings are complete, the developer can compare quality and size. Quality may be determined using a programmed or selected metric (e.g., PSNR, SSD, SSIM, etc.). Size may be expressed as absolute size, a percentage of the original size, etc.

[0158] The tool may, for example, display the original texture adjacent to the encoded texture in a user interface for visual comparison. Hovering a cursor, such as a mouse pointer, over the encoded texture may provide summary details, such as the encoder used, whether RDO was applied, quality, size, and a ranking of quality and / or size if multiple encodings were identified. Other details of the encoding configuration chosen by the texture encoder for each block (e.g., partition index, weight values, color endpoints, etc.) may also be displayed. Providing this type of information is useful for comparing, reviewing, and analyzing encoding configurations.

[0159] The developer can select a particular encoding for decompression, or a particular encoding can be selected automatically (e.g., based on a best reciprocity or other set of programmed rules). The compressed texture may then be packaged and decompressed.

[0160] Runtime refers to decoding. The decoder applies lossless reconstruction (where applicable). In one exemplary embodiment, the lossy encoded textures are then given to a GPU, which then handles lossy decoding and rendering, for example, in presenting a virtual environment.

[0161] Overview of Encoder and Decoder Related Techniques Numerous features and aspects of the disclosed technology are set forth above. The following description provides an overview of certain encoder and decoder related techniques. The following merely highlights some features of certain exemplary embodiments and therefore should not be construed as limiting all embodiments, with the scope of the claimed invention being expressly set forth in the claims.

[0162] Referring first to the encoder, FIG. 13 is a flowchart illustrating a process for generating an encoded texture according to an example embodiment, and FIG. 14 is a block diagram illustrating a computing architecture that may be used in connection with the example flowchart of FIG. 13, according to an example embodiment.

[0163] FIG. 13 illustrates the conceptual three-stage decomposition discussed in detail above, including, for example, Stage 1 (multi-component texture encoding), Stage 2 (chunk extraction), and Stage 3 (match chain creation). FIG. 13 begins at step 1302, where textures are retrieved from a raw texture repository 1402 via an encoding computing system 1404. The raw texture repository 1402 may be local to or remote from the encoding computing system 1404 in different exemplary embodiments. In the latter case, the raw textures (textures to be encoded) may be retrieved via API calls or the like over an appropriate network connection, such as the Internet. The computing system 1404 may include one or more computers in a standalone, client-server network, distributed computing (e.g., cloud computing), or other technology environment. In certain exemplary embodiments, networked and distributed computing environments may be advantageous because, as discussed above, encoding operations may be parallelized. The encoding computing system 1404 includes at least one processor 1406 operatively coupled to memory 1408 and hardware user interface elements 1410. The memory 1408 includes a working area (e.g., working RAM 1412) and an encoding application 1414. A user uses the hardware user interface elements 1410 (e.g., a keyboard, a mouse, a touch screen, etc.) to interface with and collaborate with the encoding application 1414, for example, to indicate which textures to compress, indicate the lossy and / or lossless encoding to use (e.g., based on what is available from the lossy library 1416 and the lossless library 1418), identify hyperparameters, monitor the output of the encoding operation, etc.

[0164] Once the raw texture to be encoded is read, the encoding application 1414 divides the texture into multiple blocks (e.g., of a predetermined pixel size) at step 1304. At step 1306, each of these blocks is encoded into multiple block bitstreams. In doing so, each of these block bitstreams is encoded according to an encoding configuration (based in part on lossy encoding), with each block bitstream / encoding configuration having an associated distortion value. At step 1308, the block bitstream with the lowest associated distortion value is selected for each respective block, such that up to a predetermined number of distinct block bitstreams are selected. Note that the blocks, buffers, and / or other data structures used in the operations depicted in FIG. 14 may, in an exemplary embodiment, be stored in memory 1408 (e.g., in working RAM 1412).

[0165] In an exemplary embodiment, as many distinct block bitstreams as possible (e.g., up to a predetermined number specified by a user) may be selected, and an indicator may be provided for each block for which the number of selected block bitstreams is less than the predetermined number. Additionally or alternatively, an exemplary embodiment may allow input to be received to identify a maximum distortion value for one or more identified blocks among the plurality of blocks. In such a case, for each of these identified blocks, only block bitstreams having a distortion value better than the maximum distortion value may be selected for the respective identified block.

[0166] At step 1310, data chunks are obtained. This involves extracting at least some data chunks. In an example embodiment, only some data chunks are extracted directly from the blocks. In such cases, other chunks may be generated from chunks that are extracted directly (or at least more directly) from the blocks. Thus, in an example embodiment, obtaining data chunks may comprise extracting a subset of valid data chunks for a given selected block bitstream and generating remaining data chunks for the given selected block bitstream from the extracted subset (e.g., via masking).

[0167] In any case, for each block bitstream selected for low distortion, multiple data chunks are obtained such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained. The data chunks of each selected block bitstream start at a respective position (e.g., a respective byte) in the respective selected block bitstream and extend to the end of the respective selected block bitstream such that each data chunk has a minimum data chunk size. In an exemplary embodiment, data chunks of different sizes can be obtained for the selected bitstreams. In an exemplary embodiment, the minimum data chunk size may be at least as small as the minimum size usable by the lossless compressor ultimately used.

[0168] At step 1312, collisions between the obtained data chunks are detected (e.g., using efficient sorting techniques and by eliminating non-extended, non-colliding, and / or duplicate colliding data chunks from the higher distortion block bitstream). At step 1314, based on the detected collisions, match chains with redundant data chunks are formed such that for each match chain, the redundant data chunks therein represent a common truncated value of the associated block bitstream.

[0169] During the match chain creation process, many of the match chain candidates (potential match chains) are filtered out, discarded, or otherwise not considered because it is known in advance that they cannot be selected in a subsequent iterative selection procedure because it has been detected that for each of these discarded match chains, there is always at least another equal or better match chain. In other words, a given match chain is not considered in response to a determination that another match chain has been or will be formed that is equal to or better than the given match chain. "Better" in this context should be understood as having a better cost, which is obtained via a cost function related to hyperparameters.

[0170] The match chain with the best value from among the remaining formed match chains is iteratively selected. In an exemplary embodiment, the "best value" may be determined in relation to a function related to the hyperparameter, e.g., a hyperparameter that sacrifices rate or distortion in order to gain one of the other. That is, in an exemplary embodiment, the hyperparameter is part of a defined function that determines which match chain is best at sacrificing rate or distortion in order to gain the other.

[0171] At step 1316, a lossy encoding of the texture is obtained using the iteratively selected match chains, and at step 1318, the encoded lossy texture is further encoded using a lossless compressor. The "final" encoded texture is sent to an encoded texture repository 1420, which, in different exemplary embodiments, may again be co-located with or remote from the encoding computing system 1404. These textures may be used, for example, in defining a virtual environment such as a video game.

[0172] 13 may be performed using an encoding application 1414, for example, based on user input obtained via hardware user interface element 1410. The encoding application 1414 may be implemented as instructions executable by processor 1406 in a series of program modules or other structure.

[0173] FIG. 15 is a flowchart illustrating a process for decoding textures encoded via the techniques described herein, according to an exemplary embodiment. As shown in FIG. 15 , at step 1502, the encoded textures are retrieved. The encoded textures may be retrieved as part of a virtual environment (or portions of a virtual environment) being rendered on a computing device, such as that shown in and described in connection with FIG. 16, as part of a virtual tour (e.g., of a home, a museum, etc.), and / or in other scenarios. The textures may be included in an application, retrieved from a local or remote data store, etc. Once the textures are retrieved, at step 1504 they are provided to a lossless decoder, the output of which is provided to a lossy texture decoder 1506. These decoders may be implemented in software and / or in conjunction with features provided by a processor, such as a CPU, a GPU, etc. In an exemplary embodiment, the retrieved textures may simply be passed to a GPU for decoding and rendering. In any case, at step 1508, a virtual environment is generated based on the decoded textures. The output may be provided to a display device (eg, an LCD, LED, OLED, or other display device) operably connected to the computing device.

[0174] 16 is a block diagram illustrating a computing device 1602 that may be used in connection with the example flowchart of FIG. 15 , according to an example embodiment. The computing device 1602 may be a gaming device, such as a handheld gaming device, a stationary gaming device, or the like. Alternatively, the computing device 1602 may be a television, a personal computer, a laptop, a mobile electronic device, or another type of electronic device. As will be appreciated by those skilled in the art, some or all of the blocks illustrated in FIG. 16 may be provided or omitted in different implementations depending, for example, on the type of electronic device provided.

[0175] 16, computing device 1602 includes one or more processors 1604 operatively connected to memory 1606. The one or more processors 1604 may include CPUs, GPUs, and / or other processors located within a common device, across several devices (e.g., in a networked environment), etc. The memory may include media such as, for example, an SSD, an HDD, flash memory, RAM, DRAM, etc.

[0176] The device 1602 can be turned on using a power button 1608, which is connected to a power control circuit 1610. The power control circuit 1610 is in turn connected to a battery 1612 (e.g., a rechargeable battery). The device 1602 can be powered over pin 1614, which can additionally be used to charge the battery 1612 when it is rechargeable.

[0177] A user can interact with computing device 1602 through a controller coupled thereto. The controller may, in an exemplary embodiment, be integral with the device. In an exemplary embodiment, a separate controller may be connectable to the computing device via controller terminals 1616 and / or a wireless controller interface 1618 (e.g., operating over a wireless protocol such as Bluetooth). In an exemplary embodiment, the controller may include game system-type controls such as buttons, a cross switch, a joystick, a slide pad, etc. In an exemplary embodiment, the controller may include a keyboard, a mouse, a trackball, etc. In an exemplary embodiment, a user may interact with device 1602 through a touch panel 1620 that interfaces with device 1602 in association with touch panel controller 1622. Video output is provided to a display 1624 that is integral with or otherwise connectable to device 1602. Audio is provided via integrated speakers 1624 and / or through an audio jack or other terminal 1626. The audio jack or other terminal 1626 connects to processor 1604 via codec circuitry 1628.

[0178] Slot 1630 receives physical media, such as, for example, an SD card, a micro SD card, a CD-ROM, a DVD-ROM, etc. Slot interface 1632 provides a connection between slot 1630 and processor 1604. In an exemplary embodiment, media inserted into slot 1630 may include a program that contains or otherwise accesses compressed textures to be displayed. In an exemplary embodiment, a program that contains or otherwise accesses compressed textures to be displayed may be read via network interface 1634, for example, in connection with downloaded games, cloud gaming scenarios, etc.

[0179] Although one exemplary gaming system is shown, it will be appreciated that the techniques disclosed herein may be used in connection with other gaming systems and other electronic devices that are not, in fact, dedicated gaming systems (e.g., PCs, TVs, mobile devices such as smartphones, etc.).

[0180] While the present invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. 1. A method of encoding a texture, comprising: encoding each of a plurality of blocks obtained by dividing the texture into a plurality of block bitstreams; selecting, for each block, up to a predetermined number of distinct block bitstreams; obtaining, for each selected block bitstream, a plurality of data chunks, each data chunk being a contiguous section of the respective selected block bitstream from which it was obtained; detecting collisions between the retrieved data chunks; forming, for each match chain based on the detected collisions, a match chain having redundant data chunks therein such that the redundant data chunks represent common values ​​of associated block bitstreams; iteratively selecting a set of match chains from the formed match chains based on hyperparameters, wherein at least some potential match chains are determined to be not selectable in the iterative selection and are not considered in forming the match chains; and obtaining a lossy encoding of the texture from the selected set of match chains.

2. 10. The method of claim 1, wherein in response to determining that another match chain is formed or will be formed that is equal to or better than a given match chain, the given match chain is not considered.

3. The method of claim 1 or claim 2, wherein application of the hyperparameter function to the match chains allows different match chains to be compared with each other.

4. The method of claim 1 or claim 2, wherein the iterative selection selects the best match chain.

5. The method of claim 1 or claim 2, further comprising encoding the lossy encoded texture using a lossless compressor.

6. 3. The method of claim 1, wherein for each of the blocks, the block bitstream with the smallest associated distortion value is selected, such that up to the predetermined number of distinct block bitstreams are selected.

7. The method of claim 1 or 2, wherein for a selected block bitstream, data chunks of different sizes can be obtained.

8. 1. A method of providing a virtual environment in connection with a computing system, comprising: reading an encoded texture from a storage medium, said texture being encoded using the method of claim 1 or claim 2; and providing the encoded texture to at least one processor of the computing system for decoding the texture and use of the decoded texture in the virtual environment.

9. A storage medium storing instructions that, when executed by a processor, cause a computer to perform operations corresponding to the method of claim 8.

10. a computing system through which a virtual environment can be displayed, said computing system comprising: a memory coupled to one or more processors configured to perform operations, the operations including: reading an encoded texture from a storage medium, said texture being encoded using the method of claim 1 or claim 2; and At least one of the one or more processors decodes the texture.

11. A program for encoding textures, comprising: The program causes at least one processor to: encoding each of a plurality of blocks obtained by dividing the texture into a plurality of block bitstreams; selecting, for each block, up to a predetermined number of distinct block bitstreams; obtaining, for each selected block bitstream, a plurality of data chunks, such that each data chunk is a contiguous section of the respective selected block bitstream from which it was obtained; detecting collisions between the retrieved data chunks; forming, for each match chain, a match chain having redundant data chunks therein based on the detected collisions, such that the redundant data chunks therein represent common values ​​of associated block bitstreams; iteratively selecting a set of match chains from the formed match chains based on hyperparameters; At least some potential match chains are determined to be not selectable in said iterative selection and are not considered in forming said match chains; and The program causes the at least one processor to: A program causing a program to execute a step of obtaining a lossy encoding of the texture from the selected set of match chains.

12. 1. A system for encoding and decoding textures, comprising: The system comprises: a first device for encoding the texture; a second device for decoding the encoded texture; The first device is encoding each of a plurality of blocks obtained by dividing the texture into a plurality of block bitstreams; selecting, for each block, up to a predetermined number of distinct block bitstreams; obtaining, for each selected block bitstream, a plurality of data chunks, each data chunk being a contiguous section of the respective selected block bitstream from which it was obtained; Detecting collisions between the retrieved data chunks; forming, for each match chain, a match chain having redundant data chunks therein based on the detected collisions, such that the redundant data chunks therein represent common values ​​of associated block bitstreams; Iteratively selecting a set of match chains from the formed match chains based on hyperparameters; At least some potential match chains are determined to be not selectable in said iterative selection and are not considered in forming said match chains; and The first device is obtaining a lossy encoding of the texture from the selected set of match chains; The second device is receiving an encoded texture from the first device; The system decodes the encoded texture.

Images