Machine learning techniques for creating high-resolution data structures representing textures from low-resolution data structures

Abstract

Machine learning (302) is used to generate a first mipmap (304) of a texture having a first compression based on a second mipmap (300) of the same texture and having a second compression without using compression or decompression in generating the first mipmap. The first mipmap can then be used to render a computer graphics object.

Description

[Technical field]

[0001] The present application relates to technically inventive and unconventional solutions that necessarily result from computer technology and give rise to concrete technical improvements. [Background technology]

[0002] In computer simulations, such as computer games, parts of an object are rendered using "texture" data that represents the surface of the object. The more texture data there is for a given object, the higher the resolution of the rendering can be. However, for bandwidth purposes, it is desirable to not send large texture data structures to the rendering device. Summary of the Invention

[0003] As understood herein, so-called "mipmaps" (from the Latin multum in parvo) can be used to save bandwidth in the following way: A mipmap is a series of texture data structures, each one a progressively lower resolution representation of the previous texture data structure. Typically, this reduction is by a factor of two in each dimension. Bandwidth is saved by using a higher resolution mipmap to render objects close to the viewer and a lower resolution mipmap to render objects further away from the viewer. Typically, the mipmap level is chosen to be the level that best matches the pixel density of the image. Ideally, one texture pixel is desired for every screen pixel. A texture pixel may also be called a "texel" (a combination of texture and pixel).

[0004] However, as will be understood herein, to conserve memory, texture data is typically compressed into one of a variety of block compression (BCn) modes that can be sampled natively on a GPU. The maximum resolution of a texture is typically limited by storage space constraints and artist authoring time. To generate a high-resolution image from a low-resolution one, such as using existing machine learning-based techniques, the BCn compressed texture data must first be decompressed, upsampled, and recompressed. This is undesirable because extra storage space is required for uncompressed versions of both the low-resolution and high-resolution images, and the process of BCn compression is very complex and computationally expensive.

[0005] Thus, the assembly includes at least one processor configured with instructions for receiving at least a first data structure representing at least one computer graphic texture. The first data structure has a first resolution. The instructions are executable to process the first data structure using at least one neural network (NN) to generate a second data structure representing the computer graphic texture, the second data structure having a second resolution higher than the first resolution. Thus, the second data structure is generated from the first data structure without the use of compression or decompression. The instructions are executable to use the second data structure for rendering, whether displayed directly on a screen or at an intermediate rendering stage.

[0006] Computer graphics textures may contain data used for physically based rendering (PBR) materials and may be compressed using block compression (BCn), where n is an integer.

[0007] In some examples, the first data structure includes an input mipmap and the second data structure includes a mipmap that is one mip level higher than the input mipmap. The input mipmap may include a tail mipmap.

[0008] In a non-limiting embodiment, the first data structure may include at least a first block of normal data and at least a second block of roughness data, and the instructions may be executable to generate a second data structure of at least four blocks of normal data and four blocks of roughness data together.

[0009] In a non-limiting embodiment, the first data structure may include at least a first block of texture data and at least a second block of adjacent texture data to help inform the machine learning, and the instructions may be executable to generate a second data structure of at least four blocks of texture data.

[0010] In another aspect, the rendering assembly includes at least one processor configured with executable instructions to receive a first compressed mipmap and generate a second compressed mipmap from the first compressed mipmap without using compression or decompression of the first compressed mipmap.

[0011] In another aspect, the method includes accessing at least one machine learning (ML) engine and upscaling or downscaling textures using the ML engine to present the object on a computer display.

[0012] The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram of an exemplary system including an example according to the principles of the present invention; [Diagram 2] 1 shows an example of a texture communication path. [Diagram 3] Here are some examples of PBR material components: [Figure 4]1 illustrates example logic in the form of an example flowchart for providing ground truth training data. [Diagram 5] 1 illustrates example logic in the form of an example flowchart for training a texture rendering machine learning engine. [Figure 6] 1 illustrates exemplary logic in the form of an exemplary flowchart for providing textures to a renderer. [Figure 7] 1 illustrates exemplary logic in the form of an exemplary flowchart for upscaling an input texture using a machine learning engine. [Figure 8] 1 illustrates, in flow chart form, exemplary logic for upscaling two texture data types together. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The present disclosure generally relates to a computer ecosystem including aspects of user information based on consumer electronics (CE) devices in the computer ecosystem. The system herein may include a server component and a client component connected through a network such that data may be exchanged between the client component and the server component. The client component may include one or more computing devices including portable computers such as portable televisions (e.g., smart televisions, Internet-enabled televisions), laptop computers, and tablet computers, as well as smartphones and other mobile devices including additional examples described below. These client devices may operate in a variety of operating environments. For example, some of the client computers may use operating systems obtained from Microsoft or Unix or Apple Inc. or Google, as examples. These operating environments may be used to execute one or more browsing programs such as browsers created by Microsoft or Google or Mozilla, or other browser programs that can access web applications hosted by Internet servers as described below.

[0015] A server may include one or more processors that execute instructions that configure the server to receive and transmit data over a network such as the Internet. Alternatively, clients and servers may be connected through a local intranet or a virtual private network. A server or controller may be instantiated by a gaming console such as a Sony PlayStation®, a personal computer, or the like.

[0016] Information may be exchanged between the clients and the servers over a network. For this purpose and for security, the servers and / or clients may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security. One or more servers may form an apparatus for implementing a method for providing a secure community, such as an online social website, for network members.

[0017] As used herein, instructions refer to computer-implemented steps for processing information within a system. Instructions may be implemented in software, firmware, or hardware and may include any type of programmed steps performed by components of the system.

[0018] The processor may be a single-chip processor or a multi-chip processor capable of executing logic through various lines, such as address lines, data lines, and control lines, as well as registers and shift registers. The processor may be implemented by or include one or more graphics processing units (GPUs).

[0019] The software modules described herein by flowcharts and user interfaces may include various subroutines, procedures, etc. Without limiting the disclosure, logic defined to be performed by a particular module may be redistributed among other software modules, and / or aggregated together in a single module, and / or made available in a shareable library.

[0020] The inventive principles described herein may be implemented as hardware, software, firmware, or any combination thereof. Accordingly, the illustrative components, blocks, modules, circuits, and steps have been described in terms of their functionality.

[0021] Further, of those mentioned above, the logic blocks, modules, and circuits described below may be implemented or performed by digital signal processors (DSPs), field programmable gate arrays (FPGAs), or other programmable logic devices such as application specific integrated circuits (ASICs), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor may be implemented by a controller or state machine, or combination of computing devices.

[0022] The functions and methods described below, when implemented in software, can be written in a suitable language, such as, but not limited to, Java, C#, or C++, and can be stored on or transmitted by a computer-readable storage medium, such as random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), other optical disk storage, such as compact disk read-only memory (CD-ROM) or digital versatile disk (DVD), magnetic disk storage or other magnetic storage devices, including removable thumb drives, etc. The computer-readable medium may be established by a connection. Such connections may include, by way of example, hardwire cables, including fiber optics and coaxial wire, as well as digital subscriber line (DSL) and twisted pair wire.

[0023] Components included in one embodiment may be used in other embodiments in any suitable combination, for example, any of the various components described herein and / or illustrated in the figures may be combined, interchanged, or eliminated from other embodiments.

[0024] A "system having at least one of A, B, and C" (similarly, "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.

[0025] Referring now specifically to FIG. 1, an exemplary ecosystem 10 is shown, which may include one or more of the exemplary devices described above and in detail below in accordance with the principles of the present invention. The first exemplary device included in the system 10 is an exemplary primary display device, which in the illustrated embodiment is an audio-video display device (AVDD) 12, such as, but not limited to, an Internet-enabled TV. Thus, the AVDD 12 may alternatively be an appliance or household item, such as a computer-controlled Internet-enabled refrigerator, washer, or dryer. Alternatively, the AVDD 12 may also be a computer-controlled Internet-enabled ("smart") phone, a tablet computer, a notebook computer, a wearable computer-controlled device (e.g., a computer-controlled Internet-enabled watch, a computer-controlled Internet-enabled bracelet, etc.), other computer-controlled Internet-enabled devices, a computer-controlled Internet-enabled music player, a computer-controlled Internet-enabled headphones, a computer-controlled Internet-enabled implantable device such as an implantable skin device, etc. Nonetheless, it should be understood that AVDD12 is configured to implement the principles of the present invention (e.g., to communicate with other CE devices to implement the principles of the present invention, to execute logic described herein, and to perform any other functions and / or operations described herein).

[0026] Thus, to implement such principles, AVDD 12 can be established by some or all of the components shown in FIG. 1. For example, AVDD 12 can include one or more displays 14, which can be implemented by high resolution or ultra-high resolution, i.e., "4K" or "8K" (or high resolution), flat screens and can be touch-enabled to receive consumer input signals by touching on the display. AVDD 12 can include one or more speakers 16 for outputting sound in accordance with the principles of the present invention, and at least one additional input device 18, such as a keyboard or keypad or audio receiver / microphone, for inputting audible commands to AVDD 12 to control AVDD 12. An exemplary AVDD 12 can also include one or more network interfaces 20 for communicating over at least one network 22, such as the Internet, a WAN, a LAN, etc., under the control of one or more processors 24. Thus, interface 20 can be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface. It should be understood that processor 24 controls AVDD 12 to implement the principles of the present invention, including other elements of AVDD 12 described herein, e.g., controlling display 14 to present images thereto, receiving input therefrom, etc. It should further be noted that network interface 20 may be, for example, a wired or wireless modem or router, or other suitable interface, such as, for example, a wireless telephony transceiver or a Wi-Fi transceiver as described above.

[0027] In addition to the foregoing, AVDD 12 may also include one or more input ports 26, such as, for example, a USB port for physically connecting (e.g., using a wired connection) to another CE device, and / or a headphone port for connecting headphones to AVDD 12 to present audio from AVDD 12 to the consumer through the headphones. AVDD 12 may further include one or more computer memories 28 that are not signal transitory, such as disk-based or solid-state storage (including, but not limited to, flash memory). Also, in some embodiments, AVDD 12 may include a position or location receiver, such as, but not limited to, a cellular telephone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographic location information from at least one satellite or cellular telephone tower and provide the information to processor 24 and / or determine the altitude at which AVDD 12 is located in conjunction with processor 24. However, it should be understood that another suitable position receiver other than a cellular telephone receiver, a GPS receiver, and / or an altimeter may be used in accordance with the principles of the present invention to determine the location of AVDD 12, for example, in all three dimensions.

[0028] Continuing with the description of AVDD 12, in some embodiments, AVDD 12 may include one or more cameras 32, which may be, for example, digital cameras such as infrared imaging cameras, webcams, and / or cameras integrated into AVDD 12 and controllable by processor 24 to collect pictures / images and / or videos in accordance with the principles of the present invention. AVDD 12 may also include a Bluetooth transceiver 34 and other NFC elements 36 for communicating with other devices using Bluetooth and / or Near Field Communication (NFC) technology, respectively. An exemplary NFC element may be a Radio Frequency Identification (RFID) element.

[0029] Furthermore, AVDD 12 may include one or more auxiliary sensors 37 (e.g., motion sensors such as accelerometers, gyroscopes, cyclometers, or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., sensors for detecting gesture commands, etc.)) that provide input to processor 24. AVDD 12 may include still other sensors, such as, for example, one or more climate sensors 38 (e.g., barometers, humidity sensors, wind sensors, light sensors, temperature sensors, etc.) and / or one or more biometric sensors 40, that provide input to processor 24. In addition to the above, it is noted that AVDD 12 may also include an IR transmitter and / or an IR receiver and / or an IR transceiver 42, such as an infrared (IR) data association (IRDA) device. A battery (not shown) may be provided for powering AVDD 12.

[0030] 1, in addition to the AVDD 12, the system 10 may include one or more other CE device types. In one example, a first CE device 44 may be used to send messages to a second CE device 46, which may include similar components as the first CE device 44 and will not be described in detail. In the example shown, only two CE devices 44, 46 are shown, it being understood that a fewer or greater number of devices may be used.

[0031] An exemplary, non-limiting first CE device 44 may be established by any one of the devices mentioned above, such as a portable wireless laptop computer or tablet computer or notebook computer or mobile phone, and may therefore have one or more of the components described below. The second CE device 46 may be established by, but is not limited to, a wireless telephone. The second CE device 46 may implement a portable handheld remote control (RC). The second CE device 46 may implement a virtual reality (VR) and / or augmented reality (AR), head mounted display (HMD). The CE devices 44, 46 may include some or all of the components shown for AVDD12.

[0032] At least one server 50 may include at least one server processor 52, at least one computer memory 54, such as disk-based or solid-state storage, and at least one network interface 56 that allows communication with other devices of FIG. 1 over network 22 under the control of server processor 52, and may, in fact, facilitate communication between the server and client devices in accordance with the principles of the present invention. It should be noted that network interface 56 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as, for example, a wireless telephony transceiver.

[0033] Thus, in some embodiments, server 50 may be an Internet server and may include and perform "cloud" functionality such that devices of system 10 may access a "cloud" environment, in an exemplary embodiment, via server 50. Alternatively, server 50 may be implemented by a gaming console or other computer that is in or near the same room as the other devices shown in FIG.

[0034] The devices described herein may, where appropriate, include some or all of the various components shown in FIG.

[0035] Before referring to Figure 2, a "texture" is a data structure that can be mapped to an image to characterize the surface of a rendered object. The basic data element of a texture data structure is a texture element or texel (a combination of texture and pixel). A texture is represented by an array of texels that represent a texture space. The texels are mapped to pixels in the image being rendered to define the surface of the rendered image.

[0036] Thus, because textures are data and not images, the neural network (NN) training described below does not necessarily rely on principles of perceptual error. There may be exceptions for albedo (discussed further below), but in general training on texture data uses a goal-specific error metric for the particular data.

[0037] Some types of data, such as normals and roughness, can be paired together because they are interrelated and the two data can mitigate each other's errors. More specifically, in PBR rendering, a relationship exists between normal map data and roughness data (sometimes called gloss data). Roughness essentially represents the distribution of normals across texture pixels. Thus, techniques exist to modify the roughness to account for data lost when reducing the resolution of the normal map when generating mipmaps, essentially anti-aliasing the normal map. In this case, there is a close relationship between the normal map and the corresponding roughness map.

[0038] Additional details on textures, physically based rendering (PBR) and materials are a general set of guidelines for rendering realistic material and light interactions, including using various types of data stored in textures. Light is modeled as either diffuse or specular. Diffuse light is generally view independent and generally does not change when viewing a material from any angle. Specular light, on the other hand, is view dependent to emulate, for example glare.

[0039] Typically, PBR textures include an "albedo" parameter that characterizes the diffuse light response of a material. For example, for an emulated surface of polished wood, the albedo texture contains grain patterns and color variations, but does not contain shape information and is very flat. Metals have no albedo, since all of their light response comes from reflection.

[0040] As mentioned above, PBR textures may also contain "normal" parameters. A normal map defines small geometric detail of a surface, and specifically represents the surface normal of the underlying surface. This may or may not be separate from the geometric normal of the triangle data used in rendering. This is vector data stored as a texture, and although it can be displayed, it is not, in itself, an image. In the wood example, this would be mostly flat where the wood is smooth, but may contain etched or carved detail into the surface of the wood.

[0041] As mentioned above, PBR textures may also contain a gloss / roughness parameter that defines how rough the surface is being rendered. Generally, this is thought of as the variance of sub-pixel normal data (detail smaller than a texel in the normal map). In the wood example, this may include data about scratches in the wood (think of rubbing wood with sandpaper). Sanded wood is relatively smooth, and therefore has a low roughness.

[0042] Another PBR parameter is reflectance, which describes the response of reflected light. For most non-metallic materials, reflectance is colorless. The reflectance of almost all non-metals is constant at 2% (the amount of light that reflects directly). For mirrors, reflectance approaches 100%.

[0043] Some PBR techniques attempt to take advantage of the fact that non-metals have a constant reflectance, and that metals have no albedo. This usually involves repurposing the albedo texels where the surface is metallic, to represent the reflectance instead (color, gold, brass, etc.), and storing additional information in a separate texture channel to identify which texels are metal or not. Generally, this information is called "metallicity". This encoding is generally done to save texture memory by not storing three channels for albedo and another three channels for reflectance.

[0044] In general, textures can have different purposes and can vary from game engine to game engine.

[0045] In view of the above discussion regarding textures, multiple versions of the same texture are typically generated with different compressions (and therefore different resolutions). Specifically, a single texture may be represented by a full mipmap chain of textures. Mip-mapping takes an image and successively reduces the resolution by a factor of two. Thus, for a given texture (e.g., 1024x1024, etc.), its 512x512 version and its 256x256 version, etc. are also stored in memory. This improves performance, visual fidelity, and facilitates texture streaming. Texture streaming attempts to conserve memory by loading only the "tail" of the mipmap (the lower mip level and all lower resolution mip levels below it) that is needed for a given texture based on the screen size of the rendered object. For example, the 256x256 mip level and lower than the mip chain of a 1024x1024 texture may be all that is needed to load a distant object. Which mip levels are loaded is changed on demand based on the changing game environment and the viewer's position. As an object gets closer, higher resolution mip levels or "levels of detail" are loaded into memory to render the now nearby object.

[0046] Textures may use various types of compression. One type is block compression, sometimes referred to as BCn compression. This may be a lossy texture compression that can be decompressed in-place by a graphics processing unit (GPU). Because block compression does not require the entire image to be decompressed, the GPU can decompress the data structures as it samples the texture so that it is not compressed at all.

[0047] Block compression techniques compress a 4x4 block of pixels into a single (smaller) data packet. Typically this involves choosing two or more "endpoint" colors (depending on the BC compression type), along with some per-pixel information about how to blend these two colors at each pixel. The endpoint colors are shared across the entire 4x4 pixel block. For example, for an image with just red, blue, and purple pixels, a compressor is likely to choose one endpoint to be red and the other endpoint to be blue. The purple pixels will have a value that blends the two together.

[0048] Different BC types mostly differ in the number of texture channels they have (e.g. BC4 is one channel grayscale, i.e. "black and white"). BC6 and BC7 are special because they introduce the concept of mode, which determines the interpretation of each block. For other BC modes, all blocks are encoded the same way, with the same number of bits allocated to endpoint colors and blend values. Different BC6 / 7 modes allocate their bits differently per block. This allows the compressor to make different quality tradeoffs in different areas of the texture.

[0049] 2 shows a texture source 200 that transmits textures for rendering according to the above principles over a communication path 202, such as a local data bus or a wireless / wired network link, to a texture renderer 204. The texture renderer 204 typically includes one or more GPUs with memory to render an image according to the image data and texture data on a display.

[0050] 3 illustrates that in an embodiment herein, only the tail texture 300 needs to be sent from the source 200 to the renderer 204. The renderer 204 can run a machine learning engine 302 on the texture to upscale it to the next higher level of resolution texture 304 without the need for decoding / encoding, and thus without the need for a codec. The machine learning engine 302 can include one or more trained neural networks, such as generative, noise-based, and possibly adversarial networks.

[0051] Therefore, machine learning is used to generate mipmaps for textures that are one level higher than the input mipmaps for all streaming textures (e.g., from a mipmap chain created on disk). New (higher resolution) mipmaps are similarly introduced as if they existed on disk, but are simply generated procedurally instead. Since textures can be stored on disk in BCN compressed format, the network generates new compressed mip levels from the highest of the existing compressed mip levels in memory. If the cost of generating mip levels at runtime is too high, the mip levels can be generated offline and stored on disk using the same method.

[0052] 4 shows an example of the training principle of the machine learning engine 302 of FIG. 3. An existing texture library may be accessed in block 400 for ground truth training, and compressed in block 402 to establish the next mip level down (again by half). In other words, in block 400, a full uncompressed (and thus highest resolution) mipmap may be accessed, and in block 402, compressed by half to render a compressed (and thus lower resolution) mipmap. The mipmaps generated in block 402 may be successively compressed according to the principles herein to render a full set of mipmaps, and in block 404, the ground truth is rendered for each input uncompressed mipmap.

[0053] 5 shows that a simple network may access one compressed BC block data packet from a ground truth set in block 500 and generate four BC block data packets for higher mip levels, e.g., going efficiently from a 4×4 block of pixels to an 8×8 block of pixels, in block 502. Alternatively, instead of or in addition to taking one BC block as input, a neighborhood of blocks surrounding the block (e.g., eight surrounding blocks) may also be provided as input to better inform the network of the features.

[0054] Training may be performed using backpropagation, gradient descent. Training may be performed using an 80 / 20 training test split, where a random sample of 80% of the ground truth data is used to train or set the weights in the NN, and then an additional 20% of the test data downscaled files are input and compared to those generated by the NN. In other words, the output of the NN can be compared to the 20% full resolution files of the ground truth that were not input during the training session.

[0055] Since there are mostly different types of data between different material attribute textures, a different network can be trained for each material attribute type, for example, one network can be trained to upscale reflectance texture data and another network can be trained to upscale albedo.

[0056] In the case of normals and roughness where similar data is stored and thus there is some interrelationship, a single network can be trained to upscale them together. The same considerations can be applied to other related pairs or groups of material attributes. In the case of a shared normal and roughness up-res network, one can input one BC block with normal data and one BC block of roughness data in the same relative location, and the network can output four BC blocks of normal data and four BC blocks of roughness data.

[0057] FIG. 6 shows that textures may be compressed at block 600 and sent to a renderer at block 602. In one example, the lowest resolution (most compressed) "tail" textures in the mipmap chain are sent to the renderer. The renderer can upscale them on the fly using a trained machine learning engine as described herein, without the need for a codec. In other embodiments, relatively nearly uncompressed high-resolution textures (or the uncompressed base textures themselves) may be provided as input to a renderer that can use machine learning to generate more compressed lower-resolution textures.

[0058] 7 illustrates the renderer logic. At state 700, an input texture is received. At block 702, the texture is processed by a trained machine learning engine, and at block 704, a different compressed texture is output that is either more compressed (low resolution) or less compressed (high resolution) than the input texture. The texture at block 704 is used to render an image on a display.

[0059] 8, for a shared normal and roughness up-res network, one BC block with normal data is shown received by the machine learning engine at block 800, and one BC block of roughness data at the same relative location is received at block 802. At block 804, the machine learning engine upscales the two input blocks together to output four BC blocks of normal data and four BC blocks of roughness data.

[0060] Although particular techniques have been shown and described in detail herein, it should be understood that the subject matter encompassed by this application is limited only by the scope of the claims.

Claims

1. 1. An assembly comprising: at least one processor, the at least one processor comprising: receiving at least a first data structure representing at least one computer graphic texture, the first data structure having a first resolution; processing the first data structure to generate a second data structure representing the computer graphic texture, the second data structure having a second resolution different from the first resolution; Rendering an object on at least one display using the second data structure; and It consists of instructions to Processing the first data structure to generate the second data structure comprises: processing reflectance data in the computer graphics texture using a first machine learning model to generate output represented in the second data structure; processing the albedo data in the computer graphics texture using a second machine learning model to generate an output represented in the second data structure; and jointly processing normal data and roughness data in the computer graphics texture using a third machine learning model to generate output represented in the second data structure.

2. The assembly of claim 1 , wherein the first data structure comprises a mipmap.

3. The assembly of claim 1 , wherein the first resolution is less than the second resolution.

4. The assembly of claim 1 , wherein the first resolution is greater than the second resolution.

5. The assembly of claim 1 , wherein the computer graphics texture comprises physically based rendering and materials (PBR) data.

6. 2. The assembly of claim 1, wherein the first data structure is compressed using block compression (BCn), where n is an integer.

7. 2. The assembly of claim 1, wherein the first data structure includes an input mipmap and the second data structure includes a mipmap one mip level higher than the input mipmap.

8. The assembly of claim 7 , wherein the input mipmap comprises a tail mipmap.

9. The assembly described in claim 1, wherein the first data structure includes the normal data and the roughness data.

10. 1. An assembly comprising: at least one processor, the at least one processor comprising: Receiving a first mipmap; generating a second mipmap from the first mipmap; and instructions executable to perform the steps of: Generating the second mipmap from the first mipmap includes: processing reflectance data in a computer graphics texture using a first machine learning model to generate the second mipmapped output; processing albedo data in the computer graphics texture with a second machine learning model to generate the second mipmapped output; and jointly processing normal data and roughness data in the computer graphics texture using a third machine learning model to generate output represented in the second mipmap.

11. The assembly of claim 10 , wherein the second mipmap is generated from the first mipmap using a machine learning engine.

12. the first mipmap is characterized by a first resolution; the second mipmap is characterized by a second resolution; The assembly of claim 10 , wherein the first resolution is less than the second resolution.

13. the first mipmap is characterized by a first resolution; the second mipmap is characterized by a second resolution; The assembly of claim 10 , wherein the first resolution is greater than the second resolution.

14. The assembly of claim 10 , wherein the first mipmap contains physically based rendering and materials (PBR) data.

15. 11. The assembly of claim 10, wherein the first mipmap is compressed using block compression (BCn), where n is an integer.

16. The assembly of claim 10 , wherein the second mipmap is one mip level higher than the first mipmap.

17. The assembly of claim 16 , wherein the first mipmap comprises a tail mipmap.

18. the first mipmap includes at least a first block including normal data and at least a second block including roughness data; The assembly of claim 10 , wherein the instructions are executable to generate the second mipmap comprising at least four blocks of normal data and four blocks of roughness data.

Images