Graphics system and method for using sparse textures

Abstract

The present disclosure relates to graphics systems and methods for using sparse textures. The present disclosure includes exemplary embodiments of a graphics processor memory management system that supports the use of graphics textures that are not fully bound or "backed up" in memory throughout their lifespan. Such graphics textures are referred to herein as "sparse textures." According to some embodiments, sparse textures may be split into fixed-size pages in memory, wherein, during execution, a user may indicate a desire to map certain pages to physical memory locations and fill such pages with underlying data. In other embodiments, statistical information obtained from the graphics processor is used to help determine whether a given texture (or portion of a texture) requires physical memory backup. In yet other embodiments, the graphics processor may also enforce ordering guarantees, for example, when there are fewer pages available in memory than are needed for backup at a given moment.

Description

Technical Field

[0001] The present disclosure relates generally to the field of graphics processing. More specifically, but not by way of limitation, the present disclosure relates to techniques that may be implemented on a graphics processor, such as a graphics processing unit (GPU), to sparsely back up textures and / or portions of textures (e.g., tiles) to physical memory in an intelligent and dynamic manner in order to conserve memory resources without unduly adversely affecting the graphical quality of a rendered scene. Background Art

[0002] In most graphics applications, the entirety of a given texture does not need to be accessed for the entire duration of rendering. For example, a camera view of a three-dimensional (3D) scene rendered by a graphics processor may be controlled by a user to, for example, zoom in, zoom out, move, change viewing direction, bring objects closer to the camera's view that obstruct further away portions of the 3D scene, etc. Such changes in the camera's view may make certain portions of certain textures in the 3D scene invisible at certain times, or at least less prominent in the camera's view. Thus, it follows that a satisfactory rendering of a given graphics scene may be produced without having to load the entirety of each texture represented in the scene (e.g., including the highest resolution version of the texture) into physical memory at all times. Summary of the invention

[0003] The following summary is included to provide a basic understanding of certain aspects and features of the claimed subject matter. This summary is not an extensive overview and is therefore not intended to particularly identify key or essential elements of the claimed subject matter or to delineate the scope of the claimed subject matter. The sole purpose of this summary is to present some ideas of the claimed subject matter in a simplified form as a prelude to the more detailed description presented below.

[0004] In one embodiment, a method for managing graphics hardware resources is described. The exemplary method includes, on a graphics processor: allocating a memory pool for a desired number of textures; loading a desired level of detail (LOD) version of each desired texture, e.g., from a mipmap; and then rendering a graphics scene using the loaded textures. Next, the graphics processor may begin obtaining statistics over time for each texture (or portion of a texture) that has been loaded. According to some embodiments, the statistics may include usage information, e.g., a counter value reflecting the number of times a particular texture (or portion of a texture, such as a "tile" area of ​​pixels within a texture) has been accessed. According to some embodiments, the statistics may come directly from the graphics processor itself.

[0005] Next, for each loaded texture, the textures (or portions of textures, depending on the level of granularity of the reported statistics) may be sorted based at least in part on the obtained statistics. Next, the graphics processor may determine to load only the backup data (i.e., "mapped" to locations in physical memory) for those textures or portions of textures that have statistics that exceed a threshold (e.g., a threshold minimum number of accesses from memory within a given time interval). Textures or portions of textures that do not exceed the threshold may be unbacked up or "unmapped" in physical memory, at least until the statistics for such textures or portions of textures again exceed the threshold, at which time they may again be mapped or backed up to locations in physical memory.

[0006] According to some embodiments, one or more sequencing rules may be applied to determine which textures (or portions of textures) are actually mapped in memory, for example, in situations where not enough memory is available to back up all desired textures or portions of textures at a given moment. Finally, the graphics rendering of the scene may be updated based on the newly mapped (or unmapped) textures in physical memory.

[0007] In one embodiment, the above method and its variants can be implemented as a series of computer executable instructions. Such instructions can use any of one or more convenient programming languages. Such instructions can be collected into an engine and / or program and stored in any medium that is readable and executable by a computer system, electronic device or other programmable control device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Although certain embodiments will be described in conjunction with the illustrative embodiments shown herein, the present disclosure is not limited to those embodiments. On the contrary, all alternatives, modifications and equivalents are included in the spirit and scope of the present disclosure as defined by the claims. In the drawings not drawn to scale, the same reference numerals are used throughout the specification and drawings for parts and elements having the same structure, and primed reference numerals are used for parts and elements having functions and configurations similar to those parts and elements having the same non-prime reference numerals.

[0009] Figure 1 is a schematic diagram of a graphics processing path in which embodiments of the present disclosure may operate.

[0010] Figure 2 is a block diagram of an embodiment of a graphics processor memory management system for graphics textures.

[0011] Figure 3 is an example of a scene being rendered using textures from varying mipmap levels to render the scene.

[0012] Figure 4 yes Figure 3 An example of using a mipmap level map for the rendered scene.

[0013] Figure 5 is a block diagram illustrating the relationship between tiles of an exemplary tiled texture and their corresponding memory backups.

[0014] Figure 6 is a flow diagram of operations for managing graphics hardware resources using sparsely backed texture resources.

[0015] Figure 7 is a simplified block diagram illustrating one implementation of a graphics processor.

[0016] Figure 8 is a block diagram illustrating an embodiment of a computing system that includes at least a portion of a graphics hardware resource allocation system. DETAILED DESCRIPTION

[0017] For existing graphics rendering systems, various approaches have been taken to try to manage physical memory more efficiently during graphics rendering operations. For example, in some existing approaches, the graphics rendering system may have split a single texture into multiple separate textures and then only load portions of the texture at a time in an attempt to limit the amount of information loaded into physical memory. In other existing approaches, the graphics rendering system may only always load the highest level of detail (LOD) version of a given texture into memory (e.g., to avoid repeatedly mapping and unmapping different LOD versions of the texture over time). However, the impact of this will be to sacrifice excess memory on portions of the texture that are not truly necessary for satisfactorily rendering the current 3D scene.

[0018] The graphics systems and methods disclosed herein allow for more "fine-grained" control over which parts and / or which versions of textures are loaded (or not loaded) into physical memory at a given moment when rendering a graphics scene, for example, via an intelligent prioritization scheme. Management of such a memory control system for a graphics processor can be made more efficient via implementation directly in the graphics processor, i.e., not via conventional intercommunication between a general-purpose processor (e.g., a CPU) and the graphics processor that is rendering a 3D scene.

[0019] The present disclosure includes various exemplary embodiments of graphics processor memory management systems and methods that support the use and creation of graphics textures that are not fully bound or "backed up" in memory throughout their lifetime. Such graphics textures may be referred to herein as "sparse textures," "sparsely backed textures," or "virtual textures." According to some embodiments, sparse textures may be split into fixed-size pages in memory, wherein during execution, a user may indicate a desire (e.g., via an application programming interface or API) to map certain pages to physical memory locations and fill such pages with underlying texture data. In some embodiments, for increased efficiency, the memory management process may be completely contained within the graphics processor, i.e., without the need to communicate with the CPU. In yet other embodiments, statistical information (e.g., in the form of hardware counters) may be obtained from the graphics processor to help determine whether a given texture (or portion of a texture) requires physical memory backing at a given moment. In yet other embodiments, the graphics processor may also enforce ordering guarantees, e.g., in the event that fewer pages are available in memory than are needed for backing up at a given moment.

[0020] As used herein, the term "kernel" in this disclosure refers to the operating system that is typically associated with a relatively high or highest security level (e.g., Mac OS X). TM ) is a computer program that is part of the core layer of the operating system. The "kernel" is capable of performing certain tasks, such as managing hardware interactions (e.g., the use of hardware drivers) and handling interrupts to the operating system. To prevent applications or other processes in user space from interfering with the "kernel", the code for the "kernel" is typically loaded into a separate, protected area of ​​memory. Within this context, the term "kernel" may be interchangeable with the term "operating system kernel" throughout this disclosure.

[0021] This disclosure also uses the term "compute kernel", which has another meaning and should not be confused with the term "kernel" or "operating system kernel". Specifically, the term "compute kernel" refers to a program for a graphics processor (such as a GPU, DSP, or FPGA). In the context of graphics processing operations, programs for a graphics processor are classified as "compute kernels" or "shaders". The term "compute kernel" refers to a program of a graphics processor that performs general computing operations (e.g., computing commands), and the term "shader" refers to a program of a graphics processor that performs graphics operations (e.g., rendering commands).

[0022] As used herein, the term "application program interface (API) call" in this disclosure refers to an operation that an application can perform using a graphics application program interface (API). Examples of API calls include draw calls for graphics operations and dispatch calls for compute operations. Generally speaking, the graphics driver converts the API calls into commands that the graphics processor can execute.

[0023] For the purposes of this disclosure, the term "processor" refers to a programmable hardware device capable of processing data from one or more data sources such as a memory. One type of "processor" is a general-purpose processor (e.g., a CPU or microprocessor) that is not customized to perform a specific operation (e.g., a process, calculation, function, or task), but is built to perform general computing operations. Other types of "processors" are special-purpose processors that are customized to perform specific operations (e.g., a process, calculation, function, or task). Non-limiting examples of special-purpose processors include GPUs, floating-point processing units (FPUs), DSPs, FPGAs, application-specific integrated circuits (ASICs), and embedded processors (e.g., universal serial bus (USB) controllers).

[0024] As used herein, the term "graphics processor" refers to a specialized processor for performing graphics processing operations. Examples of a "graphics processor" include, but are not limited to, a GPU, a DSP, an FPGA, and / or a CPU emulating a GPU. In one or more specific implementations, a graphics processor may also be capable of performing non-specific operations that a general-purpose processor may perform. An example of a general computing operation is a computing command associated with a computing kernel.

[0025] As used herein, the term "tile" refers to a portion of a larger graphics texture. Generally speaking, tiles are sized to be substantially smaller than the size of the texture of which they are a portion, for example, in order to reduce the memory and bandwidth requirements for processing each tile. The tile size can be set before rendering a scene and can be fixed across all textures used to render the scene. Exemplary tile sizes may include 16×16 pixel and 32×32 pixel tiles, but tiles of any size may be used. Generally speaking, when rendering a tile, the entire tile is rendered in a single pass, and multiple tiles may be rendered in parallel.

[0026] Figure 1 is a schematic diagram of a graphics processing pathway 100 in which embodiments of the present disclosure may operate. Figure 1An example is shown in which the graphics processing path 100 utilizes a processor resource 110 and a graphics processor resource 112. The processor resource 110 includes one or more general-purpose processors (e.g., CPUs), each of which has one or more cores. The processor resource 110 may also include memory, other microcontrollers, and / or any other graphics hardware resources that the processor may use to process commands for execution by the graphics processor resource 112, and / or communicate with memory, other microcontrollers, and / or any other graphics hardware resources that the processor may use to process commands for execution by the graphics processor resource 112. The graphics processor resource 112 includes one or more graphics processors (e.g., GPUs), each of which has one or more execution cores and other computational logic for performing graphics and / or general computational operations. In other words, the graphics processor resource 112 may also encompass memory (e.g., memory caches) and / or other graphics hardware resources such as shaders or compute kernels for executing programs, and / or communicate with memory (e.g., memory caches) and / or other graphics hardware resources such as shaders or compute kernels for executing programs.

[0027] Figure 1 The application 101 is shown generating graphics API calls for the purpose of encoding commands for execution by the graphics processor resource 112. To generate the graphics API calls, the application 101 includes code written in the graphics API. The graphics API (e.g., ) refers to a published and / or standardized graphics library and framework that defines the functions and / or other operations that the application 101 can have with the graphics processor. For example, the graphics API allows the application 101 to control the organization, processing and submission of rendering and computing commands, as well as the management of the related data and resources of those commands.

[0028] In one or more specific implementations, the application 101 is a graphics application that calls a graphics API to transmit a description of a graphics scene. Specifically, the user space driver 102 receives graphics API calls from the application 101 and maps the graphics API calls to operations that the graphics processor resource 112 understands and can execute. For example, before being transmitted to the kernel driver 103, the user space driver 102 can convert the API calls into commands encoded in a command buffer. The conversion operation can involve the user space driver 102 compiling the shader and / or the computing kernel into commands that can be executed by the graphics processor resource 112. The command buffer is then sent to the kernel driver 103 to prepare the command for execution on the graphics processor resource 112. For example, the kernel driver 103 can perform memory allocation and scheduling of commands to be sent to the graphics processor resource 112. For the purpose of this disclosure and for ease of description and explanation, unless otherwise specified, the user space driver 102 and the kernel driver 103 are collectively referred to as graphics drivers.

[0029] Figure 1 The graphics processor firmware 104 is illustrated as obtaining commands from the processor resources 110. The graphics processor firmware 104 may perform a variety of operations for managing the graphics processor hardware 105, including powering the graphics processor hardware 105 on and off and / or scheduling the order in which the graphics processor hardware 105 receives commands for execution (and / or determining which commands the graphics processor hardware 105 receives for execution). Figure 1 As an example, the graphics processor firmware 104 may be implemented by a graphics microcontroller that boots the firmware. Specifically, the graphics microcontroller may be embedded in the same package as the graphics processor within the graphics processor resource 112 and configured to pre-process the commands of the graphics processor. In other embodiments, the graphics microcontroller is physically separated from the graphics processor.

[0030] After the dispatch command, Figure 1 In the embodiment of the present invention, the graphics processor firmware 104 sends a command stream (e.g., multiple "kick command" jobs) to the graphics processor hardware 105. As used herein, the term "kick command" refers to a discrete unit of instructions submitted to the graphics processor. The graphics processor hardware 105 will execute the kick commands in the command stream according to the order in which the graphics processor hardware 105 receives the kick commands. The graphics processor hardware 105 includes graphics hardware resources that can execute multiple received kick commands in parallel. The graphics processor hardware 105 then outputs the rendered frame to the frame buffer 106. In one specific implementation, the frame buffer 106 is a portion of a memory such as a memory buffer that contains a bitmap that drives the display 107. The display 107 then accesses the frame buffer 106 and (e.g., using a display controller) converts the rendered frame (e.g., bitmap) into a video signal for display.

[0031] In one or more embodiments, graphics processor hardware 105 may also include a graphics hardware resource allocation system 108 that allocates graphics hardware resources ( Figure 1 100) to execute a kick command submitted by the processor resource 110 to the graphics processor resource 112. The graphics hardware resource allocation system 108 receives the kick command and decomposes the kick command into work that can be performed by the graphics hardware resources (e.g., vertex shaders, fragment shaders, joint shader clusters, registers, or compute units). For example, the user kernel driver 103 splits the commands submitted to the graphics processor for execution into finer-grained instructions for execution by the graphics processor. The finer-grained instructions represent kick commands, each of which can target a dedicated type of "data host" or channel of the graphics processor hardware (e.g., a pixel data host, a fragment data host, or a compute data host). For the purposes of this disclosure, the term "work" refers to a finer unit of instruction than a kick command. In other words, the graphics hardware resource allocation system 108 receives the kick command and decomposes the kick command into smaller-grained instructions called work.

[0032] Figure 2 is a block diagram of an embodiment of a graphics processor memory management system 200 for graphics textures. Figure 2 As shown, the graphics processor memory management system 200 is part of a graphics processor resource 112 that communicates with a CPU 205 (on which one or more processes 210 requesting graphics rendering may be executed). As described above, the graphics processor resource 112 includes one or more graphics processor hardware 105 (e.g., one or more GPUs), each of which has one or more execution cores and other computing logic for performing graphics and / or general computing operations. The graphics processor resource 112 may also include a graphics processor firmware 104, which may perform a variety of operations to manage the graphics processor hardware 105.

[0033] In some embodiments described herein, the graphics processor hardware 105 may also generate one or more hardware counters 215 for generating statistical information related to the operation of the graphics processor hardware. For example, the hardware counters 215 may be used to track the number of times a given location in on-chip memory has been accessed. In some embodiments, the locations in memory tracked by a given counter may include individual textures or portions of textures (e.g., individual tiles). Importantly, this statistical information is generated on the graphics processor hardware 105 itself (and may be communicated entirely internally), without the need to send or receive information to or from the CPU 205.

[0034] According to some embodiments, a given graphics processor hardware may include multiple channels for different types of work to be performed on the graphics processor hardware. For example, in some cases, the graphics processor hardware may include a dedicated "compute" channel that is used during Figure 2 2 is shown as compute channel 220. Compute channel 220 may include various numbers of cores and / or caches and may be used to process any number of general computational tasks (e.g., such as mapping and unmapping of textures or portions of textures to regions in on-chip memory and / or sequencing between different thread groups) in a parallelized manner on the graphics processor hardware.

[0035] Thus, in some such embodiments, the hardware counters 215 may expose their statistical information directly to the compute channel 220, e.g., via the interface 240A. The compute channel 220 may also include a mapping logic module 225A that may be configured to determine a priority order for mapping or unmapping textures or portions of textures in on-chip memory based at least in part on the statistical information received from the hardware counters 215. For example, as described above, in some embodiments, the most frequently accessed textures or portions of textures (e.g., tiles) will receive a higher priority with respect to allocating memory backups than tiles that have not been loaded recently, thereby providing a finer granularity for controlling which portions of which textures need to be resident in memory at any given moment.

[0036] The compute channel 220 may further include a sequencing logic module 230A that may be configured to provide sequencing guarantees across various textures or portions of textures requesting memory backup. For example, according to some embodiments, the sequencing logic module 230A may implement a row-first sequencing scheme in which consecutive elements in a row reside adjacent to each other in memory. Generally speaking, not all GPUs can guarantee sequencing between different thread groups within a given compute schedule. Therefore, according to some embodiments, in order to ensure the desired sequencing, the graphics processor hardware may be configured to launch a single thread group, for example, up to the maximum number of possible threads allowed by the hardware, so that these threads can coordinate with each other to ensure the desired sequencing. For example, in some embodiments, memory can be acquired in a specified order (e.g., as specified in an API), and then the memory can be applied to textures or portions of textures according to the specified order.

[0037] In one particular implementation using a single thread group dispatch to process the sequencing of the entire texture region, the following steps may be followed: 1) Calculate the total number of pages in memory required to map the entire region (note that the region may already have some pages mapped); 2) Check if a sufficient number of pages are available in memory (e.g., in some implementations, the memory may take the form of a circular buffer / queue); 3) If there are a sufficient number of pages, update the circular buffer / queue of memory with the number of pages available; 4) If there are not a sufficient number of pages, record the number of pages that are actually available and update the circular buffer / memory queue once; and 5) Each thread in the thread group checks if there is a valid page to map to the thread, and if so, maps to the thread. In this way, optimal allocation of all pages in row-major order may be guaranteed, while (ideally) using the compute channel 220 capabilities of the GPU to maximally parallelize the execution of the algorithm.

[0038] In some embodiments, sequencing logic 230 may perform sequencing across mapping operations and / or sequencing within a single mapping operation. In the case of sequencing across mapping operations, a data host-level synchronization process may be employed to ensure the desired sequencing. In the case of sequencing within a single mapping operation, techniques such as those described above (e.g., a "row-first sequencing scheme") may be employed to ensure the desired sequencing, thereby enforcing guarantees of repeatability and determinism.

[0039] like Figure 2 As shown, mapping logic 225 and sequencing logic 230 may be implemented in various places within graphics processor resources 112. As described above, mapping logic 225A and sequencing logic 230A may be implemented in hardware 220 of the compute channel. However, in yet other embodiments, mapping logic 225B and sequencing logic 230B may be implemented in graphics processor firmware 104, for example, via different interfaces 240B. For example, if the graphics processor has a dedicated firmware processor or a programmable front end, the mapping and sequencing logic may be implemented in such firmware. In yet other embodiments, if so desired, the mapping and sequencing logic may be implemented in fixed function hardware on the graphics processor.

[0040] Figure 3 is an example of a scene 300 rendered using textures from varying mipmap levels. For example, areas of the 3D scene 300 that are closest to the user's perspective and / or most likely to be interacted with, inspected by the user, etc., may benefit from being mapped with a higher level of detail texture from a corresponding texture mipmap, while the system may benefit from mapping areas of the 3D scene 300 that are more peripheral or less prominent from the user's perspective at a lower level of detail.

[0041] Figure 4 yes Figure 3The mipmap levels of the rendered scene 300 use an example of diagram 400. Figure 4 As shown, various LOD versions of textures that appear in scene 300 may be loaded for different portions of the scene being rendered. For example, the upper-leftmost area of ​​mipmap level usage map 400 is shaded in the lightest gray tone and labeled "LOD 0," indicating that the portion of scene 300 corresponding to the area shaded in this gray tone in mipmap level usage map 400 will utilize the highest quality level texture available in the texture mipmap. Just to the right of the area labeled "LOD 0" is an area shaded in a slightly darker gray tone, labeled "LOD 1." LOD 1 may reflect a lower quality LOD than LOD 0, e.g., LOD 1 may be Figure 4 The area marked "LOD0" in the .

[0042] Continuing to move on the usage map 400, the next slightly darker shaded area is labeled "LOD 2", which indicates an even lower quality than the "LOD 1" area (e.g., the resolution is two times lower than LOD 1). Figure 4 The darkest grey shaded areas marked as "LOD 3" in the Graph represent areas where the lowest quality level textures (e.g., two times lower resolution compared to LOD 2) are used for texture mapping. According to some embodiments, and to achieve even greater efficiency improvements, only necessary portions of the necessary LOD mipmaps (e.g., the most recently and / or most frequently accessed tiles) may be backed up in memory at any given moment, thereby providing the programmer with finer grained control over which portions of the various textures in the scene are or are not loaded into memory.

[0043] Figure 5 5 is a block diagram 500 illustrating the relationship between tiles 503 of an exemplary tiled texture 502 and their corresponding memory backups 508 in a memory heap 504. As described above, according to some embodiments, statistical information obtained, for example, from hardware counters, can be used to help prioritize the mapping and unmapping of textures, even on a per-tile basis for a given texture. For a given tile mapped 506 to memory, the memory backup in a given pool of allocated memory (e.g., memory heap 504) can include a corresponding memory page 508 in which the relevant portion of the relevant texture is written.

[0044] According to some embodiments, by default, all tiles may be unmapped. According to such embodiments, a read from an unmapped area 510 may result in the return of all zero values ​​512. An attempt to write to an unmapped area may be abandoned. In the case where one or more textures are blended with each other, the return of all zero values ​​for an unmapped area in one of the textures may not present any undesirable visual artifacts, but may only produce a less strong value or version of the mapped value in the blending operation. In some embodiments, the graphics processor management system may also provide a positive ACK / NACK (i.e., confirmation) indicator about the mapping status of a given area, so that the developer will know whether the sample is from an unmapped area in memory or from a mapped area in memory that happens to have all zero values.

[0045] In an embodiment, a developer may manually map tiles into physical memory, for example, by using an API. However, in embodiments where mapping and unmapping are performed on the timeline of the graphics processor, the developer may also be responsible for unmapping textures before they are released from memory. If the developer fails to unmap a given texture before it is released, the mapped tile will occupy the memory heap until the heap itself is released (which may or may not be desired by the developer).

[0046] According to some embodiments, the size and position of each mapped region may be aligned to a tile or texture boundary. If this requirement is not met, the region may expand outward to the nearest tile boundary. If a region maps more tiles than are available on the heap, the tiles may be mapped on a first-come, first-served basis (e.g., following row-major order).

[0047] For each texture, the mipmaps (starting from a certain level) can be placed in a single structure (referred to herein as the "tail") to save additional memory. From a developer's perspective, the tail can interact with a single page in memory, and thus, the developer only has to explicitly map memory for the first mip level in the tail structure. This mapping ensures that all lower mip levels are mapped.

[0048] Figure 6 is a flow diagram of operation 600 for managing graphics hardware resources using sparsely backed-up texture resources. In some embodiments, operation 600 may be initiated or performed by one or more processors in response to one or more instructions stored in a computer-readable storage medium. For example, operation 600 may be performed at least in part by Figure 1 The graphics hardware resource allocation system 108 shown in FIG. Figure 6 The use and discussion of is merely an example to facilitate explanation and is not intended to limit the present disclosure to this specific example. Figure 6While the blocks within operation 600 are shown to be performed in a sequential order, operation 600 is not limited to that order.

[0049] At block 602, operation 600 may initially allocate a pool of memory for a desired number of textures (e.g., all textures or a subset thereof required to render the current graphics scene). Next, at block 604, the operation may load a desired LOD version for each desired texture for which memory has been allocated at block 602. As described above, the various LOD versions of a given texture may be stored in the form of mipmaps. Next, at block 606, the loaded textures may be used to render the graphics scene.

[0050] At block 608, operation 600 may begin obtaining and / or accumulating statistical information over time for each texture (or portion of a texture) that has been loaded. As detailed above, according to some embodiments, the statistical information may include usage information, such as a counter value reflecting the number of times a particular texture (or portion of a texture, such as a "tile" area within a texture) has been accessed.

[0051] Next, at block 610, for each loaded texture, the textures (or portions of textures, depending on the level of granularity of the reported statistical information) may be sorted based at least in part on the obtained statistical information. For example, the textures or portions of a given texture that have been accessed and loaded from memory the most times may be sorted to the top of the sort order, and the textures or portions of a given texture that have been accessed and loaded from memory the least times may be sorted to the bottom of the sort order.

[0052] Next, at block 612, operation 600 may determine to load backup data (i.e., "map" to locations in memory) for only those textures or portions of textures that have statistics that exceed a threshold (e.g., a threshold minimum number of times loaded from memory within a given time interval). Textures or portions of textures that do not exceed the threshold may be "unbacked up" or "unmapped" in physical memory, at least until the statistics for such textures or portions of textures again exceed the threshold, at which time they may be mapped or backed up at locations in physical memory again. As described above, according to some embodiments, one or more sequencing rules may be applied to determine which textures (or portions of textures) are actually mapped in memory, for example, in situations where there is not enough physical memory available to back up all desired textures or portions of textures at a given moment. Finally, at block 614, the graphics rendering of the scene may be updated based on the newly mapped (or unmapped) textures in physical memory. Operation 600 may end without additional frames presenting graphics information.

[0053] Other Exemplary Systems

[0054] Figure 7 700 is a simplified block diagram of one embodiment of an exemplary graphics processor. In the illustrated embodiment, the graphics processor 700 includes a vertex pipeline 705, a fragment pipeline 710, a programmable shader 715, a texture processing unit (TPU) 720, an image write buffer 725, and a memory interface 730. In some embodiments, the graphics unit 700 can be configured to process both vertex data and fragment data using the programmable shader 715, which can be configured to process data (e.g., graphics data) in parallel using multiple execution pipelines or instances. In other embodiments, the multiple execution pipelines correspond to multiple execution units of the processing circuit hardware resource allocation system.

[0055] The vertex pipeline 705 may include various fixed-function hardware configured to process vertex data. The vertex pipeline 705 may be configured to communicate with the programmable shader 715 to coordinate vertex processing, and to send the processed data to the fragment pipeline 710 and / or the programmable shader 715 for further processing. The fragment pipeline 710 may include various fixed-function hardware configured to process pixel data. The fragment pipeline 710 may be configured to communicate with the programmable shader 715 to coordinate fragment processing. The fragment pipeline 710 may also be configured to perform rasterization on polygons received from the vertex pipeline 705 and / or the programmable shader 715 to generate fragment data. The vertex pipeline 705 and / or the fragment pipeline 710 may be coupled to a memory interface 730 (coupling not shown) to access graphics data.

[0056] Programmable shader 715 can be configured to receive vertex data from vertex pipeline 705 and fragment data from fragment pipeline 710 and / or TPU 720. Programmable shader 715 can be further configured to perform vertex processing tasks on vertex data, including various transformations and / or adjustments of vertex data. For example, programmable shader 715 can also be configured to perform fragment processing tasks, such as texturing and shading, on pixel data. Programmable shader 715 can include multiple execution instances for processing data in parallel. In various embodiments, parts (e.g., execution units, registers, arithmetic logic units, memory locations, etc.) of programmable shader 715 can be used by multiple processes (e.g., vertex processing tasks, computational processing tasks, and fragment processing tasks). In practice, different parts of programmable shader 715 can be assigned to different processes during the execution of those processes. In one or more embodiments, programmable shader 715 can be coupled to other programmable and / or fixed function elements in a graphics unit in any of various suitable configurations. Figure 7 The configurations shown in are included for illustration only.

[0057] The TPU 720 may be configured to schedule fragment processing tasks from the programmable shader 715. In some embodiments, the TPU 720 may be configured to pre-fetch texture data and assign initial colors to fragments for further processing by the programmable shader 715 (e.g., via the memory interface 730). In other embodiments, the TPU 720 may be configured to provide fragment compositions in one or more normalized integer formats or one or more floating point formats. In yet other embodiments, the TPU 720 may be configured to provide a group of four ("fragment quads") of fragments in a 2×2 format that will be processed by a group of four in the programmable shader 715. The image write buffer 725 may be configured to store the processed tiles of the image, and may further perform final operations on the rendered image before the image is transferred to the frame buffer (e.g., in the system memory via the memory interface 730). In various embodiments, the memory interface 730 may facilitate communication between the graphics unit 700 and one or more of the various memory hierarchical structures.

[0058] Next go to Figure 8 , which is a block diagram illustrating an exemplary embodiment of a computing system 800 that includes at least a portion of a graphics hardware resource allocation system. The computing system 800 includes a graphics processor 850.

[0059] In some embodiments, some or all of the elements of computing system 800 may be included in a system on a chip (SoC). In other embodiments, computing system 800 may be included in a mobile device. Therefore, in at least some embodiments, the area and power consumption of computing system 800 may be important design considerations. In the illustrated embodiment, computing system 800 includes communication fabric 805, graphics processor 850, computing complex 810, input / output (I / O) bridge 815, cache / memory controller 820, and display unit 825. Although computing system 800 illustrates graphics processor 850 as a separate device of computing system 800 connected to fabric 805, in other embodiments, graphics processor 850 may be connected to or included in other components of computing system 800.

[0060] In addition, the computing system 800 may include a plurality of graphics processors 850. The plurality of graphics processors 850 may correspond to different embodiments or the same embodiment. The structure 805 may include various interconnects, buses, MUXs, controllers, etc., and may be configured to facilitate communication between various elements of the computing system 800. In some embodiments, portions of the structure 805 are configured to implement various communication protocols. In other embodiments, the structure 805 implements a single communication protocol, and the elements coupled to the structure 805 may be converted internally from a single communication protocol to other communication protocols.

[0061] In the illustrated embodiment, the computing complex 810 includes a bus interface unit (BIU) 830, a cache 835, and cores 840 and 845. In some embodiments, cores 840 and 845 may correspond to execution cores. In various embodiments, the computing complex 810 includes various numbers of cores and / or caches. For example, the computing complex 810 may include 1, 2, or 4 processor cores, or any other suitable number. In some embodiments, cores 840 and / or cores 845 include internal instruction and / or data caches. In some embodiments, a consistency unit (not shown) in the structure 805, cache 835, or elsewhere in the computing system 800 is configured to maintain consistency between the various caches of the computing system 800. The BIU 830 may be configured to manage communications between the computing complex 810 and other elements of the computing system 800. Processor cores such as cores 840 and 845 may be configured to execute instructions of a specific instruction set architecture (ISA), which may include operating system instructions and user application instructions. The I / O bridge 815 may include various elements configured to implement, for example, universal serial bus (USB) communications, security, audio, and / or low-power always-on functionality. For example, the I / O bridge 815 may also include interfaces such as pulse width modulation (PWM), general purpose input / output (GPIO), serial peripheral interface (SPI), and / or inter-integrated circuit (I2C). Various types of peripherals and devices may be coupled to the computing system 800 via the I / O bridge 815.

[0062] In some embodiments, the graphics processor 850 may be coupled to the computing system 800 via the I / O bridge 815. The cache / memory controller 820 may be configured to manage data transfer between the structure 805 and one or more caches and / or memories (e.g., non-transitory computer-readable media). For example, the cache / memory controller 820 may be coupled to an L3 cache, which in turn may be coupled to system memory. In other embodiments, the cache / memory controller 820 is directly coupled to the memory. In some embodiments, the cache / memory controller 820 includes one or more internal caches. In some embodiments, the cache / memory controller 820 may include or be coupled to one or more caches and / or memories that include instructions that, when executed by one or more processors (e.g., the computing complex 810 and / or the graphics processor 850), cause a processor, multiple processors, or cores to initiate or execute the above-referenced Figures 1 to 7Some or all of the operations described. The display unit 825 may be configured to read data from the frame buffer and provide a stream of pixel values ​​for display. In some embodiments, the display unit 825 may be configured as a display pipeline. In addition, the display unit 825 may be configured to mix multiple frames to generate an output frame. In addition, the display unit 825 may include one or more interfaces (e.g., MIPI or embedded display port eDP) for coupling to a user display (e.g., a touch screen or an external display).

[0063] As used herein, the term "coupled to" may indicate one or more connections between elements, and a coupling may include intermediate elements. Figure 8 , display unit 825 may be described as being "coupled to" computing complex 810 via structure 805. In contrast, in Figure 8 In the illustrated embodiment, the display unit 825 is "directly coupled" to the structure 805 because there are no intervening elements.

[0064] Within the present disclosure, different entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured to" perform one or more tasks or operations. This expression—[an entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., a physical thing, such as an electronic circuit). More specifically, this expression is used to indicate that this structure is arranged to perform one or more tasks during operation. A structure may be said to be "configured to" perform a task even if the structure is not currently being operated. "A memory device configured to store data" is intended to cover, for example, an integrated circuit having a circuit system that performs that function during operation, even if the integrated circuit in question is not currently in use (e.g., a power source is not connected to it). Thus, an entity described or stated as "configured to" perform a task refers to a physical thing used to implement the task, such as a device, a circuit, a memory storing executable program instructions, etc. The phrase is not used herein to refer to an intangible thing. The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed field programmable gate array (FPGA) would not be considered to be "configured to" perform a particular function, although it might be "configurable to" perform that function after being programmed.

[0065] As used herein, the term "based on" is used to describe one or more factors that influence a determination. This term does not exclude that there may be additional factors that may also influence the determination. That is, the determination may be based only on the specified factors or on the specified factors and other unspecified factors. Consider the phrase "A is determined based on B." This phrase specifies that B is a factor used to determine A or that B influences the determination of A. This phrase does not exclude situations where the determination of A may also be based on some other factor such as C. This phrase is also intended to cover embodiments in which A is determined based only on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0066] As used herein, the phrase "in response to" describes one or more factors that trigger an effect. The phrase does not exclude the possibility that other factors may influence or otherwise trigger the effect. That is, the effect may be responsive only to these factors, or may be responsive to the specified factors as well as other unspecified factors. Consider the phrase "in response to B, A is executed." This phrase specifies that B is the factor that triggers the execution of A. This phrase does not exclude situations in which the execution of A may also be responsive to some other factor, such as C. The phrase is also intended to cover embodiments in which A is executed only in response to B.

[0067] As used herein, the terms "first," "second," and the like serve as labels for the nouns that follow them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise specified. For example, in a processing circuit that includes six clusters, the terms "first cluster" and "second cluster" may be used to refer to any two of the six clusters, rather than to, for example, two specific clusters (e.g., logical clusters 0 and 1).

[0068] In the following description, for the purpose of explanation, a lot of specific details are set forth in order to provide a thorough understanding of the disclosed concept. As part of this description, some of the drawings in the drawings of the present disclosure represent structures and devices in the form of block diagrams to avoid blurring the novel aspects of the disclosed concept. For the sake of clarity, all features of the actual specific implementation may not be described. In addition, as part of this specification, some drawings of the present disclosure may be provided in the form of flow charts. The blocks in any particular flow chart may be presented in a specific order. However, it should be understood that the specific order of any given flow chart is only used to illustrate an embodiment. In other embodiments, any element in the various elements depicted in the flow chart may be deleted, or the illustrated sequence of operations may be performed in a different order, or even simultaneously. In addition, other embodiments may include additional steps that are not shown as a part of the flow chart. In addition, the language used in the present disclosure has been selected primarily for readability and guidance purposes, and may not be selected to delimit or limit the subject matter of the present invention, thereby resorting to the necessary claims to determine such subject matter of the invention. Reference in this disclosure to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter, and multiple references to "one embodiment" or "an embodiment" should not be construed as necessarily all referring to the same embodiment.

[0069] It should be understood that in the development of any actual implementation (as in any software and / or hardware development project), many decisions must be made to achieve the developer's specific goals (e.g., to meet system and business-related constraints), and that these goals may vary from one implementation to another. It should also be understood that such development work may be complex and time-consuming, but is nonetheless a routine task for those of ordinary skill in the design and implementation of a graphics processing system having the benefit of this disclosure.

[0070] It should be understood that the above description is intended to be illustrative and not limiting. The material has been presented to enable any person skilled in the art to make and use the disclosed subject matter protected by the claims and the material is provided in the context of specific embodiments, variations of which will be apparent to those skilled in the art (e.g., some of the disclosed embodiments may be used in conjunction with each other). Therefore, the scope of the present invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. A non-transitory program storage device readable by one or more processors and comprising instructions stored thereon to cause the one or more processors to: allocating a first region in a first physical memory for a first one or more graphics textures; loading a desired level-of-detail (LOD) version of each of the first one or more graphics textures; rendering a first graphics scene using the loaded versions of the first one or more graphics textures; obtaining statistical information related to at least a portion of at least one of the first one or more graphics textures, wherein the statistical information for a respective at least a portion of each of the first one or more graphics textures comprises a number of memory accesses; sorting each portion of each of the first one or more graphics textures in descending order based at least in part on the obtained statistical information; attempting to load into the first physical memory only those sorted portions having statistics exceeding a threshold; as well as The rendering of the first graphics scene is updated based at least in part on the loaded portion.

2. The non-transitory program storage device of claim 1, wherein the one or more processors are located on a graphics processing unit (GPU).

3. The non-transitory program storage device of claim 2, wherein the instructions for loading only those sorted portions having statistics exceeding a threshold are executed by a compute channel of the GPU. 4 . The non-transitory program storage device of claim 1 , wherein the instructions for loading only those sorted portions having statistics exceeding a threshold are executed in a single thread group. 5 . The non-transitory program storage device of claim 1 , wherein the portion of each of the first one or more graphics textures comprises a tile.

6. The non-transitory program storage device of claim 1, wherein the instructions for loading only those sorted portions having statistics exceeding a threshold further comprise instructions for loading the sorted portions according to row-major order.

7. The non-transitory program storage device of claim 1, wherein the obtained statistical information comprises a counter value obtained from a GPU.

8. The non-transitory program storage device of claim 7, wherein each counter value corresponds to a tile from one of the first one or more graphics textures.

9. A system comprising: a first physical memory; and a processor that interacts with the first physical memory, wherein the processor is configured to execute instructions to: allocating a first region in the first physical memory for a first one or more graphics textures; loading a desired LOD version of each of the first one or more graphics textures; rendering a first graphics scene using the loaded versions of the first one or more graphics textures; obtaining statistical information related to at least a portion of at least one of the first one or more graphics textures, wherein the statistical information for a respective at least a portion of each of the first one or more graphics textures comprises a number of memory accesses; sorting each portion of each of the first one or more graphics textures in descending order based at least in part on the obtained statistical information; attempting to load into the first physical memory only those sorted portions having statistics exceeding a threshold; as well as The rendering of the first graphics scene is updated based at least in part on the loaded portion.

10. The system of claim 9, wherein the processor is part of a GPU.

11. The system of claim 10, wherein the instructions for loading only those sorted portions having statistics exceeding a threshold are executed by a compute channel of the GPU.

12. The system of claim 9, wherein the instructions for loading only those sorted portions having statistics exceeding a threshold are executed in a single thread group.

13. The system of claim 9, wherein the portion of each of the first one or more graphics textures comprises a tile.

14. The system of claim 9, wherein the instructions for loading only those sorted portions having statistics exceeding a threshold further comprise instructions for loading the sorted portions in row-major order.

15. The system of claim 9, wherein the obtained statistical information comprises a counter value obtained from a GPU.

16. The system of claim 15, wherein each counter value corresponds to a tile from one of the first one or more graphics textures.

17. A computer-implemented method comprising: allocating a first region in a first physical memory for a first one or more graphics textures; loading a desired LOD version of each of the first one or more graphics textures; rendering a first graphics scene using the loaded versions of the first one or more graphics textures; obtaining statistical information related to at least a portion of at least one of the first one or more graphics textures, wherein the statistical information for a respective at least a portion of each of the first one or more graphics textures comprises a number of memory accesses; sorting each portion of each of the first one or more graphics textures in descending order based at least in part on the obtained statistical information; attempting to load into the first physical memory only those sorted portions having statistics exceeding a threshold; as well as The rendering of the first graphics scene is updated based at least in part on the loaded portion.

18. The computer-implemented method of claim 17, wherein the method is performed on a GPU.

19. The computer-implemented method of claim 18, wherein the obtained statistical information comprises a counter value obtained from the GPU.

20. The computer-implemented method of claim 17, wherein loading only those sorted portions having statistics exceeding a threshold further comprises loading the sorted portions according to row-major order.

Images