Techniques for preloading textures when rendering graphics

Abstract

The present invention provides a technique for preloading textures when rendering graphics. Specifically, systems and methods for improved texture mapping and graphics processing are described. According to an exemplary implementation, all or a portion of a texture block is prefetched by a processing unit into an intermediate cache so that the same processing unit or another processing unit can subsequently obtain the prefetched texture block from the intermediate cache. Furthermore, in some exemplary implementations, control circuitry associated with the intermediate cache can throttle prefetch requests to avoid an excessive number of prefetch requests being received by a memory system and / or an interconnect system. Additionally, in some implementations, deduplication of prefetch requests can be performed at the intermediate cache and / or the processing unit. Some implementations also include efficient techniques for calculating the address of the next texture block to be prefetched.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 62 / 876,957, filed on July 22, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to computer graphics, texture mapping, and more particularly, to systems and methods for fetching textures from memory for graphics processing. Background Art

[0004] Smart TVs, portable phones and other smart devices, video game systems, transportation user interfaces (such as car and airplane head-up displays), smart glasses and virtual reality goggles, and many other devices all have the ability to generate 3D computer graphics. Users often expect their computer graphics systems to generate realistic images of arbitrarily complex scenes in real time. A common approach to reducing the complexity of a scene is to model it as a surface onto which additional images are mapped. Like a painting on canvas, a rendered 3D model can appear to contain many complex elements (e.g., the shadows and lighting of Notre Dame, the thousands of leaves on a tree, the hundreds of bricks in a brick wall, or the delicate wood grain of a conference room table) without having to model each such element. Essentially, an image displaying this complexity is virtually pasted or mapped onto the surface of the rendered model. The additional image is often called a "texture," and the process of applying a texture to a model's surface is called "texture mapping."

[0005] Typically, textures are captured or created in advance and stored in memory. When the graphics system renders a surface, it fetches the necessary texture image data (texture elements or "texels") from memory and applies the texture image to the surface using a mapping process that scales and filters the image appropriately. Because textures themselves can be rich, complex images, they are often compressed to reduce storage requirements. A graphics processing "texture mapper" typically includes the ability to decompress the texture image after fetching it from memory. Additionally, textures can often be stored in a variety of different formats, depending on their intended use.

[0006] For example, some previous graphics systems provide special texture caches and surface memories for storing textures in a special internal format that provides more efficient access. As an example, according to the NVIDIA CUDA Programming Guide v10.1.105 for programming certain NVIDIA products, a CUDA array (an opaque memory layout optimized for texture acquisition) is one-dimensional, two-dimensional, or three-dimensional and consists of elements, each of which has 1, 2, or 4 components, which can be signed or unsigned integer values ​​or floating-point values ​​of different resolutions. In certain exemplary non-limiting NVIDIA graphics systems, a CUDA array can be accessed by a kernel through texture acquisition or surface reads and writes. A texture object or texture reference specifies a texture based on a piece of texture memory being acquired. The dimensions specify whether the texture is addressed as a one-dimensional array using one texture coordinate, as a two-dimensional array using two texture coordinates, or as a three-dimensional array using three texture coordinates. The elements of the array are texture pixels. The width, height, and depth of a texture refer to the size of the array in each dimension. In such an implementation, the texture fetch itself may provide linear filtering or "mip mapping," which may include resampling to map the texture to a specific location of the surface on which the texture is to be displayed.

[0007] Storing textures in different formats for more efficient access is also advantageous. See, for example, U.S. Patent No. 7,916,149, which is incorporated herein by reference. In particular, the so-called "pitch linear" format is a common format for storing textures. In this "pitch linear" format, texels are stored sequentially in memory, i.e., texels are sequentially stored in a given row ("pitch") of a rectangular texture area, then texels are sequentially stored in the next row, and so on. In this format, each row of texels is stored in sequentially increasing memory locations. This "pitch linear" layout works well if texels are accessed horizontally across each row. However, typical access patterns typically have 2D (or 3D for 3D textures) spatial locations. Therefore, accesses that are closely spaced in time may proceed to nearby texels in any direction (not just horizontally). The direction of this spatial locality is generally unpredictable, as it often depends on the viewer's point of view. Consequently, the same texture will have different spatial locality patterns as the viewer moves around.

[0008] Alternatively, the texture pixel data can be stored in a computer-readable medium in a "block linear" format. In a "block linear" format, the memory is logically organized into multiple blocks that are a function of the specific page size (e.g., cache line size) of the particular implementation. The texture pixels are stored in xy (z) block order. Within each block, the texture pixels are stored sequentially in memory, that is, the texture pixels in a given block are stored sequentially, then the texture pixels in the next block are stored sequentially, and so on. In this format, each texture pixel block is stored in sequentially increasing memory locations. In an exemplary block linear format, a page of memory contains 8 consecutive 4-byte texture pixels (e.g., 32 bytes) in 32 sequential rows. The memory access cost (e.g., latency) of the block linear format is proportional to the total number of page boundaries.

[0009] One way to further reduce paging penalties using this "linear block" format is to organize or define such blocks into memory "tiles." All texture pixels within a tile reside in the same physical or virtual memory page. This tiling can reduce the penalty for traversing memory pages. See, for example,

[0010] https: / / developer.nvidia.com / gpugems / GPUGems2 / gpugems2_chapter12.html Electronic version of Chapter 12 of GPU Gems 2 (NVIDIA 2005).

[0011] While legacy and backward compatibility may be an issue, the above approach has been significantly impacted by advances in technology and concepts. In particular, graphics processing units (GPUs) are now more general-purpose, high-performance computing devices that are not limited to graphics processing in their structure and operation. For these and other reasons, the specialized texture caches of the past are rarely used and useful. Instead, more general-purpose memory architectures using multiple cache memory levels (similar in some respects to CPU cache memory architectures) have been increasingly used to reduce memory latency in high-performance GPUs.

[0012] More specifically, as processor speeds increase, other aspects of processor latency, such as memory access latency and intra-chip communication latency, become more significant. Parallel processors, such as modern high-performance graphics processing units (GPUs), have multiple fast processing cores and are therefore typically even more significantly affected by memory access latency, etc. The trend of modern GPUs towards increasingly significant memory access latency may impair the performance scaling of new multi-processor GPUs. For example, in recently released GPUs, it is estimated that the overall computing power of the entire processing unit has increased by 50%, while the performance improvement is only 32-39% on average, depending on the workload. As larger chips are manufactured, the proportion of the overall system performance loss caused by processor latency limitations is expected to continue to increase.

[0013] The memory access requirements of a GPU performing graphics processing can be significantly different from the memory access requirements of a CPU performing a typical CPU workload. Because textures can be very large, these texture accesses are the most affected by memory latency, and they also typically constitute the largest chunk of dynamic random access memory (DRAM) read traffic in typical graphics workloads. The texture processing unit and / or other processors can load textures from memory (also known as "texture mapping") so that sampled texture values ​​can be provided for use in shaders and image rendering. Because there is insufficient latency hiding in the texture mapping unit or the texture mapping unit does not have other capabilities to prevent texture retrieval latency from causing a bottleneck, the processor may not be able to achieve its optimal performance when long memory accesses are encountered in the workload. For example, memory latency outliers of greater than 1000 clock cycles are not uncommon. This slowdown of the processor can often lead to underutilization of the memory subsystem and prevent the memory system from servicing requests.

[0014] Conventional systems employ a number of techniques to mitigate the performance degradation caused by long memory access latencies. Such techniques include pipelining (in both CPU and GPU environments) and data prefetching (primarily in CPU environments). However, these conventional techniques may not be sufficient to address the latency associated with accessing the large and varied textures of modern graphics processing. Therefore, there is a need for improved low-latency systems and techniques for accessing textures in graphics processing systems. Summary of the Invention

[0015] The exemplary embodiments remedy some of the deficiencies of the above-described techniques for texture processing.

[0016] An exemplary embodiment provides a method for displaying a scene. The method includes: while performing texture mapping on the scene using a first block of a stored texture, generating, by a processor, a prefetch request to retrieve all or a portion of a second block of the stored texture from a first level memory in a memory hierarchy; retrieving all or a portion of the second block from the first level memory in response to the prefetch request; storing all or a portion of the retrieved second block in an area of ​​a second level memory in the memory hierarchy, wherein the area of ​​the second level memory is accessible to the processor and another processor, and wherein the second level memory is an intermediate level memory between the first level memory and a third level memory in the memory hierarchy; performing texture mapping on the scene using all or a portion of the retrieved second block; and rendering the scene to a display device. In some exemplary implementations, texture mapping the scene using the first block may be performed by the processor, and texture mapping the scene using the second block may be performed by the other processor.

[0017] Another exemplary embodiment provides a parallel processing system for displaying a scene. The system includes multiple processors, a cache hierarchy including at least a first-level cache memory and a second-level cache memory, a display interface, and a memory interface configured to provide the multiple processors with access to off-chip memory. The multiple processors and control circuitry associated with the cache hierarchy are configured to, while performing texture mapping on a scene using a first block of a stored texture, generate a prefetch request by a first processor from the multiple processors to retrieve all or a portion of a second block of the stored texture from a memory hierarchy including the cache hierarchy. The multiple processors and control circuitry associated with the cache hierarchy are further configured to: retrieve all or a portion of the requested second block from the off-chip memory via the memory interface in response to the prefetch request; store the retrieved second block in a region of the second-level cache memory, wherein the region of the second-level cache memory is accessible by the first processor and the second processor from the multiple processors, and wherein the second-level cache memory is an intermediate level of memory between the off-chip memory and the first-level cache memory; perform texture mapping on the scene using the retrieved second block; and render the scene to a display device via the display interface.

[0018] An exemplary embodiment provides a system on chip (SoC) including at least one central processing unit (CPU) and at least one parallel processing unit (PPU) connected to the CPU. Each PPU includes a plurality of multiprocessors, a plurality of special function units, a cache hierarchy including at least a first-level cache memory and a second-level cache memory, and a memory interface configured to provide access to an off-chip memory.

[0019] A plurality of special function units and control circuitry associated with a cache hierarchy are configured to perform a plurality of operations in response to instructions received from one of the multiprocessors. The operations include: while texture mapping a scene using a first block of a stored texture, generating a prefetch request by a first special function unit from the plurality of special function units to retrieve all or a portion of a second block of the stored texture from a memory hierarchy comprising the cache hierarchy. The operations also include: in response to the prefetch request, retrieving all or a portion of the requested second block from an off-chip memory via a memory interface; storing all or a portion of the retrieved second block in an area of ​​a second-level cache memory, wherein the area of ​​the second-level cache memory is accessible by the first special function unit and the second special function unit from the plurality of special function units, and wherein the second-level cache memory is an intermediate level of memory between the off-chip memory and the first-level cache memory; texture mapping the scene using all or a portion of the retrieved second block; and rendering the scene to a display device via a display interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A is a block diagram of a system that provides improved texture rendering, according to certain exemplary embodiments.

[0021] Figure 1B An exemplary allocation of textures to processor cores is shown.

[0022] Figure 1C Schematically shows a texture organized in memory according to a pitch linear format.

[0023] Figure 1D Schematically shown is a texture organized in memory according to a block linear format.

[0024] Figure 1E The relationship between the virtual address of a chunk of texture data in memory and the rendering time of the chunk is shown according to some example embodiments.

[0025] Figure 1F Another exemplary illustration of texture data organized in memory and its relationship to textures rendered on screen is provided according to some exemplary embodiments.

[0026] Figure 1G The relationship between textures, chunks, and blocks is shown according to some example embodiments.

[0027] Figure 2A According to some exemplary embodiments, Figure 1A Block diagram of a graphics processing unit used in a system.

[0028] Figure 2B A flow chart illustrating a process performed by a texture processing unit of a processor and / or GPU to prefetch texture data, according to some example embodiments.

[0029] Figure 2C shows a response to a Figure 2B A flow chart of a process for prefetching initiated by a texture processing unit of a processor and / or GPU and performed by a memory hierarchy is shown.

[0030] Figure 3 A parallel processing unit according to one embodiment is shown.

[0031] Figure 4A According to one embodiment, Figure 3 A general-purpose processing cluster within a parallel processing unit.

[0032] Figure 4B According to one embodiment, Figure 3 A memory partitioning unit of a parallel processing unit.

[0033] Figure 5A According to one embodiment, Figure 4A Streaming multiprocessor.

[0034] Figure 5B According to one embodiment, the Figure 3 A conceptual diagram of a processing system implemented with a parallel processing unit (PPU).

[0035] Figure 5C Illustrated are exemplary systems in which the various architecture and / or functionality of various previous embodiments may be implemented.

[0036] Figure 6 According to one embodiment, Figure 3 Conceptual diagram of the graphics processing pipeline implemented by the PPU. DETAILED DESCRIPTION

[0037] Embodiments of the present invention provide more efficient texture mapping by reducing memory access latency associated with accessing texture data in memory during graphics processing. While pipelining can reduce the slowdown associated with texture access in various applications, pipelining often provides insufficient latency hiding for the longer latencies associated with certain texture accesses. Conventional prefetching techniques, such as those used by CPUs, may not be effective enough for texture processing in environments with multiple processors. Exemplary embodiments provide improved texture prefetching techniques and systems that result in improved memory utilization and / or reduced effective memory latency during graphics processing, thereby resulting in improved system performance.

[0038] Certain exemplary embodiments improve memory utilization and / or reduce effective memory latency in a GPU. For example, these improvements can be achieved by speculatively fetching data that has not yet been actually requested by any processing unit into an intermediate cache, such as a Level 2 (L2) cache. Speculative fetching before a processing unit actually requests data is referred to herein as "prefetching," and the request to initiate such a speculative fetch is referred to as a "prefetching request." More specifically, certain exemplary embodiments provide for prefetching part or all of another (e.g., the next sequential) block of texture information into the GPU's L2 cache while the currently loaded block of texture information is being used and processed by the texture processing unit and / or other processing units of the GPU. This speculative prefetching can sometimes retrieve texture data that is ultimately not actually used or needed (e.g., like a NASCAR pit crew fetching a new set of tires in case a racer needs to change tires before the end of a race), but can effectively reduce the latency of retrieving new texture data from memory when the prefetched data is called upon and needed for texture mapping.

[0039] While speculative prefetching is primarily known in the CPU context, the challenge in applying such techniques in the context of texture retrieval involves predicting which texture mapping data the texture mapper will need next or in the future. Due to a variety of factors, texture mapping is typically not a completely predictable process. Unlike a NASCAR pit crew who knows exactly what tire the driver will need the next time he changes tires, it can be difficult to predict which blocks of texture data the texture mapper will need next, once it has finished processing the texture data already retrieved from memory. Consequently, past attempts at prefetching / speculative loading of texture information have not necessarily proven useful for highly parallel graphics workloads. The lack of demonstrated success is at least partially attributed to the difficulty in predicting or inferring texture mapping access patterns. This challenge is compounded by the high variability of texture memory accesses, which depends on factors such as viewpoint and surface orientation. Adding to the complexity is the need to avoid performance degradation due to cache thrashing and / or extraneous DRAM traffic.

[0040] Certain exemplary embodiments target frequent memory access patterns (such as, for example, full-screen drawing) to improve efficiency. These exemplary embodiments, while they can improve system performance in many types of texture-mapped displays, are particularly effective in improving texture access associated with full-screen drawing. Full-screen drawing is common in gaming applications, but also occurs in many other applications. Full-screen drawing involves rendering an image to the entire screen, typically caused by specific instructions contained in the application. In many games and other applications, when full-screen drawing is performed, the manner in which texture data stored in memory is consumed involves a specific pattern of memory address traversals. The texture prefetching techniques of the exemplary embodiments take advantage of this memory address traversal pattern.

[0041] For example, a full-screen rendering of a texture can actually be most efficiently served by a pitch-linear texture format, since a full-screen rendering can generally be expected to access each texture pixel in ascending order of horizontal rows. However, the same texture can also be used in environments other than full-screen rendering where spatial locality cannot be predicted in advance. As an example, consider a virtual reality scene in which the scene contains a virtual computer display (i.e., a display within a virtual reality display). The user can view the virtual display in the scene from a variety of different directions depending on the viewpoint (e.g., from the side or even upside down). But the user can then move his viewing direction and position to view the virtual display head-on to fill the entire viewing frustum. In this simple example, efficiency can be improved by storing the texture in a block-linear format, but the block-linear format may not be the most efficient storage representation for all uses of the texture (e.g., when it is displayed on the entire screen of a virtual reality display).

[0042] In some exemplary embodiments, when prefetching texture data, the offset of the block from which data is to be prefetched is dynamically calculated on a per-texture basis using information from the texture header. Dynamic calculation allows the system to use separate offsets for textures of different sizes.

[0043] In some exemplary embodiments, the processing unit that issues the prefetch request may be different from the processing unit that subsequently uses and thereby benefits from the prefetched data.

[0044] Furthermore, in certain exemplary embodiments, a prefetch request throttling mechanism is included to ensure that the memory hierarchy is not overwhelmed by prefetches. Additionally, request deduplication can be performed to improve the efficiency of prefetches. These optimizations can result in reduced memory latency as seen by the texture unit and / or other processing units, and can also improve cache hit rates.

[0045] In experiments using exemplary embodiments, prefetching texture information stored in a block linear layout substantially improved system performance. Figure 1D Description) and the associated address calculations implemented by the block linear layout provide certain efficiencies, but embodiments are not limited to the use of block linear layouts of textures.

[0046] System for texture prefetching

[0047] Figure 1A A schematic block diagram of a system 100 configured to perform texture prefetching that reduces performance degradation associated with memory access latency effects, according to some exemplary embodiments, is shown. The system 100 may be a system on a chip (SoC) including one or more central processing units (CPUs) 104 and / or graphics processing units (GPUs) 102.

[0048] GPU 102 may receive instructions and data from CPU 104 via host interface 110. GPU 102 accesses graphics memory 114 via graphics memory controller 105 and may also request data from system memory 108 via host interface 110 and system memory controller 106. System memory 108 may be shared among multiple processing units in the system (e.g., CPU 104 and GPU 102). Graphics data stored in graphics memory 114 (such as, for example, frame buffer information) may be displayed on display 118 via display controller 116.

[0049] The GPU 102 includes multiple processing cores 122 and multiple texture processing units 120. The processing cores may include parallel processing processor units that can run a large number of parallel threads. The processing cores receive and execute instructions from the CPU 104 or other processors in the system 100.

[0050] The GPU 102 also includes a cache hierarchy 124. The cache hierarchy 124 includes two or more levels of cache memory. The cache hierarchy can be constructed so that each higher level of cache memory serves two or more cache memories at the next lower level. For example, a second level cache memory (i.e., "L2 cache") 126 can be accessed by two or more first level cache memories (i.e., "L1 cache") 128. In this disclosure, the cache hierarchy 124, combined with the graphics memory 114 and the system memory 108, is referred to as the "memory hierarchy." During operation of the GPU 102, when attempting to access a piece of data, a particular processing core 122 may first search its L1 cache, then its L2 cache, and then either the graphics memory 114 or the system memory 108.

[0051] This is similar to a chef searching their pantry for a specific ingredient needed for a recipe before going to the store. If the item is in the pantry, the chef can use it immediately. If the item isn't, the chef must send someone to the store to get it, which takes time. If the store doesn't have it in stock, it must order it from a distributor, which takes even more time. Therefore, the "latency," or time delay, associated with getting ingredients from your local pantry at home is very low. However, there's not enough room in your pantry to store everything you might need. Furthermore, some of these items may spoil or lose their freshness if stored in the pantry. Getting such items from the store may be worth the extra delay. However, it can be helpful for chefs to plan ahead so they have all the necessary ingredients on hand before they start cooking.

[0052] Just like the pantry analogy above, the memory access latency associated with accessing various levels in the memory hierarchy is different. For example, a particular core 122 can access data in its L1 cache 128 with less latency than data in its L2 cache 126. Accessing graphics memory 114 or system memory 108 can incur significantly more clock cycles than accessing the L2 cache.

[0053] The texture processing unit 120 is a specialized processing unit for processing texture information on a scene to be rendered into an image to be displayed. For example, when performing graphics processing (e.g., a graphics processing pipeline) related to rendering an image to a display, the core 122 may rely on the texture processing unit 120 to perform texture mapping for the image. Each GPU 102 may include one or more texture processing units 120. The texture processing unit 120 may not be limited to texture mapping and may also perform processing tasks other than those related to texture information.

[0054] Texture information includes metrics that define the texture of an image and individual objects in the image. Texture information may include information about the spatial arrangement of colors or intensities in various areas of the image. Texture data may be computed through image processing. Typically, texture information associated with an image to be displayed on a display screen is too large to be stored in on-chip memory on a GPU and is therefore typically stored in external memory (e.g., off-chip memory), such as system memory 108 or graphics memory 114 (e.g., texture data 130 in system memory 108 or texture data 132 in graphics memory 114).

[0055] As described above, retrieving texture information from system memory typically results in long delays, leading to system inefficiencies. In some cases, such delays can be on the order of thousands of clock cycles. In an exemplary embodiment, texture unit 120, processing core 122, and / or cache memory hierarchy 124 may be configured with additional circuitry to perform prefetching of texture data in order to reduce the delay associated with texture data access. Figure 2A etc. provide a description of the configuration of GPU 102 to provide prefetching according to an exemplary embodiment. Figure 3 and the associated description provide more detailed information regarding exemplary parallel processing units (PPUs) that may correspond to GPU 102 .

[0056] Figure 1B An example is shown in which the entire display screen area 140 is logically divided into rectangular tiles so that texture information to be rendered on the screen 140 can be distributed among multiple texture units 120. The example shown shows texture information 144 occupying an area of ​​the screen 140. Figure 1B 142 to the screen, one or more processing cores 122 of the GPU 102 may allocate a texture map for the object 142 in the manner shown. That is, the texture map for the object 142 may be allocated to each of the four (T1-T4) texture processing units 120 in the manner shown. For example, Figure 1B As shown, each texture processing unit 120 applies a texture 144 to a corresponding portion of one or more surfaces of an object 142 .

[0057] The distribution of texture information 144 among texture processing units T1-T4 can be controlled by a core processing unit (e.g., core processing unit 120) in the GPU. In some exemplary embodiments, instructions from a core processing unit to a texture processing unit may include information describing a texture 144, an object 142 to be texture mapped, and a portion of the object 142 to be texture mapped by the texture processing unit. The texture information 144 provided by the core processing unit to the texture processing unit may include a header for the texture information 144 that describes the size and coordinates of the texture, the size of the block in which the texture information is stored in memory, and the like. Based on the information in the texture header, the object information, and information related to the allocated portion of the object, each texture processing unit can determine the texture information to retrieve in order to perform texture mapping on the portion of the object assigned to it.

[0058] like Figure 1BAs can be seen in Figure 1, during texture mapping of a particular scene, multiple portions of the same object may be assigned to each texture processing unit T1-T4, along with corresponding portions of the texture. The assignment may be determined by the tiling pattern of the texture processing units across the screen area, as well as the position, size, and shape of the texture and the object or scene being texture mapped.

[0059] Figure 1B The allocation pattern of texture processors shown in FIG is an example and not limiting. In an exemplary embodiment, texture information can be allocated between any number of two or more texture processing units according to any allocation pattern. The exemplary embodiment is not limited to any particular tiling pattern of texture processing units on the screen area or to any particular texture or scene / object characteristics.

[0060] Figure 1C An exemplary, non-limiting texture 154 is shown that is organized in memory according to a pitch-linear format. For example, a 16x16 texel region is shown, where the numbers indicated within the texture represent the corresponding virtual memory addresses (or offsets from virtual memory addresses) where the texels are stored. The size (in bytes) of the texels can vary from texture to texture, depending on the level of detail, such as the resolution of the texture. The size (in bytes) of the texels in a particular texture can be indicated in the header of that texture.

[0061] According to the pitch linear format for storing texture information in memory, each row of texels from the top to the bottom of the texture is laid out in a manner that starts at the upper left corner of the memory area and then extends horizontally from left to right before starting to store the next row of texels (e.g., from left to right, top to bottom). Figure 1C The texels 0-255 of the texture 154 are shown arranged in a pitched linear manner in memory. That is, each row of texels of the texture 154 (as it can be mapped into a rendered image on a display screen) is stored in a row in virtual memory. The memory storing the texture 154 can be, for example, external memory such as system memory 108 or graphics memory 114.

[0062] Figure 1D A texture 156 is shown organized in memory according to a block linear format. Figure 1C Same as in Figure 1D An example 16x16 pixel area screen layout is also shown, where the numbers inside texture 156 represent the individual virtual memory addresses (or offsets relative to virtual memory addresses) specified in the texture pixels. The block linear format is optimized for many 2D and 3D access patterns. Figure 1DAs shown, the texels 0-15 and 16-31 corresponding to the first and second rows of the texture 156 arrangement in memory are 4x4 rectangular areas of texels ("blocks") on the screen, respectively. In the example block linear layout, each row of texels in memory corresponds to a 4x4 block, and every four consecutive rows of texels in memory correspond to an 8x8 block. Figure 1C and Figure 1D In the memory layout, each 16-texel row is represented by a distinguishable fill pattern.

[0063] Figure 1E The texture requests generated by full screen drawing are shown. The example pattern shown shows the virtual addresses of memory blocks that hold texture information over time. The x-axis represents time increasing from left to right, and the y-axis represents virtual address increasing from bottom to top. Each shaded area represents a chunk of memory being accessed, and the position of the shaded area in the graph represents its time position and its virtual address relative to the start of accessing texture data for the full screen. Each chunk consists of one or more consecutive blocks. Overall, Figure 1E Shows accessing texture data for drawing full screen.

[0064] Chunks 152 represent the virtual address space currently being used by the texture unit processing the screen's textures. As shown, the offset associated with each chunk 152 is the height 150 of each chunk (e.g., in texels). That is, the height of each chunk (e.g., in texels) can be used to represent the offset between consecutively requested chunks. In some exemplary embodiments, a chunk may have a height of one block.

[0065] Figure 1F Another example illustration of texture data organized in memory and its relationship to textures rendered on screen is provided according to some example embodiments. The figure shows how each row of texture information 160 in memory is mapped to a group of blocks 158 according to a block linear layout.

[0066] Figure 1G As further illustrated graphically, in some exemplary embodiments, textured surface 160 includes a plurality of blocks, such as block 158, and each block includes a plurality of blocks, such as in Figure 1D The blocks of the texture shown in the linear layout are shown in FIG. In the example shown, each row of texture information 160 corresponds to a respective group of blocks.

[0067] Figure 2A According to some exemplary embodiments, Figure 1A For example, GPU 200 may correspond to Figure 1A GPU 102 is shown.

[0068] GPU 200 includes processing cores 201 (processing core 201a and processing core 201b), texture processing units 202 (texture unit A 202a and texture unit B 202b), a cache hierarchy 204, and an interface 210 to graphics memory (e.g., graphics memory 114) and / or external memory (such as system memory, e.g., system memory 108).

[0069] In cache hierarchy 204, each L1 cache 206 is accessible to a processing core and a texture unit. For example, processing core 201a and texture unit A 202a can access L1 cache 206a, and processing core 201b and texture unit B 202b can access L1 cache 206b. Processor core 201 or texture unit 202 can access L2 cache 208 to retrieve data not found in L1 cache 206. If the data is not found in L2 cache 208, external memory (e.g., off-chip memory, such as system memory or graphics memory) can be accessed through interface 210 to retrieve the data.

[0070] An exemplary embodiment enables each texture unit to prefetch all or part of the data from the second texture information block while processing the first texture information block (or after it obtains the first texture information block and before it completes processing the first texture information block). Processing the texture information block may include using the texture information block to texture map an object and / or scene. The prefetch logic 214 (e.g., prefetch logic 214a and 214b) block includes hardware circuitry for calculating the address of the texture information block to be prefetched. The address of the block to be prefetched is calculated based on the address and offset of the block currently being processed. The texture prefetch logic 214 may also implement request deduplication to reduce the number of duplicate prefetch requests transmitted to the L2 cache.

[0071] According to an exemplary embodiment, texture information 215 pre-fetched from external memory is stored in L2 cache 208 rather than being provided to the requesting texture processing unit and / or processing core. The pre-fetched texture information 215 in the L2 cache is available to multiple texture processing units and / or multiple processor cores connected to the L2 cache. Texture pre-fetch logic 216 hardware circuitry configured to store pre-fetched information may be located in control circuitry associated with L2 cache 208.

[0072] Also associated with the texture prefetch logic 216 circuit is a pending prefetch request counter 218. Pending prefetch counter 218 maintains a count of the number of outstanding texture prefetch requests 212. That is, according to one embodiment, in the L2 cache, the counter is incremented when a texture prefetch request is sent from the L2 cache circuit to external memory, and the counter is decremented when texture information corresponding to the request is received and stored in the L2 cache. In conjunction with pending prefetch request counter 218, the texture prefetch logic can implement a throttling mechanism to silently discard incoming prefetch requests when the count of pending prefetch requests exceeds a predefined threshold. The texture prefetch logic 216 can also implement request deduplication to reduce the number of duplicate prefetch requests sent to external memory. In at least some embodiments, deduplication and / or throttling may be necessary to efficiently perform texture prefetching without overwhelming the memory system and interconnect systems with redundant prefetch requests.

[0073] According to some embodiments, GPU 200 may correspond to, for example, Figure 3 Shown in and about Figures 3 to 6 The core processor 201 may correspond to the streaming multiprocessor (SM) 440, and the texture processing unit 202 may correspond to the special function unit (SFU) 552. In such an embodiment, the texture prefetching described with respect to processes 220 and 240 may be performed in, for example, a manner similar to that described with respect to Figure 6 Describes the graphics pipeline during processing.

[0074] Exemplary Texture Prefetching Methods

[0075] Figure 2B FIG2 shows a flow chart of a process 220 by which a processor requests texture information for processing according to certain exemplary embodiments. The process 220 may be performed by a core processing unit and / or a texture processing unit. In some exemplary embodiments, the process 220 is performed by Figure 2A Prefetch logic 214 is shown performing (eg, in prefetch logic circuits 214a and / or 214b). For clarity, process 220 is described below with respect to texture processing unit 202a and core processing unit 201a, but process 220 is not limited to these processing units.

[0076] After entering process 220, while processing texture information block n, at operation 222, texture processing unit 202a determines texture information block n+k to prefetch data from. k is an offset size (also called a prefetch offset) expressed as a multiple of the block size. Each block n and n+k can be a block of a larger texture information A (e.g., a block consisting of Figure 1D156). According to an exemplary embodiment, texture processing unit 202a begins processing block n in response to being assigned processing of block n by a core processing unit (e.g., core processing unit 201a or another core processing unit 201b). For example, core processing unit 201a may assign instructions to texture processing unit 202a to process block n. Texture processing unit 202a may receive texture information for block n from the memory hierarchy in response to its own request or a request made by a core processing unit.

[0077] During processing of block n, the texture processing unit determines block n+k to prefetch data from. In some embodiments, each texture processing unit is configured to prefetch data from the immediately next block to the block that the texture processing unit is currently processing. However, in some embodiments, blocks other than the immediately next block may be used for prefetching.

[0078] According to an exemplary embodiment, determining the data to be prefetched may require determining the offset (in bytes) of the address of the block. In an exemplary embodiment employing a block linear address mapping layout, the offset is the size of the chunk (in bytes). The size of the chunk is a multiple of the size of the block. In such an embodiment, the block and chunk structure is a direct result of the block linear address mapping, which is used to store many textures used by modern games. In such an embodiment, the address of the next chunk can be determined based on the address and offset of the current chunk, and similarly for the next chunk.

[0079] The size of the blocks and chunks may not be the same for all textures and may be an intrinsic property of the texture. Therefore, the offset may need to be calculated dynamically on a per-texture basis. Additionally, if multiple textures are used simultaneously, a separate offset may be maintained for each active texture.

[0080] Each texture may include a header ("texture header") that stores certain kinds of information about the texture, such as, for example, its height, width, level of detail, data type, etc. In some exemplary embodiments, the texture header (and / or parameter values ​​from the header) are already available in the texture unit for all textures that are active at any given point in time. By accessing the parameters describing a particular texture in the corresponding header, some exemplary embodiments calculate the size of a chunk for a particular block linear texture as:

[0081] Size chunk =Width texture *Height chunk *bytes_per_texel

[0082] Offset-Size or Prefetch-Offset=Scaling Factor*Size chunk

[0083] Thus, the offset size can be calculated economically at runtime using information available in the texture processing unit and / or processing core. For example, the offset can be calculated as an offset size (Offset-Size) in bytes. A scaling factor F can be used to fine-tune the prefetch offset (prefetch-offset) by multiplying it by the chunk size. For example, smaller GPUs may benefit from an F value less than 1. The default value of the scaling factor can be 1.

[0084] After calculating the offset, the address of the block or portion of the block to be prefetched is determined based on the block currently being processed in the texture processing unit and the calculated offset. For example, the address of the block or portion of the next block to be prefetched can be determined by adding the offset to the address of the data from the current block. When the texture unit working on the current block (belonging to the current group of blocks) has been shown to improve the performance of many workloads / applications (such as gaming applications), prefetching the address of the block or portion of the block belonging to the next group of blocks (i.e., the address of the block or portion of the block belonging to the next group of blocks) into the L2 cache.

[0085] At operation 224, a check may optionally be performed to determine if the intended prefetch request is a duplicate. This optional check may be performed economically using information available in the texture processing unit. For example, a register that tracks the identities of a predetermined number of immediately preceding prefetches requested by the texture processing unit may be tested against the calculated address of the next block or portion of a block to be prefetched. If the next block or portion of a block is determined to match any of the most recent prefetch requests (e.g., the intended prefetch request is determined to be a duplicate), the next block or portion of a block currently determined for prefetch is discarded 226, and operation 222 is repeated to determine another block or portion of a block to prefetch (i.e., to determine another request) that is not a duplicate of a block or portion of a block that has already been prefetched.

[0086] When it is determined at optional operation 224 that the next block or portion of the block determined for prefetching is not a duplicate, process 220 proceeds to operation 228. For example, when the prefetch request for all or part of block n+k is determined to be non-duplicate, then at operation 228, a texture prefetch request is generated for data from block n+k. The prefetch request may include a prefetch indication to distinguish it from a texture get request. According to an exemplary embodiment, a prefetch request (e.g., prefetch request 212) includes a header having one or more bits identifying it as a prefetch request and including the address or offset of the requested block of texture information. In some embodiments, the prefetch request header may also include a unique identifier or address of the texture (e.g., the address of the first block in the texture).

[0087] The generated prefetch request is sent to the memory hierarchy. This sending can be performed in a similar manner to sending a get request from the texture unit to the memory hierarchy.

[0088] The processing of generated prefetch requests for all or part of block n+k in the memory hierarchy is described below with respect to process 240 .

[0089] After completing texture processing (e.g., texture mapping) using block n, process 220 proceeds to operation 230. At operation 230, texture processing unit 202a receives all or part of the next texture block to be processed from texture A (e.g., another block from surface texture 156, for example). This next texture block may be obtained by one or more memory accesses made in response to one or more instructions received from the core processing unit. This next block may not be the block that immediately follows (depending on the memory access pattern) the block that texture processing unit 202a just completed processing, and may be any other block m in the same texture A. Texture processing unit 202a may have already obtained block m from L2 cache 208, where it may be stored after being prefetched by another texture processing unit.

[0090] In some exemplary embodiments, at operation 222, before generating a prefetch request, a determination is made as to whether the texture mapping is part of a particular type of rendering. In such embodiments, the prefetch request may be generated in response to the determination. For example, if it is determined that the texture mapping is part of a full-screen rendering operation, a prefetch request is generated and prefetch processes 220 and 240 are completed. If it is determined that the texture mapping is not part of a full-screen rendering operation, no prefetch is performed. This determination may be based on the instruction type. This determination may not be limited to the determination described above for full-screen rendering.

[0091] Figure 2C shows a response to a Figure 2BA flowchart of a process 240 for performing a processor-initiated prefetch by the memory hierarchy is shown. The process 240 may be performed in whole or in part in the texture prefetch logic 216 associated with the L2 cache 208 and / or the cache hierarchy 204.

[0092] At operation 242, process 240 may begin for each prefetch request received from a texture processing unit. The generation and sending of prefetch requests from a texture processing unit are described above with respect to Figure 2B Described.

[0093] At operation 244, it is optionally determined whether the received prefetch request is a duplicate of a prefetch request that has already been received and / or serviced. To avoid sending multiple prefetch requests to the same target address, some embodiments include a prefetch deduplication scheme in which a second prefetch request is not sent if a prefetch request has already been sent due to an earlier read of the same cache line (e.g., for multiple read requests to the same 128B / 256B aligned address, only one prefetch request is sent to an address that is a byte offset away from the 128B / 256B aligned read address). This deduplication can save bandwidth (e.g., request bandwidth and / or data bandwidth).

[0094] If the prefetch request is determined to be a duplicate, then at operation 246 , the received prefetch request is discarded. If it is determined that the request is not a duplicate request, then process 240 proceeds to operation 248 .

[0095] At operations 248-250, it is optionally determined whether the received prefetch request increases the number of pending prefetch requests above a predetermined threshold. That is, after receiving a prefetch request and optionally determining that it is not a duplicate, at operation 248, a counter that tracks the number of currently pending prefetch requests is incremented. Then, at operation 250, it is determined whether the incremented count of pending prefetch requests exceeds a predetermined threshold. If the number of pending requests exceeds the threshold, then at operation 246, the request is discarded.

[0096] The throttling techniques of operations 248-250 provide a solution to scenarios where the memory system may already be performance-limiting due to excessive access requests. In such situations, it is desirable not to further burden the memory system with speculative prefetch requests. Therefore, the throttling scheme utilizes available information about cache sectors with outstanding fill requests to dynamically discard prefetch requests if the number of such outstanding sectors exceeds a certain threshold.

[0097] If the number of pending requests does not exceed the threshold, process 240 proceeds to operation 252 .

[0098] At operation 252, a prefetch request is sent to an external (e.g., off-chip) memory. In some embodiments, access requests for fetch requests and prefetch requests sent from the L2 cache to the external memory may be indistinguishable from one another. In such embodiments, the L2 cache circuitry may track requested and received texture blocks so that received blocks may be appropriately processed at the L2 cache based on whether each block was received for a fetch request or a prefetch request. In some other embodiments, access requests sent from the L2 cache for prefetch requests may be distinguished from access requests sent for fetch requests. In some embodiments, memory blocks received at the L2 cache from the external memory may be accompanied by header information indicating whether they are related to prefetch requests.

[0099] At operation 254, the requested data is received at the L2 cache from the prefetch block. The L2 cache control circuitry may distinguish between requests for texture blocks associated with prefetches and those associated with fetch requests in any manner. Some exemplary techniques for performing this distinction are described with respect to operation 252.

[0100] At operation 256, in view of receiving the data for the requested prefetch block (or portion thereof), the pending request counter is decremented.

[0101] At operation 258, the partially or completely prefetched block is stored in the L2 cache. According to some embodiments, unlike the case where a get request is used, the prefetched block data is not provided to the L1 cache or the requesting texture processing unit. The prefetched block data is stored only in the L2 cache. The processor or other processor requesting the prefetched block data can then retrieve the prefetched block data stored in the L2 cache and store it in the L1 cache through a get request. Although in some embodiments, when the prefetch is completed, the prefetched data may also be stored in the L1 cache and / or the texture processing unit that issued the prefetch request in addition to the L2 cache, such embodiments may be less efficient than embodiments that do not store the prefetched data in the L1 cache or the texture processing unit.

[0102] After operation 258, the prefetch operation is complete.

[0103] Parallel processing architecture for video noise reduction

[0104] Now, more illustrative information about various optional architectures and features that can be used to implement the aforementioned framework will be described, depending on the user's needs. It should be noted that the following information is provided for illustrative purposes and should not be construed as limiting in any way. Any of the following features may optionally be combined with, or not exclusively combine with, the other features described.

[0105] Figure 3A parallel processing unit (PPU) 300 is shown according to one embodiment. In one embodiment, the PPU 300 is a multi-threaded processor implemented on one or more integrated circuit devices. The PPU 300 is a latency-hiding architecture designed for processing many threads in parallel. A thread (i.e., an execution thread) is an instance of an instruction set configured to be executed by the PPU 300. In one embodiment, the PPU 300 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device (such as a liquid crystal display (LCD) device). In other embodiments, the PPU 300 can be used to perform general-purpose computations. Although an exemplary parallel processor is provided herein for illustrative purposes, it should be specifically noted that the processor is described for illustrative purposes only and any processor may be used in addition to and / or in place of the processor.

[0106] One or more PPUs 300 can be configured to accelerate thousands of high-performance computing (HPC), data center, and machine learning applications. PPUs 300 can be configured to accelerate numerous deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analysis, molecular simulations, drug discovery, disease diagnosis, weather forecasting, big data analytics, astronomy, molecular dynamics simulations, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations, among others.

[0107] like Figure 3 As shown, the PPU 300 includes an input / output (I / O) unit 305, a front-end unit 315, a scheduler unit 320, a work distribution unit 325, a hub 330, a crossbar switch (Xbar) 370, one or more general processing clusters (GPCs) 350, and one or more partition units 380. The PPU 300 can be connected to a host processor or other PPUs 300 via one or more high-speed NVLink 310 interconnects. The PPU 300 can be connected to a host processor or other peripheral devices via interconnect 302. The PPU 300 can also be connected to a local memory including multiple memory devices 304. In one embodiment, the local memory can include multiple dynamic random access memory (DRAM) devices. The DRAM devices can be configured as a high-bandwidth memory (HBM) subsystem, in which multiple DRAM dies are stacked within each device.

[0108] The NVLink 310 interconnect enables the system to scale and include one or more PPUs 300 in conjunction with one or more CPUs, supporting cache coherency between the PPU 300 and the CPU, and CPU mastering. Data and / or commands can be transferred by the NVLink 310 through the hub 330 to or from other units of the PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). Figure 5B NVLink 310 is described in more detail.

[0109] I / O unit 305 is configured to transmit and receive communications (e.g., commands, data, etc.) from a host processor (not shown) via interconnect 302. I / O unit 305 can communicate with the host processor directly via interconnect 302, or through one or more intermediary devices (such as a memory bridge). In one embodiment, I / O unit 305 can communicate with one or more other processors (e.g., one or more PPUs 300) via interconnect 302. In one embodiment, I / O unit 305 implements a Peripheral Component Interconnect Express (PCIe) interface for communicating over a PCIe bus, and interconnect 302 is a PCIe bus. In alternative embodiments, I / O unit 305 can implement other types of known interfaces for communicating with external devices.

[0110] I / O unit 305 decodes data packets received via interconnect 302. In one embodiment, the data packets represent commands configured to cause PPU 300 to perform various operations. I / O unit 305 transmits the decoded commands to various other units of PPU 300 as specified by the commands. For example, some commands may be transmitted to front-end unit 315. Other commands may be transmitted to hub 330 or other units of PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, I / O unit 305 is configured to route communications between and among the various logical units of PPU 300.

[0111] In one embodiment, a program executed by a host processor encodes a command stream in a buffer that provides a workload to the PPU 300 for processing. The workload may include many instructions and data to be processed by those instructions. A buffer is an area of ​​memory that is accessible (e.g., read / write) by both the host processor and the PPU 300. For example, the I / O unit 305 may be configured to access a buffer in system memory connected to the interconnect 302 via a memory request transmitted over the interconnect 302. In one embodiment, the host processor writes a command stream into the buffer and then transmits a pointer to the start of the command stream to the PPU 300. The front end unit 315 receives pointers to one or more command streams. The front end unit 315 manages the one or more streams, reads commands from the streams, and forwards the commands to the various units of the PPU 300.

[0112] Front-end unit 315 is coupled to scheduler unit 320, which configures various GPCs 350 to process tasks defined by one or more streams. Scheduler unit 320 is configured to track state information related to the various tasks managed by scheduler unit 320. The state may indicate which GPC 350 a task is assigned to, whether the task is active or inactive, the priority associated with the task, and the like. Scheduler unit 320 manages the execution of multiple tasks on one or more GPCs 350.

[0113] Scheduler unit 320 is coupled to work distribution unit 325, which is configured to dispatch tasks for execution on GPCs 350. Work distribution unit 325 can track a number of scheduled tasks received from scheduler unit 320. In one embodiment, work distribution unit 325 manages a pending task pool and an active task pool for each GPC 350. The pending task pool can include a number of time slots (e.g., 32 time slots) containing tasks assigned to be processed by a particular GPC 350. The active task pool can include a number of time slots (e.g., 4 time slots) for tasks being actively processed by GPC 350. When a GPC 350 completes execution of a task, the task is evicted from the active task pool of GPC 350, and one of the other tasks from the pending task pool is selected and scheduled for execution on GPC 350. If an active task on GPC 350 has become idle, such as while waiting for a data dependency to be resolved, the active task may be evicted from GPC 350 and returned to the pending task pool, while another task in the pending task pool is selected and scheduled for execution on GPC 350 .

[0114] Work distribution unit 325 communicates with one or more GPCs 350 via XBar (crossbar) 370. XBar 370 is an interconnect network that couples many units of PPU 300 to other units of PPU 300. For example, XBar 370 can be configured to couple work distribution unit 325 to a specific GPC 350. Although not explicitly shown, one or more other units of PPU 300 can also be connected to XBar 370 via hub 330.

[0115] Tasks are managed by the scheduler unit 320 and dispatched to the GPCs 350 by the work distribution unit 325. The GPCs 350 are configured to process tasks and generate results. The results can be consumed by other tasks within the GPC 350, routed to different GPCs 350 via the XBar 370, or stored in the memory 304. The results can be written to the memory 304 via the partition unit 380, which implements a memory interface for reading data from and writing data to the memory 304. The results can be transferred to another PPU 304 or CPU via NVLink 310. In one embodiment, the PPU 300 includes a number U of partition units 380, which is equal to the number of independent and different memory devices 304 coupled to the PPU 300. Figure 4B The partition unit 380 is described in more detail.

[0116] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications to be executed on the host processor to schedule operations for execution on the PPU 300. In one embodiment, multiple computing applications are executed simultaneously by the PPU 300, and the PPU 300 provides isolation, quality of service (QoS), and independent address spaces for the multiple computing applications. The application can generate instructions (e.g., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 300. The driver kernel outputs the tasks to one or more streams being processed by the PPU 300. Each task can include one or more related groups of threads, referred to herein as warps. In one embodiment, a warp includes 32 related threads that can execute in parallel. Collaborating threads can refer to multiple threads that include instructions to perform tasks and that can exchange data through shared memory. In combination Figure 5A Describes threads and cooperative threads in more detail.

[0117] Figure 4A According to one embodiment, Figure 3 PPU 300 GPC 350. Figure 4AAs shown, each GPC 350 includes multiple hardware units for processing tasks. In one embodiment, each GPC 350 includes a pipeline manager 410, a pre-raster operation unit (PROP) 415, a raster engine 425, a work distribution crossbar (WDX) 480, a memory management unit (MMU) 490, and one or more data processing clusters (DPCs) 420. It should be understood that Figure 4A The GPC 350 may include instead Figure 4A Other hardware units or other than the units shown in Figure 4A Other hardware units besides those shown in .

[0118] In one embodiment, the operation of GPC 350 is controlled by pipeline manager 410. Pipeline manager 410 manages the configuration of one or more DPCs 420 for processing tasks assigned to GPC 350. In one embodiment, pipeline manager 410 can configure at least one of one or more DPCs 420 to implement at least a portion of a graphics rendering pipeline. For example, DPC 420 can be configured to execute vertex shading programs on programmable streaming multiprocessors (SMs) 440. Pipeline manager 410 can also be configured to route packets received from work distribution unit 325 to appropriate logic units within GPC 350. For example, some packets can be routed to fixed-function hardware units within PROP 415 and / or raster engine 425, while other packets can be routed to DPC 420 for processing by primitive engine 435 or SM 440. In one embodiment, pipeline manager 410 can configure at least one of one or more DPCs 420 to implement a neural network model and / or a compute pipeline.

[0119] PROP unit 415 is configured to route data generated by raster engine 425 and DPC 420 to the raster operations (ROP) unit, in conjunction with Figure 4B The PROP unit 415 may also be configured to perform optimizations for color blending, organize pixel data, perform address translation, and the like.

[0120] The raster engine 425 includes several fixed-function hardware units configured to perform various raster operations. In one embodiment, the raster engine 425 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a tile aggregation engine. The setup engine receives the transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices. The plane equations are transmitted to the coarse raster engine to generate coverage information for the primitives (e.g., the x, y coverage mask of the tile). The output of the coarse raster engine is transmitted to the culling engine, where fragments associated with primitives that fail the z-test are culled, and unculled fragments are transmitted to the clipping engine, where fragments outside the viewing frustum are clipped. Those fragments remaining after clipping and culling can be passed to the fine raster engine to generate attributes for the pixel fragments based on the plane equations generated by the setup engine. The output of the raster engine 425 includes, for example, fragments to be processed by the fragment shader implemented in the DPC 420.

[0121] Each DPC 420 included in the GPC 350 includes an M pipeline controller (MPC) 430, a primitive engine 435, and one or more SMs 440. The MPC 430 controls the operation of the DPC 420 and routes data packets received from the pipeline manager 410 to appropriate units in the DPC 420. For example, data packets associated with vertices may be routed to the primitive engine 435, which is configured to fetch vertex attributes associated with the vertices from the memory 304. Conversely, data packets associated with shading programs may be transmitted to the SM 440.

[0122] SM 440 includes a programmable streaming processor configured to process tasks represented by multiple threads. Each SM 440 is multi-threaded and configured to simultaneously execute multiple threads (e.g., 32 threads) from a particular thread group. In one embodiment, SM 440 implements a SIMD (single instruction, multiple data) architecture, in which each thread in a thread group (e.g., a warp) is configured to process a different data set based on the same instruction set. All threads in a thread group execute the same instructions. In another embodiment, SM 440 implements a SIMT (single instruction, multiple thread) architecture, in which each thread in a thread group is configured to process a different data set based on the same instruction set, but in which individual threads in a thread group are allowed to diverge during execution. In one embodiment, a program counter, call stack, and execution state are maintained for each warp, enabling concurrency between warps and serial execution within warps when threads within a warp diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, thereby achieving equal concurrency between all threads within and between warps. When execution state is maintained for each individual thread, threads executing the same instruction can be converged and executed in parallel for maximum efficiency. Figure 5A Describe SM440 in more detail.

[0123] MMU 490 provides an interface between GPC 350 and partition unit 380. MMU 490 can provide virtual to physical address translation, memory protection, and arbitration of memory requests. In one embodiment, MMU 490 provides one or more translation lookaside buffers (TLBs) for performing translations from virtual addresses to physical addresses in memory 304.

[0124] Figure 4B According to one embodiment, Figure 3 The memory partition unit 380 of the PPU 300. Figure 4B As shown, the memory partition unit 380 includes a raster operations (ROP) unit 450, a level 2 (L2) cache 460, and a memory interface 470. The memory interface 470 is coupled to the memory 304. The memory interface 470 can implement a 32-, 64-, 128-, or 1024-bit data bus for high-speed data transfer. In one embodiment, the PPU 300 incorporates U memory interfaces 470, one for each pair of partition units 380, where each pair of partition units 380 is connected to a corresponding memory device 304. For example, the PPU 300 can be connected to up to Y memory devices 304, such as a high-bandwidth memory stack or graphics double data rate version 5 synchronous dynamic random access memory or other types of persistent memory.

[0125] In one embodiment, memory interface 470 implements an HBM2 memory interface, and Y is equal to half of U. In one embodiment, the HBM2 memory stack is located on the same physical package as PPU 300, providing significant power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack includes four memory dies and Y is equal to 4, where the HBM2 stack includes two 128-bit channels per die, for a total of 8 channels and a data bus width of 1024 bits.

[0126] In one embodiment, memory 304 supports single-error correction, double-error detection (SECDED) error correction code (ECC) to protect data. ECC provides increased reliability for computing applications sensitive to data corruption. Reliability is particularly important in large cluster computing environments where PPU 300 processes very large data sets and / or long-running applications.

[0127] In one embodiment, the PPU 300 implements a multi-level memory hierarchy. In one embodiment, the memory partitioning unit 380 supports unified memory to provide a single, unified virtual address space for the CPU and PPU 300 memory, enabling data sharing between virtual memory systems. In one embodiment, the frequency of PPU 300 accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 300 where the pages are accessed more frequently. In one embodiment, NVLink 310 supports address translation services that allow the PPU 300 to directly access the CPU's page tables and provide full access to the CPU's memory by the PPU 300.

[0128] In one embodiment, the copy engine transfers data between multiple PPUs 300 or between a PPU 300 and a CPU. The copy engine can generate a page fault for an address that is not mapped to a page table. The memory partition unit 380 can then service the page fault, map the address into a page table, and then the copy engine can perform the transfer. In conventional systems, multiple copy engines operate on fixed memory (e.g., non-pageable) between multiple processors, which significantly reduces the available memory. Due to hardware page faults, addresses can be passed to the copy engine without worrying about whether the memory page is resident, and the copy process is transparent.

[0129] Data from memory 304 or other system memory can be retrieved by memory partition unit 380 and stored in L2 cache 460, which is located on-chip and shared between various GPCs 350. As shown, each memory partition unit 380 includes a portion of L2 cache 460 associated with the corresponding memory device 304. Lower-level caches can then be implemented in multiple units within a GPC 350. For example, each SM 440 can implement a level 1 (L1) cache. The L1 cache is a dedicated memory dedicated to a particular SM 440. Data from the L2 cache 460 can be retrieved and stored in each L1 cache for processing in the functional units of the SM 440. The L2 cache 460 is coupled to the memory interface 470 and the XBar 370.

[0130] The ROP unit 450 performs graphics raster operations related to pixel color such as color compression, pixel blending, etc. The ROP unit 450 also implements depth testing in conjunction with the raster engine 425, receiving the depth of the sample position associated with the pixel fragment from the culling engine of the raster engine 425. The depth of the sample position associated with the fragment is tested against the corresponding depth in the depth buffer. If the fragment passes the depth test for the sample position, the ROP unit 450 updates the depth buffer and transmits the result of the depth test to the raster engine 425. It will be understood that the number of partition units 380 can be different than the number of GPCs 350, and therefore each ROP unit 450 can be coupled to each GPC 350. The ROP unit 450 tracks packets received from different GPCs 350 and determines to which GPC 350 the results generated by the ROP unit 450 are routed via the Xbar 370. Although in Figure 4B In some embodiments, ROP unit 450 is included within memory partition unit 380, but in other embodiments, ROP unit 450 may be external to memory partition unit 380. For example, ROP unit 450 may reside in GPC 350 or another unit.

[0131] Figure 5A According to one embodiment, Figure 4A Streaming multiprocessor 440. Figure 5A As shown, SM 440 includes an instruction cache 505, one or more scheduler units 510, a register file 520, one or more processing cores 550, one or more special function units (SFUs) 552, one or more load / store units (LSUs) 554, an interconnect network 580, and a shared memory / L1 cache 570.

[0132] As described above, the work distribution unit 325 schedules tasks for execution on the GPCs 350 of the PPU 300. Tasks are assigned to specific DPCs 420 within the GPC 350 and, if the task is associated with a shader program, may be assigned to an SM 440. The scheduler unit 510 receives tasks from the work distribution unit 325 and manages the scheduling of instructions for one or more thread blocks assigned to the SM 440. The scheduler unit 510 schedules thread blocks for execution as warps of parallel threads, where each thread block is assigned at least one warp. In one embodiment, each warp executes 32 threads. The scheduler unit 510 can manage multiple different thread blocks, assign warps to different thread blocks, and then dispatch instructions from multiple different cooperative groups to various functional units (i.e., cores 550, SFUs 552, and LSUs 554) during each clock cycle.

[0133] Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, enabling the expression of richer and more efficient decompositions of parallelism. The cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. Conventional programming models provide a single, simple construct for synchronizing cooperating threads: a barrier across all threads of a thread block (e.g., the syncthreads() function). However, programmers often want to define thread groups at a granularity smaller than the thread block granularity and synchronize within the defined group, enabling higher performance, design flexibility, and software reuse in the form of a collective group-wide function interface.

[0134] Cooperative Groups enable programmers to explicitly define thread groups at sub-block (e.g., as small as a single thread) and multi-block granularity and perform collective operations, such as synchronization, on threads in a cooperative group. The programming model supports clean composition across software boundaries so that libraries and utility functions can safely synchronize in their local environment without making assumptions about convergence. Cooperative Group primitives enable new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire grid of thread blocks.

[0135] The dispatch unit 515 is configured to deliver instructions to one or more functional units. In this embodiment, the scheduler unit 510 includes two dispatch units 515, which enable scheduling of two different instructions from the same warp during each clock cycle. In alternative embodiments, each scheduler unit 510 may include a single dispatch unit 515 or additional dispatch units 515.

[0136] Each SM 440 includes a register file 520 that provides a set of registers for the functional units of the SM 440. In one embodiment, the register file 520 is divided between each functional unit so that each functional unit is allocated a dedicated portion of the register file 520. In another embodiment, the register file 520 is divided between the different warps executed by the SM 440. The register file 520 provides temporary storage for operands connected to the data paths of the functional units.

[0137] Each SM 440 includes L processing cores 550. In one embodiment, the SM 440 includes a large number (e.g., 128, etc.) of different processing cores 550. Each core 550 may include a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, including a floating-point arithmetic logic unit (FLU) and an integer arithmetic logic unit (ALU). In one embodiment, the FLU implements the IEEE 754-2008 standard for floating-point operations. In one embodiment, the core 550 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0138] Tensor cores are configured to perform matrix operations, and in one embodiment, one or more tensor cores are included in core 550. Specifically, the tensor cores are configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In one embodiment, each tensor core operates on a 4×4 matrix and performs a matrix multiplication and accumulation operation D=A×B+C, where A, B, C, and D are 4×4 matrices.

[0139] In one embodiment, the matrix multiplication inputs A and B are 16-bit floating point matrices, while the accumulation matrices C and D can be 16-bit floating point or 32-bit floating point matrices. The tensor cores operate on 16-bit floating point input data as well as 32-bit floating point accumulations. The 16-bit floating point multiplication requires 64 operations to produce a full-precision product, which is then accumulated using 32-bit floating point additions with other intermediate products of the 4×4×4 matrix multiplication. In practice, tensor cores are used to perform larger two-dimensional or higher-dimensional matrix operations built from these smaller elements. APIs (such as the CUDA9 C++ API) expose specialized matrix load, matrix multiplication and accumulation, and matrix store operations to efficiently use tensor cores from CUDA-C++ programs. At the CUDA level, the warp-level interface assumes that the 16×16 size matrix spans all 32 threads of the warp.

[0140] Each SM 440 also includes M SFUs 552 that perform special functions (e.g., attribute evaluation, reciprocal square root, etc.). In one embodiment, the SFUs 552 may include a tree traversal unit configured to traverse a hierarchical tree data structure. In one embodiment, the SFUs 552 may include a texture unit configured to perform texture map filtering operations. In one embodiment, the texture unit is configured to load a texture map (e.g., a 2D array of texels) from memory 304 and sample the texture map to generate sampled texture values ​​for use in shader programs executed by the SM 440. In one embodiment, the texture map is stored in shared memory / L1 cache 470. The texture unit implements texture operations, such as filtering operations using mip maps (i.e., texture maps at different levels of detail). In one embodiment, each SM 440 includes two texture units.

[0141] Each SM 440 also includes N LSUs 554, which implement load and store operations between the shared memory / L1 cache 570 and the register file 520. Each SM 440 includes an interconnect network 580 that connects each functional unit to the register file 520 and the LSUs 554 to the register file 520 and the shared memory / L1 cache 570. In one embodiment, the interconnect network 580 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 520 and to connect the LSUs 554 to memory locations in the register file and the shared memory / L1 cache 570.

[0142] Shared memory / L1 cache 570 is an on-chip memory array that allows data storage and communication between the SM 440 and the primitive engine 435, as well as between threads in the SM 440. In one embodiment, the shared memory / L1 cache 570 includes 128KB of storage capacity and is in the path from the SM 440 to the partition unit 380. The shared memory / L1 cache 570 can be used to cache reads and writes. One or more of the shared memory / L1 cache 570, the L2 cache 460, and the memory 304 is a backing store.

[0143] Combining data cache and shared memory functionality into a single memory block provides the best overall performance for both types of memory access. This capacity can be used by programs as a cache that does not utilize the shared memory. For example, if the shared memory is configured to use half of its capacity, texture and load / store operations can use the remaining capacity. Integration within shared memory / L1 cache 570 enables shared memory / L1 cache 570 to function as a high-throughput pipeline for streaming data, while providing high-bandwidth and low-latency access to frequently reused data.

[0144] When configured for general parallel computing, a simpler configuration can be used compared to graphics processing. Specifically, Figure 3 The fixed-function graphics processing unit is shown bypassed, creating a simpler programming model. In a general-purpose parallel computing configuration, work distribution unit 325 assigns and distributes thread blocks directly to DPC 420. The threads in the block execute the same program, using unique thread IDs in computations to ensure each thread produces unique results, using SM 440 to execute the program and perform computations, using shared memory / L1 cache 570 to communicate between threads, and using LSU 554 to read and write global memory through shared memory / L1 cache 570 and memory partition unit 380. When configured for general-purpose parallel computing, SM 440 can also write commands that scheduler unit 320 can use to start new work on DPC 420.

[0145] The PPU 300 may be included in a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, etc. In one embodiment, the PPU 300 is included on a single semiconductor substrate. In another embodiment, the PPU 300 is included on a system-on-chip (SoC) along with one or more other devices (such as an additional PPU 300, a memory 304, a reduced instruction set computer (RISC) CPU, a memory management unit (MMU), a digital-to-analog converter (DAC), etc.).

[0146] In one embodiment, PPU 300 may be included on a graphics card that includes one or more memory devices 304. The graphics card may be configured to interface with a PCIe slot on a motherboard of a desktop computer. In another embodiment, PPU 300 may be an integrated graphics processing unit (iGPU) or parallel processor included in a chipset of the motherboard.

[0147] Exemplary Computing System

[0148] Systems with multiple GPUs and CPUs are being used across various industries as developers expose and exploit greater parallelism in applications such as artificial intelligence computing. High-performance GPU-accelerated systems with tens to thousands of computing nodes are deployed in data centers, research institutions, and supercomputers to solve larger problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to scale to support this increased bandwidth.

[0149] Figure 5B According to one embodiment, the Figure 3 A conceptual diagram of a processing system 500 implemented by a PPU 300. The exemplary system 500 may be configured to implement Figure 2B-2C The methods 220 and 240 shown in and / or Figure 1A and Figure 2A The processing system 500 includes a CPU 530, a switch 555, and each of the plurality of PPUs 300 and a corresponding memory 304. The NVLink 310 provides a high-speed communication link between each PPU 300. Figure 5B A specific number of NVLink 310 and interconnect 302 connections are shown, but the number of connections connected to each PPU 300 and CPU 530 can vary. Switch 555 interfaces between interconnect 302 and CPU 530. PPU 300, memory 304, and NVLink 310 can be located on a single semiconductor platform to form parallel processing module 525. In one embodiment, switch 555 supports two or more protocols that interface between various different connections and / or links.

[0150] In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between each PPU 300 and the CPU 530, and a switch 555 interfaces between the interconnect 302 and each PPU 300. The PPUs 300, memory 304, and interconnect 302 may be located on a single semiconductor platform to form a parallel processing module 525. In yet another embodiment (not shown), the interconnect 302 provides one or more communication links between each PPU 300 and the CPU 530, and the switch 555 interfaces between each PPU 300 using NVLink 310 to provide one or more high-speed communication links between the PPUs 300. In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between the PPUs 300 and the CPU 530 through the switch 555. In yet another embodiment (not shown), the interconnect 302 provides one or more communication links directly between each PPU 300. One or more NVLink 310 high-speed communication links may be implemented as a physical NVLink interconnect or as an on-chip or on-die interconnect using the same protocol as NVLink 310 .

[0151] In the context of this specification, a single semiconductor platform may refer to a unique, single semiconductor-based integrated circuit fabricated on a die or chip. It should be noted that the term single semiconductor platform may also refer to a multi-chip module with increased connectivity that emulates on-chip operation and is substantially improved by utilizing conventional bus implementations. Of course, various circuits or devices may also be placed separately or in various combinations of semiconductor platforms, depending on the needs of the user. Alternatively, the parallel processing module 525 may be implemented as a circuit board substrate, and each of the PPU 300 and / or memory 304 may be a packaged device. In one embodiment, the CPU 530, switch 555, and parallel processing module 525 are located on a single semiconductor platform.

[0152] In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 Gbit / s, and each PPU 300 includes six NVLink 310 interfaces (e.g., Figure 5B As shown, each PPU 300 includes five NVLink 310 interfaces. Each NVLink 310 provides a data transfer rate of 25 Gbit / s in each direction, with six links providing 300 Gbit / s. When the CPU 530 also includes one or more NVLink 310 interfaces, the NVLink 310 can be used exclusively for Figure 5B PPU to PPU communication shown, or some combination of PPU to PPU and PPU to CPU.

[0153] In one embodiment, NVLink 310 allows direct load / store / atomic access from the CPU 530 to the memory 304 of each PPU 300. In one embodiment, NVLink 310 supports coherency operations, allowing data read from memory 304 to be stored in the cache hierarchy of the CPU 530, reducing cache access latency for the CPU 530. In one embodiment, NVLink 310 includes support for Address Translation Services (ATS), allowing the PPU 300 to directly access page tables within the CPU 530. One or more NVLinks 310 can also be configured to operate in a low-power mode.

[0154] Figure 5C An exemplary system 565 is shown in which the various architecture and / or functionality of the various previous embodiments may be implemented.

[0155] As shown, a system 565 is provided that includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or one or more point-to-point communication protocols. The system 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of random access memory (RAM).

[0156] System 565 also includes an input device 560, a parallel processing system 525, and a display device 545, such as a conventional CRT (cathode ray tube), an LCD (liquid crystal display), an LED (light emitting diode), a plasma display, etc. User input can be received from input device 560 (e.g., a keyboard, a mouse, a touchpad, a microphone, etc.). Each of the aforementioned modules and / or devices can even be located on a single semiconductor platform to form system 565. Alternatively, the modules can be located separately or in various combinations of semiconductor platforms, depending on the user's needs.

[0157] Furthermore, system 565 can be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) through network interface 535 for communication purposes.

[0158] System 565 may also include auxiliary storage (not shown). Auxiliary storage 610 includes, for example, a hard drive and / or a removable storage drive, representative of a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk (DVD) drive, a recording device, or a universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to a removable storage unit in a well-known manner.

[0159] Computer programs or computer control logic algorithms may be stored in the main memory 540 and / or the secondary storage. These computer programs, when executed, enable the system 565 to perform various functions. The memory 540, storage, and / or any other storage are possible examples of computer-readable media.

[0160] The architecture and / or functionality of the various preceding figures can be implemented in the context of a general-purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, a dedicated system, and / or any other desired system. For example, system 565 can take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, a mobile telephone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic.

[0161] Although various embodiments have been described above, it should be understood that they are presented by way of example only, not limitation. Therefore, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

[0162] Graphics processing pipeline

[0163] In one embodiment, the PPU 300 includes a graphics processing unit (GPU). The PPU 300 is configured to receive commands specifying a shader for processing graphics data. Graphics data can be defined as a set of primitives, such as points, lines, triangles, quadrilaterals, triangle strips, etc. Typically, a primitive includes data specifying a plurality of vertices of the primitive (e.g., in a model space coordinate system) and attributes associated with each vertex of the primitive. The PPU 300 can be configured to process the primitives to generate a frame buffer (e.g., pixel data for each of the pixels of a display).

[0164] An application writes model data for a scene (e.g., a collection of vertices and attributes) to memory (such as system memory or memory 304). The model data defines each of the objects that may be visible on the display. The application then makes an API call to the driver kernel, requesting the model data to be rendered and displayed. The driver kernel reads the model data and writes commands to one or more streams to perform operations to process the model data. These commands may reference different shading programs to be implemented on the SMs 440 of the PPU 300, including one or more of vertex shading, hull shading, domain shading, geometry shading, and pixel shading. For example, one or more of the SMs 440 may be configured to execute a vertex shading program that processes multiple vertices defined by the model data. In one embodiment, different SMs 440 may be configured to execute different shading programs simultaneously. For example, a first subset of SMs 440 may be configured to execute a vertex shading program, while a second subset of SMs 440 may be configured to execute a pixel shading program. The first subset of SMs 440 processes the vertex data to generate processed vertex data and writes the processed vertex data to the L2 cache 460 and / or memory 304. After the processed vertex data is rasterized (e.g., converted from three-dimensional data to two-dimensional data in screen space) to generate fragment data, a second subset of SMs 440 performs pixel shading to generate processed fragment data, which is then blended with other processed fragment data and written to a frame buffer in memory 304. Vertex shading programs and pixel shading programs can be executed simultaneously, processing different data from the same scene in a pipelined manner, until all model data for the scene has been rendered to the frame buffer. The contents of the frame buffer are then transmitted to a display controller for display on a display device.

[0165] Figure 6 According to one embodiment, Figure 3 300 . The graphics processing pipeline 600 is an abstract flow chart of the processing steps implemented to generate a 2D computer-generated image from 3D geometric data. As is well known, pipeline architectures can perform long-latency operations more efficiently by breaking the operations into multiple stages, where the output of each stage is coupled to the input of the next consecutive stage. Thus, the graphics processing pipeline 600 receives input data 601 that is passed from one stage of the graphics processing pipeline 600 to the next stage to generate output data 602. In one embodiment, the graphics processing pipeline 600 may represent a graphics processing pipeline composed of API-defined graphics processing pipeline. Alternatively, graphics processing pipeline 600 can be implemented in the context of the functionality and architecture of the previous figures and / or one or more of any subsequent figures.

[0166] like Figure 6As shown, graphics processing pipeline 600 includes a pipeline architecture comprising multiple stages. These stages include, but are not limited to, a data assembly stage 610, a vertex shading stage 620, a primitive assembly stage 630, a geometry shading stage 640, a viewport scale, cull, and clip (VSCC) stage 650, a rasterization stage 660, a fragment shading stage 670, and a raster operation stage 680. In one embodiment, input data 601 includes commands that configure a processing unit to implement the stages of graphics processing pipeline 600 and configure geometric primitives (e.g., points, lines, triangles, quads, triangle strips, or fans, etc.) to be processed by these stages. Output data 602 may include pixel data (i.e., color data), which is copied to a frame buffer or other type of surface data structure in memory.

[0167] The data assembly stage 610 receives input data 601, which specifies vertex data for high-level surfaces, primitives, etc. The data assembly stage 610 collects the vertex data in temporary storage or queues, such as by receiving a command from the host processor that includes a pointer to a buffer in memory and reading the vertex data from the buffer. The vertex data is then passed to the vertex shading stage 620 for processing.

[0168] The vertex shading stage 620 processes vertex data by executing a set of operations (e.g., a vertex shader or program) on each vertex at a time. A vertex may be specified, for example, as a 4-coordinate vector (e.g., ) associated with one or more vertex attributes (e.g., color, texture coordinates, surface normal, etc.).<x,y,z,w> ). The vertex shading stage 620 can manipulate various vertex attributes, such as position, color, texture coordinates, etc. In other words, the vertex shading stage 620 performs operations on the vertex coordinates or other vertex attributes associated with the vertex. These operations typically include lighting operations (e.g., modifying the color attribute of a vertex) and transformation operations (e.g., modifying the coordinate space of a vertex). For example, a vertex can be specified using coordinates in an object coordinate space, which is transformed by multiplying the coordinates by a matrix that converts the coordinates from the object coordinate space to world space or normalized-device-coordinate (NCD) space. The vertex shading stage 620 generates transformed vertex data that is passed to the primitive assembly stage 630.

[0169] The primitive assembly stage 630 collects the vertices output by the vertex shading stage 620 and groups the vertices into geometric primitives for processing by the geometry shading stage 640. For example, the primitive assembly stage 630 can be configured to group every three consecutive vertices into geometric primitives (e.g., triangles) for transmission to the geometry shading stage 640. In some embodiments, particular vertices can be reused for consecutive geometric primitives (e.g., two consecutive triangles in a triangle strip can share two vertices). The primitive assembly stage 630 transmits the geometric primitives (e.g., a collection of associated vertices) to the geometry shading stage 640.

[0170] The geometry shading stage 640 processes geometric primitives by executing a set of operations (e.g., geometry shaders or programs) on the geometric primitives. A tessellation operation can generate one or more geometric primitives from each geometric primitive. In other words, the geometry shading stage 640 can subdivide each geometric primitive into a finer mesh of two or more geometric primitives for processing by the rest of the graphics processing pipeline 600. The geometry shading stage 640 passes the geometric primitives to the viewport SCC stage 650.

[0171] In one embodiment, the graphics processing pipeline 600 may operate within a streaming multiprocessor and vertex shading stage 620, primitive assembly stage 630, geometry shading stage 640, fragment shading stage 670, and / or hardware / software associated therewith, and may perform processing operations sequentially. Once the sequential processing operations are completed, in one embodiment, the viewport SCC stage 650 may utilize the data. In one embodiment, primitive data processed by one or more stages in the graphics processing pipeline 600 may be written to a cache (e.g., an L1 cache, a vertex cache, etc.). In this case, in one embodiment, the viewport SCC stage 650 may access the data in the cache. In one embodiment, the viewport SCC stage 650 and the rasterization stage 660 are implemented as fixed function circuits.

[0172] The viewport SCC stage 650 performs viewport scaling, culling, and clipping of geometric primitives. Each surface being rendered is associated with an abstract camera position. The camera position represents the position of the viewer viewing the scene and defines a viewing cone that surrounds the objects of the scene. The viewing cone can include a viewing plane, a back plane, and four clipping planes. Any geometric primitives that are completely outside the viewing cone can be culled (e.g., discarded) because they will not contribute to the final rendered scene. Any geometric primitives that are partially inside the viewing cone and partially outside the viewing cone can be clipped (e.g., converted to new geometric primitives that are enclosed within the viewing cone). In addition, each geometric primitive can be scaled based on the depth of the viewing cone. All potentially visible geometric primitives are then transferred to the rasterization stage 660.

[0173] The rasterization stage 660 converts 3D geometric primitives into 2D fragments (e.g., capable of being used for display, etc.). The rasterization stage 660 can be configured to use the vertices of the geometric primitives to set a set of plane equations from which various attributes can be interpolated. The rasterization stage 660 can also calculate a coverage mask for multiple pixels, which indicates whether one or more sample positions of the pixel intercept the geometric primitive. In one embodiment, a z test can also be performed to determine whether the geometric primitive is occluded by other geometric primitives that have already been rasterized. The rasterization stage 660 generates fragment data (e.g., interpolated vertex attributes associated with a specific sample position for each covered pixel), which is passed to the fragment shading stage 670.

[0174] The fragment shading stage 670 processes the fragment data by executing a set of operations (e.g., a fragment shader or program) on each of the fragments. The fragment shading stage 670 can generate pixel data (e.g., color values) for the fragment, such as by performing lighting operations or sampling texture mapping using the fragment's interpolated texture coordinates. The fragment shading stage 670 generates pixel data, which is passed to the raster operations stage 680.

[0175] The raster operations stage 680 may perform various operations on the pixel data, such as performing alpha tests, stencil tests, and blending the pixel data with other pixel data corresponding to other fragments associated with the pixel. When the raster operations stage 680 has completed processing the pixel data (e.g., output data 602), the pixel data may be written to a render target, such as a frame buffer, a color buffer, etc.

[0176] It should be appreciated that one or more additional stages may be included in the graphics processing pipeline 600 in addition to or in place of one or more of the above-described stages. Various implementations of the abstract graphics processing pipeline may implement different stages. Furthermore, in some embodiments, one or more of the above-described stages may be excluded from the graphics processing pipeline (such as the geometry shading stage 640). Other types of graphics processing pipelines are contemplated within the scope of the present disclosure. Furthermore, any stage of the graphics processing pipeline 600 may be implemented by one or more dedicated hardware units within a graphics processor (such as the PPU 300). Other stages of the graphics processing pipeline 600 may be implemented by programmable hardware units (such as the SM 440 of the PPU 300).

[0177] The graphics processing pipeline 600 can be implemented via an application program executed by a host processor (such as a CPU). In one embodiment, a device driver can implement an application programming interface (API) that defines various functions that can be utilized by the application program to generate graphics data for display. A device driver is a software program that includes multiple instructions that control the operation of the PPU 300. The API provides an abstraction for programmers, allowing them to utilize specialized graphics hardware (such as the PPU 300) to generate graphics data without requiring them to utilize the specific instruction set of the PPU 300. An application program can include API calls that are routed to the device driver of the PPU 300. The device driver interprets the API calls and performs various operations in response to the API calls. In some cases, the device driver can perform operations by executing instructions on the CPU. In other cases, the device driver can perform operations at least in part by initiating operations on the PPU 300 using an input / output interface between the CPU and the PPU 300. In one embodiment, the device driver is configured to implement the graphics processing pipeline 600 using the hardware of the PPU 300.

[0178] Various programs may be executed within the PPU 300 to implement the various stages of the graphics processing pipeline 600. For example, a device driver may launch a kernel on the PPU 300 to execute the vertex shading stage 620 on one SM 440 (or multiple SMs 440). The device driver (or the initial kernel executed by the PPU 400) may also launch other kernels on the PPU 400 to execute other stages of the graphics processing pipeline 600, such as the geometry shading stage 640 and the fragment shading stage 670. In addition, some of the stages of the graphics processing pipeline 600 may be implemented on fixed unit hardware, such as a rasterizer or data assembler implemented within the PPU 400. It should be appreciated that the results from one kernel may be processed by one or more intermediate fixed-function hardware units before being processed by subsequent kernels on the SM 440.

[0179] Machine Learning

[0180] Deep neural networks (DNNs) developed on processors such as the PPU 300 are already being used in a variety of use cases: from self-driving cars to faster drug development, from automatic image captioning in online image databases to intelligent real-time language translation in video chat applications. Deep learning is a technology that models the neural learning process of the human brain, constantly learning, getting smarter, and delivering more accurate results faster over time. A child, initially taught by an adult to correctly identify and classify various shapes, eventually becomes able to recognize shapes without any tutoring. Similarly, deep learning or neural learning systems need to be trained in object recognition and classification in order to become smarter and more efficient at recognizing basic objects, occluded objects, and assigning context to objects.

[0181] At the simplest level, neurons in the human brain examine the various inputs they receive, assign a level of importance to each of these inputs, and pass outputs to other neurons for processing. An artificial neuron, or perceptron, is the most basic model of a neural network. In one example, a perceptron can receive one or more inputs representing various features of the object it is being trained to recognize and classify, and each of these features is assigned a certain weight based on its importance in defining the object's shape.

[0182] Deep neural network (DNN) models consist of multiple layers of connected nodes (e.g., perceptrons, Boltzmann machines, radial basis functions, convolutional layers, etc.), which can be trained with large amounts of input data to solve complex problems quickly and accurately. In one example, the first layer of a DNN model breaks down an input image of a car into its components and looks for basic patterns (such as lines and angles). The second layer assembles the lines to find higher-level patterns, such as wheels, windshields, and mirrors. The next layer identifies the type of vehicle, and the final layers generate labels for the input image, identifying the model of a specific car brand.

[0183] Once trained, a DNN can be deployed and used to recognize and classify objects or patterns in a process called inference. Examples of inference (the process by which a DNN extracts useful information from a given input) include recognizing handwritten numbers on check deposits at an ATM, identifying images of friends in photos, providing movie recommendations to over 50 million users, identifying and classifying different types of cars, pedestrians, and road hazards in self-driving cars, or translating human speech in real time.

[0184] During training, data flows through the DNN in a forward propagation phase until a prediction is produced, which indicates the label corresponding to the input. If the neural network does not correctly label the input, the error between the correct label and the predicted label is analyzed, and the weights are adjusted for each feature during a backward propagation phase until the DNN correctly labels that input and other inputs in the training dataset. Training complex neural networks requires a large amount of parallel computing performance, including floating-point multiplications and additions supported by the PPU 300. Inference is a less computationally intensive and latency-sensitive process than training, in which a trained neural network is applied to new inputs it has not seen before to classify images, translate speech, and generally reason about new information.

[0185] Neural networks rely heavily on matrix math operations, and complex, multi-layer networks require significant floating-point performance and bandwidth for efficiency and speed. With thousands of processing cores optimized for matrix math operations and delivering tens to hundreds of TFLOPS of performance, the PPU 300 is a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications.

[0186] Exemplary Technical Advantages of Some Embodiments

[0187] Certain exemplary embodiments provide improved system performance by reducing memory access latency for texture information access during graphics processing.

[0188] As described above, conventional latency hiding techniques such as pipelining may not be sufficient to mitigate the performance degradation caused by the relatively long latency in accessing texture maps, etc., during graphics processing. Similarly, most conventional prefetching techniques that have been applied to CPUs may not be applicable to highly parallel GPUs and texture information that has different storage and access characteristics than typical CPU workloads. Therefore, by reducing memory access latency for texture information to provide improved system performance, certain embodiments of the present invention enable retaining more of the processing speed improvements provided by faster processors, rather than experiencing performance degradation due to long memory access latency.

[0189] In certain exemplary embodiments, the additional circuitry required at the texture unit and L2 cache is cost-effectively incorporated into the texture processing unit circuitry and the L2 cache control circuitry, thereby providing an efficient technique by which the speed associated with texture accesses can be increased. Furthermore, exemplary embodiments accelerate at least some memory accesses by optimizing for certain frequent memory access patterns.

[0190] While the speedup achieved in exemplary embodiments through more efficient memory access to texture maps is helpful in many types of applications, the speedup may be particularly useful in games with very rapid screen changes and / or action and in highly time-critical applications, such as, but not limited to, automotive displays (e.g., autonomous driving car displays, etc.), and the like.

[0191] Many modifications and variations of the present invention are possible in light of the above teachings.It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

Claims

1. A method for displaying a scene, comprising: while performing texture mapping on a scene by a processor using a first block of a stored texture, generating, by the processor, a prefetch request to retrieve all or a portion of a second block of the stored texture from a first level memory of a memory hierarchy; Retrieving the whole or portion of the second block from the first level memory in response to the prefetch request; storing all or part of the retrieved second block in an area of ​​a second level memory of the memory hierarchy, wherein the area of ​​the second level memory is accessible by the processor and another processor, and wherein the second level memory is an intermediate level memory between the first level memory and a third level memory in the memory hierarchy; performing texture mapping on the scene using all or a portion of the retrieved second block; as well as Rendering the scene to a display device. 2 . The method of claim 1 , wherein the texture mapping of the scene using the first block is performed by the processor, and the texture mapping of the scene using the second block is performed by the other processor.

3. The method of claim 1 , wherein the address of the whole or portion of the second block is dynamically calculated by the processor based on a block size value determined from header information of the texture, and wherein the block size value is different from another block size value indicated for another texture.

4. The method of claim 3 , wherein the address of the whole or part of the second block is further determined based on the address of the first block, and wherein the prefetch request includes the determined address of the whole or part of the second block, and / or the block size value determined from the header information of the texture. 5 . The method of claim 4 , wherein the first block and the second block each comprise an integer number of texels stored in the first level memory in a block linear layout in the same texture map.

6. The method according to claim 1, further comprising: The number of outstanding prefetch requests is counted, and if the number of outstanding prefetch requests is less than a threshold, the prefetch request is sent to the first-level memory, otherwise the prefetch request is discarded without being sent to the first-level memory.

7. The method of claim 6, wherein said counting, said sending, and said discarding are performed in circuitry associated with said second level of memory.

8. The method according to claim 1, further comprising: A determination is made as to whether all or part of the requested second block exists in the second level of the memory hierarchy, and if so, the prefetch request is discarded without sending the prefetch request to the first level of memory.

9. The method of claim 8, wherein the discarding is performed in circuitry associated with the second level of memory.

10. The method of claim 8, wherein the discarding is performed in circuitry associated with the processor.

11. The method according to claim 1, wherein the processor and the another processor respectively access different areas in the third level of their respective L1 caches.

12. The method according to claim 1, further comprising: The texture map is detected as being part of a full screen rendering, and the generating prefetch request is performed in response to the detecting.

13. The method of claim 1, wherein the storing comprises: All or part of the retrieved second block is stored in an area of ​​the second-level memory, but not in the third-level memory.

14. The method of claim 13, wherein all or part of the retrieved second block stored in the second level memory is subsequently stored in the tertiary memory in response to a subsequent fetch request from the processor or the further processor.

15. A parallel processing system for displaying a scene, comprising: Multiple processors; a cache hierarchy comprising at least a first level cache memory and a second level cache memory; Display interface; as well as a memory interface configured to provide the plurality of processors with access to off-chip memory, wherein the plurality of processors and control circuitry associated with the cache hierarchy are configured to: while performing texture mapping on the scene by a first processor from the plurality of processors using a first block of a stored texture, generating, by the first processor, a prefetch request to retrieve all or a portion of a second block of the stored texture from a memory hierarchy including the cache hierarchy; In response to the prefetch request, retrieving the requested all or part of the second block from the off-chip memory through the memory interface; storing all or part of the retrieved second block in a region of a second level cache memory, wherein the region of the second level cache memory is accessible by the first processor and the second processor from the plurality of processors, and wherein the second level cache memory is an intermediate level of memory between the off-chip memory and the first level cache memory; performing texture mapping on the scene using all or a portion of the retrieved second block; as well as The scene is rendered to a display device through the display interface.

16. The parallel processing system of claim 15, wherein the texture mapping of the scene using the first block is performed by the first processor, and the texture mapping of the scene using the second block is performed by the second processor.

17. The parallel processing system of claim 15 , wherein the address of the whole or portion of the second block is dynamically calculated by the first processor based on a block size value determined from header information of the texture, and wherein the block size value is different from another block size value indicated for another texture.

18. The parallel processing system of claim 17 , wherein the address of all or part of the second block is further determined based on the address of the first block, and wherein the prefetch request includes the determined address of the second block and / or the block size value determined from the header information of the texture.

19. The parallel processing system of claim 18, wherein the first block and the second block each comprise an integer number of texels stored in the off-chip memory in a block linear layout in a same texture map.

20. The parallel processing system of claim 15 , further comprising a request throttling circuit connected to the second-level cache memory, wherein the throttling circuit is configured to count the number of outstanding prefetch requests and send the prefetch requests to the off-chip memory via the memory interface if the number of outstanding prefetch requests is less than a threshold, and otherwise discard the prefetch requests without sending them to the off-chip memory.

21. The parallel processing system of claim 15 , further comprising a request deduplication circuit, wherein the deduplication circuit is configured to determine whether the second block exists in the second-level cache memory, and if so, discard the prefetch request without sending the prefetch request to the off-chip memory.

22. The parallel processing system of claim 21, wherein the request deduplication circuit is connected to the first processor.

23. A system on a chip (SoC), comprising: at least one central processing unit (CPU); as well as at least one parallel processing unit (PPU) connected to the CPU, Each PPU includes: Multiple multiprocessors; Multiple special function units; a cache hierarchy comprising at least a first level cache memory and a second level cache memory; and a memory interface configured to provide access to off-chip memory, wherein the plurality of special function units and control circuitry associated with the cache hierarchy are configured to, in response to an instruction received from one of the multiprocessors, perform: while a first special function unit from the plurality of special function units performs texture mapping on a scene using a first block of a stored texture, generating, by the first special function unit, a prefetch request to retrieve all or a portion of a second block of the stored texture from a memory hierarchy including the cache hierarchy; In response to the prefetch request, retrieving the requested all or part of the second block from the off-chip memory through the memory interface; storing all or part of the retrieved second block in a region of a second-level cache memory, wherein the region of the second-level cache memory is accessible by the first and second special function units from the plurality of special function units, and wherein the second-level cache memory is an intermediate level of memory between the off-chip memory and the first-level cache memory; performing texture mapping on the scene using all or part of the retrieved second block; and The scene is rendered to a display device through a display interface.

Images