Random optimization of surface cacheability in parallel processing units

Abstract

A processing system [100] selectively allocates storage at a local cache [120] of a parallel processing unit [110] for cache lines of a repeating data pattern [420] that exceeds a storage capacity of the cache. The processing system identifies a repeating data pattern having cache lines with a reuse distance that exceeds the storage capacity of the cache. A cache controller [130] allocates storage at the cache only for a subset [140] of the cache lines of the repeating data pattern and excludes a remaining portion [415] of the cache lines of the repeating data pattern from the cache. By limiting the cache to store only the subset of cache lines of the repeating data pattern, the cache controller increases a hit rate of the subset of cache lines at the cache.

Description

BACKGROUND

[0001] Processing systems that include parallel processing units, such as graphics processing units (GPUs), implement a cache hierarchy (or multi-level cache) that uses different speed cache levels to store frequently accessed data. Data that is more frequently requested is typically cached in a relatively fast cache, such as an LI cache, that is physically (or logically) disposed closer to a processor core or compute unit. Higher level caches, such as L2, L3, and so on, store less frequently requested data. The last level cache (LLC) is the highest level (and lowest access speed) cache, and the LLC reads data directly from system memory and writes data directly to system memory. Caches differ from system memory in that they implement a cache replacement policy to replace data in a cache line in response to new data needing to be written to the cache line. For example, a least recently used (LRU) policy replaces data in a cache line that has not been accessed for the longest time interval by evicting the data in the LRU cache line and writing the new data to the LRU cache line. SUMMARY

[0002] Examples include a method that includes identifying a repeating data pattern having a number of cache lines that exceeds a storage capacity of a cache, and allocating storage at the cache only for a subset of cache lines of the identified repeating data pattern. In some examples, the method further includes bypassing storage of a remaining portion of the cache lines of the repeating data pattern at the cache. In some examples, allocating storage only for the subset includes selecting the cache lines of the repeating data pattern that exceed a maximum number of cache lines storable at the cache as the remaining portion.

[0003] In some examples, identifying the repeating pattern includes determining that a reuse distance of the cache lines of the repeating data pattern is greater than a number of cache lines storable at the cache, and limiting the subset to the number of cache lines storable at the cache. In some examples, the method further includes iteratively selecting cache lines of the repeating data pattern to exclude from the cache, determining a hit rate of a subset of cache lines of the repeating data pattern that are not excluded from the cache, and selecting cache lines of the repeating data pattern to exclude from the cache based on the hit rate.

[0004] In some examples, the method further includes partitioning the cache into two or more parts, and assigning a first part of the cache to a duplicated data pattern. In some examples, determining that the reuse distance of cache lines for a duplicated data pattern is greater than the number of cache lines that can be stored in the cache includes determining that the reuse distance of cache lines for a duplicated data pattern is greater than the number of cache lines that can be stored in the first part of the cache. In some examples, the duplicated data pattern includes textures.

[0005] Another exemplary method includes allocating cache entries only to a subset of cache lines representing the duplicate data pattern in response to determining that all cache lines representing the duplicate data pattern exceed the cache's storage capacity. In some examples, the method also includes bypassing the storage of the remaining cache lines representing the duplicate data pattern at the cache location. In some examples, allocating cache entries only to this subset includes selecting cache lines representing the duplicate data pattern that exceed the maximum number of cache lines the cache can store as the remaining portion.

[0006] In some examples, the method further includes: determining that the reuse distance of cache lines for a duplicated data pattern is greater than the number of cache lines that can be stored in the cache, and limiting the subset to the number of cache lines that can be stored in the cache. In some examples, the method further includes: iteratively selecting cache lines for a duplicated data pattern to exclude from the cache, determining the hit rate of cache lines for a duplicated data pattern that are not excluded from the cache, and selecting cache lines for a duplicated data pattern to exclude from the cache based on the hit rate.

[0007] In some examples, the method further includes partitioning the cache into two or more parts, and assigning a first part of the cache to a duplicated data pattern. Determining that the reuse distance of cache lines for the duplicated data pattern is greater than the number of cache lines that can be stored in the cache includes determining that the reuse distance of cache lines for the duplicated data pattern is greater than the number of cache lines that can be stored in the first part of the cache. In some examples, the duplicated data pattern includes textures.

[0008] In another example, the device includes a parallel processing unit. The parallel processing unit includes a cache and a cache controller that identifies duplicate data patterns with a number of cache lines exceeding the cache's storage capacity, and allocates storage at the cache only for a subset of cache lines representing the duplicate data patterns. In some examples, the cache controller is used to bypass storing the remaining cache lines representing the duplicate data patterns at the cache.

[0009] In some examples, the cache controller is configured to select cache lines in a duplicate data pattern that exceed the maximum number of cache lines the cache can store as the remainder. In some examples, the cache controller is configured to determine whether the reuse distance of a cache line in a duplicate data pattern is greater than the number of cache lines that can be stored in the cache, and to limit that subset to the number of cache lines that can be stored in the cache. In one example, the cache controller is configured to iteratively select cache lines in a duplicate data pattern to exclude from the cache, determine the hit rate of cache lines in a duplicate data pattern that are not excluded from the cache, and select cache lines in a duplicate data pattern to exclude from the cache based on that hit rate.

[0010] In another example, the cache controller is configured to partition the cache into two or more parts and assign a first part of the cache to a duplicated data pattern. The cache controller is further configured to determine that the reuse distance of a cache line in the first duplicated data pattern is greater than the number of cache lines that can be stored in the first part of the cache. Attached Figure Description

[0011] This disclosure can be better understood by referring to the accompanying drawings, and many of its features and advantages will be apparent to those skilled in the art. The same reference numerals are used in different drawings to denote similar or identical items.

[0012] Figure 1 It is a block diagram of a processing system configured to selectively cache a subset of repetitive data patterns according to some implementation schemes.

[0013] Figure 2 This is a diagram illustrating a cache for repeating data patterns without selecting a subset of the repeating patterns, based on some implementation schemes.

[0014] Figure 3 This is a diagram illustrating a cache for a recurring data pattern based on some implementation schemes.

[0015] Figure 4 This is a block diagram of a cache controller that allocates storage at the cache only for a subset of cache lines that represent duplicate data patterns, according to some implementation schemes.

[0016] Figure 5 This is a block diagram of a cache controller according to some implementation schemes, which partitions the cache and allocates storage only for a subset of cache lines with duplicate data patterns in parts of the cache.

[0017] Figure 6 This is a diagram illustrating cache lines that selectively cache duplicate data patterns according to some implementation schemes.

[0018] Figure 7 This is a diagram illustrating cache lines that selectively cache duplicate data patterns according to some implementation schemes.

[0019] Figure 8 This is a flowchart illustrating a method for selectively caching cache lines of duplicate data patterns, according to some implementation schemes. Detailed Implementation

[0020] The processor cores of a parallel processing unit typically perform the dispatch of work items, which sequentially access recurring data patterns, such as surfaces. To illustrate, this involves reading surfaces (such as textures) attached to a series of work item dispatches in consecutive rendering or denoising passes. For example, as the position or angle of the camera view changes, many elements of a video game scene are rendered in consecutive frames. Successive passes require the same texture data mapped to pixels. In some cases, the size of the texture exceeds the storage capacity of the parallel processing unit's lowest-level cache (L1 cache).

[0021] Traditionally, L1 cache controllers employ an LRU cache replacement policy, retrieving data requested by the processor core that is not currently stored in the L1 cache (i.e., a cache miss) from higher-level caches in the cache hierarchy. To make room for the requested cache line, the least recently used cache line in the L1 cache is evicted. However, if the reuse distance of a duplicated data pattern (i.e., the number of cache lines in each repetition of the pattern) exceeds the storage capacity of the L1 cache, the cache line may have already been evicted by the time the cache controller receives a new request for it, resulting in a low (or zero) hit rate. Retrieving the data from further levels of the memory hierarchy for subsequent allocation to make it accessible consumes resources and increases latency.

[0022] Figures 1 to 8This paper illustrates a technique by which a processing system selectively allocates storage for cache lines of a duplicate data pattern at the local cache of a parallel processing unit when the size of the duplicate pattern exceeds the storage capacity of the cache. The processing system identifies a duplicate data pattern with cache lines having a reuse distance exceeding the storage capacity of the cache. The cache controller allocates storage at the cache only for a subset (i.e., some but not all) of the cache lines of the duplicate data pattern, excluding the remaining cache lines of the duplicate data pattern from the cache. For example, if the maximum number of cache lines that can be stored at one time is X cache lines, and the processing system identifies a duplicate data pattern as X+Y cache lines, in some embodiments, the cache controller allocates storage only for the first X cache lines of the duplicate data pattern, excluding the remaining Y cache lines from being stored at the cache. In response to a request from the processor core for the remaining Y cache lines, the cache controller provides the remaining Y cache lines directly to the processor core from a higher level of the cache hierarchy, bypassing the storage of the Y cache lines at the cache. By limiting the cache to storing only the first X cache lines of a duplicate data pattern, the cache controller increases the cache hit rate of the first X cache lines, thereby reducing latency and improving the user experience.

[0023] In some implementations, the processing system randomly selects a subset of cache lines from duplicate data patterns during an iterative process to store in the cache. The processing system measures the hit rate of each selected subset of cache lines and, based on that hit rate, selects cache lines of the duplicate data pattern to exclude from the cache. In some implementations, the cache controller partitions the cache into two or more parts and assigns one part of the cache to store a subset of cache lines of a first duplicate data pattern, and another part of the cache to store a subset of cache lines of a second duplicate data pattern. For each partitioned cache, the cache controller compares the reuse distance of the cache lines of the first duplicate data pattern with the storage capacity of the cache portion to which the first duplicate data pattern is assigned. Based on the reuse distance and the storage capacity of the cache portion, the cache controller selects a subset of cache lines of the first duplicate data pattern to store in the cache and excludes the remaining cache lines of the first duplicate data pattern from the cache.

[0024] Figure 1A processing system 100 according to some embodiments is shown, which is configured to selectively allocate storage at the local cache of a parallel processing unit for cache lines of repetitive data patterns that exceed the storage capacity of the cache. According to some embodiments, the processing system 100 includes a parallel processing unit 110, such as a graphics processing unit (GPU), for creating visual images intended for output to a display 175. A parallel processor is a processor capable of executing a single instruction on multiple data or threads in a parallel manner.

[0025] Examples of parallel processors include processors such as graphics processing units (GPUs), massively parallel processors, single-instruction multiple-data (SIMD) architecture processors, and single-instruction multiple-thread (SIMT) architecture processors for performing graphics, machine intelligence, or computational operations. In some implementations, the parallel processor is a separate device included as part of a computer. In other implementations, such as advanced processor units, the parallel processor is included in a single device along with a host processor such as a central processing unit (CPU). Although the following description uses graphics processing units (GPUs) for illustrative purposes, the implementations and specific embodiments described below are applicable to other types of parallel processors.

[0026] Processing system 100 includes system memory 150 (referred to herein as memory 150). Some embodiments of memory 150 are implemented as dynamic random access memory (DRAM). However, memory 150 may also be implemented using other types of memory, including static random access memory (SRAM), non-volatile RAM, etc. In an illustrated embodiment, parallel processing unit 110 communicates with memory 150 via bus 160. However, some embodiments of parallel processing unit 110 communicate with memory 150 via direct connection or via other buses, bridges, switches, routers, etc. Parallel processing unit 110 executes instructions stored in memory 150, and parallel processing unit 110 stores information (such as the results of executed instructions) in memory 150. For example, memory 150 may store copies of instructions from application program 155 to be executed by parallel processing unit 110. Some embodiments of parallel processing unit 110 include multiple processor cores (referred to as compute units) 115 that execute instructions independently, either simultaneously or in parallel.

[0027] Processing system 100 is typically configured to execute a set of instructions (e.g., a computer program), such as application 155, to perform specified tasks of an electronic device. Examples of such tasks include controlling various aspects of the operation of the electronic device, displaying information to a user to provide a specified user experience, communicating with other electronic devices, and so on. Therefore, in different embodiments, processing system 100 is used in one of a variety of types of electronic devices, such as desktop computers, laptop computers, servers, game consoles, tablet computers, smartphones, and so on. Components of processing system 100 are implemented as hardware, firmware, software, or any combination thereof. It should be understood that processing system 100 may include more than Figure 1 The processing system 100 may include more or fewer components. For example, it may additionally include one or more input interfaces, non-volatile storage devices, one or more output interfaces, network interfaces, and one or more displays or display interfaces.

[0028] Processing system 100 includes a central processing unit (CPU) 105 for executing instructions. Some embodiments of CPU 105 include multiple processor cores (not shown for clarity) that execute instructions independently, either simultaneously or in parallel. CPU 105 is also connected to bus 160 and thus communicates with parallel processing unit 110 and memory 150 via bus 160. CPU 105 executes instructions (such as program code of application 155 stored in memory 150), and CPU 105 stores information (such as the results of the executed instructions) in memory 150. CPU 105 is also capable of initiating graphics processing by issuing a draw call to parallel processing unit 110. A draw call is a command generated by CPU 105 and transmitted to parallel processing unit 110 to instruct parallel processing unit 110 to render an object (or a portion of an object) in a frame. Some embodiments of a draw call include information defining the textures, states, shaders, rendering objects, buffers, etc., used by parallel processing unit 110 to render the object or a portion thereof. Parallel processing unit 110 renders an object to produce pixel values ​​provided to display 175, which uses the pixel values ​​to display an image representing the rendered object.

[0029] In some implementations, each frame to be rendered is processed in multiple channels by the graphics pipeline of the parallel processing unit 110. For example, during the first channel on the scene geometry, only the properties required for per-pixel lighting are calculated and written to the G-buffer. During the second channel, the graphics pipeline outputs only diffuse and specular lighting data. In the third channel as the frame passes through the graphics pipeline, the graphics pipeline reads backlighting data and outputs the final per-pixel shading. Thus, in multi-channel rendering, the scene and the associated objects of the frame are rendered multiple times. Whenever an object is drawn, the graphics pipeline calculates additional aspects of the object's appearance and combines these additional aspects with previous results. Each time a frame or an object of a frame is rendered by the graphics pipeline, that graphics pipeline is referred to as a rendering channel.

[0030] Input / output (I / O) engine 170 handles input or output operations associated with display 175 and other components of processing system 100, such as keyboard, mouse, printer, external disk, etc. I / O engine 170 is coupled to bus 160, enabling I / O engine 170 to communicate with parallel processing unit 110, memory 150, or CPU 105. In an illustrated embodiment, I / O engine 170 is configured to read information stored on external storage medium 180, such as optical disc (CD), digital video disc (DVD), etc. External storage medium 180 stores information representing program code used to implement applications such as video games. The program code on external storage medium 180 can be written to memory 150 to form a copy of instructions to be executed by parallel processing unit 110 or CPU 105.

[0031] In some implementations, the parallel processing unit 110 implements the graphics pipeline (for clarity, Figure 1 (Not shown in the image), the graphics pipeline includes multiple stages configured to process different primitives simultaneously in response to draw calls. The stages of the graphics pipeline in the parallel processing unit 110 can simultaneously process different primitives generated by applications such as video games. When geometry is submitted to the graphics pipeline, hardware state settings are selected to define the state of the graphics pipeline. Examples of states include rasterizer state, blending state, depth stencil state, the primitive topology type of the submitted geometry, and the shaders used to render the scene (e.g., vertex shader, domain shader, geometry shader, hull shader, pixel shader, etc.). Shaders implemented in the graphics pipeline state are represented by corresponding bytecode. In some cases, information representing the graphics pipeline state is hashed or compressed to provide a more efficient representation of the graphics pipeline state.

[0032] Driver 165 is a computer program that allows higher-level graphics computing programs, such as those from application 155, to interact with parallel processing unit 110. For example, driver 165 translates standard code received from application 155 into a native-format command stream understood by parallel processing unit 110. Driver 165 allows input from application 155 to guide settings on parallel processing unit 110. Such settings include selection of rendering mode, anti-aliasing control, texture filtering control, batch tiling control, and deferred pixel shading control.

[0033] To execute a set of commands received from the CPU, the parallel processing unit 110 includes multiple computing units 115 and an L1 cache 120. The multiple computing units 115 together perform shading operations on the assignment of work items (not shown). In various embodiments, the computing units 115 perform geometric operations, texturing operations, tessellation operations, vertex operations, mesh operations, primitive operations, ray tracing operations, computational operations, etc., or any combination thereof, based on commands received from a command processor (not shown). In some embodiments, to perform these operations, each computing unit 115 includes one or more SIMD elements configured to perform a specified operation using the received assigned work item.

[0034] L1 cache 120 stores data for the multiple computation units 115. Therefore, during shader operations, the multiple computation units 115 store and retrieve data from L1 cache 120, where the stored and retrieved data is based on a specific work item being processed. For example, in some embodiments, each dispatched work item corresponds to an individual pixel of an image, and L1 cache 120 stores data (e.g., texture values) for each individual pixel or subset of individual pixels included in the dispatch. In some embodiments, parallel processing units 110 are associated with a memory hierarchy having multiple cache levels and system memory 150, and L1 cache 120 represents the lowest level among the multiple cache levels.

[0035] To increase the hit rate of duplicate data patterns exceeding the storage capacity of L1 cache 120 at L1 cache 120, parallel processing unit 110 includes a cache controller 130 configured to allocate storage at L1 cache 120 only for a subset 140 of cache lines representing duplicate data patterns, and to exclude the remainder of cache lines representing duplicate data patterns from the cache. In response to a request from compute unit 115 for the remainder of cache lines representing duplicate data patterns, cache controller 130 provides the remaining cache lines directly from a higher level of the cache hierarchy (not shown) to compute unit 115, bypassing the storage of the remaining cache lines at L1 cache 120. By restricting L1 cache 120 to store only cache lines representing duplicate data patterns that simultaneously meet the storage capacity of L1 cache 120, cache controller 130 increases the hit rate of the selected subset 140 of cache lines at L1 cache 120.

[0036] Cache controller 130 includes a pattern recognition unit 135 configured to identify duplicate data patterns that exceed the storage capacity of L1 cache 120. Pattern recognition unit 135 measures the reuse distance (DR) of cache lines in the duplicate data pattern (i.e., the number of cache lines in each repetition of the pattern) and compares the reuse distance DR to the total number of cache lines CL1 that L1 cache 120 can store (i.e., the storage capacity of L1 cache 120). If the reuse distance DR exceeds the storage capacity CL1 of L1 cache 120, cache controller 130 limits the number of cache lines in L1 cache 120 that have allocated storage space for the duplicate data pattern to a subset 140 of cache lines conforming to the storage capacity CL1 of L1 cache 120. Cache controller 130 excludes the remaining DR-CL1 cache lines of the duplicate data pattern from L1 cache 120. In some implementations, in response to a relatively low hit rate exhibited by the L1 cache 120, the pattern recognition unit 135 randomly selects a subset of cache lines from the duplicate data patterns during the iteration process to store in the L1 cache 120. The cache controller 130 measures the hit rate of each selected subset of cache lines and selects cache lines of the duplicate data pattern to be excluded from the cache based on the hit rate.

[0037] In some implementations, each draw call is associated with an identifier (not shown), and the request stream for a series of work item dispatches includes the draw call identifier and the surface on which the draw call will operate. Pattern recognition unit 135 tracks the number of requests for draw call identifier / surface combinations excluded from L1 cache 120. Pattern recognition unit 135 lists draw call identifier / surface combinations based on the frequency of exclusion from L1 cache 120. In some implementations, cache controller 130 simulates the hit rate of a selected subset of cache lines 140 based on including or excluding draw call identifier / surface combinations from L1 cache 120 to determine which draw call identifier / surface combinations to exclude from L1 cache 120 to increase the hit rate.

[0038] In some implementations, cache controller 130 partitions L1 cache 120 into two or more portions (not shown), and assigns one portion of L1 cache 120 to store a subset of cache lines for a first data duplication pattern, and assigns another portion of L1 cache 120 to store a subset of cache lines for a second data duplication pattern. For each partition of L1 cache 120, cache controller 130 compares the reuse distance DR of the cache lines for the first data duplication pattern with the storage capacity of the cache portion to which the first data duplication pattern is assigned. Based on the reuse distance DR and the storage capacity of the cache portion, cache controller 130 selects a subset of cache lines for the first data duplication pattern to store in the cache and excludes the remaining cache lines for the first data duplication pattern from the cache.

[0039] Figure 2This is a diagram 200 illustrating a cache with a duplicate data pattern that selects a subset of cache lines without based on reuse distance, according to some implementation schemes. In the illustrated example, the size of L1 cache 120 is set to hold up to three cache lines at a time. After L1 cache 120 is flushed, at time T1, cache controller 130 receives a request for cache line -1 202 from compute unit 115. Because L1 cache 120 does not contain valid data, the request results in a miss at L1 cache 120, and cache controller 130 copies cache line -1 202 from a higher-level cache (L2 cache 220) in the cache hierarchy to L1 cache 120 and provides cache line -1 202 to compute unit 115. At a subsequent time T2, cache controller 130 receives a request for cache line -2 204 from compute unit 115. Because L1 cache 120 does not contain cache line -2 204, the request results in a cache miss at L1 cache 120, and cache controller 130 copies cache line -2 204 from L2 cache 220 to L1 cache 120 and provides cache line -2 204 to compute unit 115. At a subsequent time T3, the cache controller receives a request from compute unit 115 for cache line -3 206. Because L1 cache 120 does not contain cache line -3 206, the request results in a cache miss at L1 cache 120, and cache controller 130 copies cache line -3 206 from L2 cache 220 to L1 cache 120 and provides cache line -3 206 to compute unit 115.

[0040] At a subsequent time T4, cache controller 130 receives a request for cache line -4 208 from compute unit 115. Because L1 cache 120 does not contain cache line -4 208, the request results in a miss at L1 cache 120. To make room for cache line -4 208 at L1 cache 120, cache controller evicts cache line -1 202, as it is the least recently used cache line. Cache controller 130 copies cache line -4 208 from L2 cache 220 to L1 cache 120 and makes cache line -4 208 available to compute unit 115.

[0041] At a subsequent time T5, cache controller 130 receives a request for cache line -5 210 from compute unit 115. Because L1 cache 120 does not contain cache line -5 210, the request results in a cache miss at L1 cache 120. To make room for cache line -5 208 at L1 cache 120, cache controller evicts cache line -2 204, as it is the least recently used cache line. Cache controller 130 copies cache line -5 210 from L2 cache 220 to L1 cache 120 and makes cache line -5 210 available to compute unit 115. Therefore, the cache hit rate is 0% for the first five cycles T1-T5 of the access requests for cache lines 202, 204, 206, 208, and 210.

[0042] Figure 3 yes Figure 2 The diagram 300 illustrates the continuation of the duplicate data pattern cache. At a subsequent time T6, the cache controller 130 receives a request for cache line -1 202 from the compute unit 115. Because L1 cache 120 does not contain cache line -1 202 (since cache line -1 202 was previously evicted at time T4), the request results in a miss at L1 cache 120. To make room for cache line -1 202 at L1 cache 120, the cache controller 130 evicts cache line -3 206, as it is the least recently used cache line. The cache controller 130 copies cache line -1 202 from L2 cache 220 to L1 cache 120 and provides cache line -1 202 to compute unit 115.

[0043] At a subsequent time T7, cache controller 130 receives a request for cache line -2 204 from compute unit 115. Because L1 cache 120 does not contain cache line -2 204 (since cache line -2 202 was previously evicted at time T5), the request results in a miss at L1 cache 120. To make room for cache line -2 204 at L1 cache 120, cache controller 130 evicts cache line -4 208, as it is the least recently used cache line. Cache controller 130 copies cache line -2 204 from L2 cache 220 to L1 cache 120 and provides cache line -2 204 to compute unit 115.

[0044] At subsequent time T8, cache controller 130 receives a request for cache line -3 206 from compute unit 115. Because L1 cache 120 does not contain cache line -3 206 (since cache line -3 206 was previously evicted at time T6), the request results in a miss at L1 cache 120. To make room for cache line -3 206 at L1 cache 120, cache controller 130 evicts cache line -5 210, as it is the least recently used cache line. Cache controller 130 copies cache line -3 206 from L2 cache 220 to L1 cache 120 and makes cache line -3 206 available to compute unit 115.

[0045] At time T9, cache controller 130 receives a request for cache line -4 208 from compute unit 115. Because L1 cache 120 does not contain cache line -4 208 (since cache line -4 208 was previously evicted at time T7), the request results in a miss at L1 cache 120. To make room for cache line -4 208 at L1 cache 120, cache controller evicts cache line -1 202, as it is the least recently used cache line. Cache controller 130 copies cache line -4 208 from L2 cache 220 to L1 cache 120 and makes cache line -4 208 available to compute unit 115.

[0046] At subsequent time T10, cache controller 130 receives a request for cache line -5 210 from compute unit 115. Because L1 cache 120 does not contain cache line -4 208 (since cache line -5 210 was previously evicted at time T8), the request results in a miss at L1 cache 120. To make room for cache line -5 210 at L1 cache 120, cache controller evicts cache line -2 204, as it is the least recently used cache line. Cache controller 130 copies cache line -5 210 from L2 cache 220 to L1 cache 120 and provides cache line -5 210 to compute unit 115. Therefore, for the repeated 5 cache line data pattern of cache lines 202, 204, 206, 208, and 210, the cache using the LRU replacement policy results in a 0% hit rate for L1 cache 120 in cycles T6-T10, compared to... Figure 2 The 0% hit rate shown in the cold start scenario is no improvement.

[0047] Figure 4According to some implementation schemes, storage is allocated at L1 cache location 120 only for a subset 140 of cache lines representing duplicate data pattern 420. Figure 1 A block diagram 400 of the cache controller 130 is shown. In the illustrated example, the size of the L1 cache 120 is set to hold up to three cache lines at a time. The duplicate data pattern 420 includes five cache lines (1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, ...), thus exceeding the storage capacity of the L1 cache 120. The cache controller 130 employs a cache replacement policy 405 that evicts the least recently used cache line if the L1 cache 120 is full when fetching a cache line from a higher-level cache.

[0048] When retrieving data from GPU memory 410, pattern recognition unit 135 identifies duplicate data pattern 420 and determines that cache lines of duplicate data pattern 420 have a reuse distance exceeding the storage capacity of L1 cache 120. In response to pattern recognition unit 135 determining that cache lines of duplicate data pattern 420 have a reuse distance exceeding the storage capacity of L1 cache 120, cache controller 130 allocates storage only for a subset 140 of cache lines that can be stored in L1 cache 120 at once (e.g., cache lines 1, 2, and 3), and excludes the remaining 415 (cache lines 4 and 5) of cache lines of duplicate data pattern 420 from L1 cache 120. Therefore, when compute unit 115 requests each of cache lines 1, 2, and 3 of subset 140 of cache lines, cache controller allocates entries for cache lines 1, 2, and 3 and stores them at L1 cache 120. However, when the compute unit 115 requests the remaining portion 415 of the cache line (cache lines 4 and 5), the cache controller provides the remaining portion 415 of the cache line directly from the L2 cache 220 to the compute unit 115, bypassing the storage of the remaining portion 415 of the cache line in the L1 cache 120.

[0049] Figure 5This is a block diagram 500 of a cache controller 130 according to some embodiments, which partitions an L1 cache 120 and allocates storage only for a subset of cache lines for duplicate data patterns in portions of the cache. Sometimes, the parallel processing unit 110 handles concurrent access to more than one texture's draw call. In the illustrated example, the cache controller 130 retrieves two textures from GPU memory 410 for overlapping access: a first duplicate data pattern 520 and a second duplicate data pattern 530. To increase the hit rate of the first duplicate data pattern 520 and the second duplicate data pattern 530, the cache controller 130 partitions the L1 cache 120 into a first portion 540 and a second portion 545.

[0050] In response to pattern recognition unit 135 identifying a first duplicate data pattern 520 and determining that the reuse distance 550 of the cache lines of the first duplicate data pattern 520 exceeds the storage capacity 555 of the first portion 540 of the L1 cache, cache controller 130 allocates storage in the first portion 540 to store a subset 525 of cache lines of the first duplicate data pattern 520. Similarly, in response to pattern recognition unit 135 identifying a second duplicate data pattern 530 and determining that the reuse distance 560 of the cache lines of the second duplicate data pattern 530 exceeds the storage capacity 565 of the second portion 545 of the L1 cache, cache controller 130 allocates storage in the second portion 545 to store a subset 535 of cache lines of the second duplicate data pattern 530.

[0051] Figure 6 This is illustration 600, which illustrates the selective caching of cache lines based on repetitive data patterns according to some implementation schemes. Figure 2 Similar to Figure 200, the duplicate data pattern includes five cache lines, while the L1 cache 120 has a storage capacity of three cache lines. However, with Figure 2 Unlike Figure 200, in the illustrated example, cache controller 130 implements selective caching for duplicate data patterns. In response to pattern recognition unit 135 determining that cache lines of duplicate data patterns have a reuse distance exceeding the storage capacity of L1 cache 120, the cache controller allocates space in L1 cache 120 only for a subset of cache lines of duplicate data patterns (cache lines 202, 204, and 206) and excludes the remainder of cache lines of duplicate data patterns from L1 cache 120.

[0052] After L1 cache 120 is refreshed, at time T1, cache controller 130 receives a request for cache line -1 202 from compute unit 115. Because L1 cache 120 does not contain valid data, the request results in a cache miss at L1 cache 120. Since cache line -1 202 is included in the subset of cache lines for which cache controller 130 has already allocated storage in L1 cache 120, cache controller 130 copies cache line -1 202 from L2 cache 220 to L1 cache 120 and provides cache line -1 202 to compute unit 115.

[0053] At a subsequent time T2, cache controller 130 receives a request for cache line -2 204 from compute unit 115. Because L1 cache 120 does not contain cache line -2 204, the request results in a miss at L1 cache 120. Since cache controller 130 has included cache line -2 204 in the subset of cache lines for which storage has already been allocated in L1 cache 120, cache controller 130 copies cache line -2 204 from L2 cache 220 to L1 cache 120 and provides cache line -2 204 to compute unit 115.

[0054] At a subsequent time T3, the cache controller receives a request for cache line -3206 from compute unit 115. Since L1 cache 120 does not contain cache line -3206, the request results in a miss at L1 cache 120. Cache controller 130 has included cache line -3206 in the subset of cache lines for which storage has already been allocated in L1 cache 120; therefore, cache controller 130 copies cache line -3206 from L2 cache 220 to L1 cache 120 and provides cache line -3206 to compute unit 115.

[0055] At a subsequent time T4, cache controller 130 receives a request for cache line -4 208 from compute unit 115. L1 cache 120 does not contain cache line -4 208, therefore the request results in a miss at L1 cache 120. Because cache controller 130 has not yet included cache line -4 208 in the subset of cache lines that already have storage allocated for it at L1 cache 120 for the duplicate data pattern, the cache controller provides cache line -4 208 directly from L2 cache 220 to compute unit 115 and excludes cache line -4 208 from L1 cache 120.

[0056] At a subsequent time T5, cache controller 130 receives a request for cache line -5 210 from compute unit 115. L1 cache 120 does not contain cache line -5 210, therefore the request results in a miss at L1 cache 120. Because cache controller 130 has not yet included cache line -5 210 in the subset of cache lines that already have storage allocated for it at L1 cache 120 for the duplicate data pattern, the cache controller provides cache line -5 210 directly from L2 cache 220 to compute unit 115 and excludes cache line -5 210 from L1 cache 120. Therefore, with Figure 2 Similar to Figure 200, the cache hit rate is 0% in the first five cycles T1-T5 of the access requests to cache lines 202, 204, 206, 208, and 210.

[0057] Figure 7 yes Figure 6 The diagram 700 illustrates the continuation of selective caching for duplicate data patterns. At a subsequent time T6, cache controller 130 receives a request from compute unit 115 for cache line -1 202. Because L1 cache 120 contains cache line -1 202, the request results in a hit at L1 cache 120, and cache controller 130 provides cache line -1 202 to compute unit 115. At a subsequent time T7, cache controller 130 receives a request from compute unit 115 for cache line -2 204. Because L1 cache 120 contains cache line -2 204, the request results in a hit at L1 cache 120, and cache controller 130 provides cache line -2 204 to compute unit 115. At a subsequent time T8, cache controller 130 receives a request from compute unit 115 for cache line -3 206. Because L1 cache 120 contains cache line -3 206, the request results in a hit at L1 cache 120, and cache controller 130 provides cache line -3 206 to compute unit 115.

[0058] At a subsequent time T9, cache controller 130 receives a request for cache line -4 208 from compute unit 115. L1 cache 120 does not contain cache line -4 208 because cache controller 130 has excluded cache line -4 208 from L1 cache 120. Therefore, the request results in a miss at L1 cache 120. Cache controller 130 retrieves cache line -4 208 from L2 cache 220 and provides cache line -4 208 directly to compute unit 115, thereby bypassing the storage of cache line -4 208 at L1 cache 120, since cache line -4 208 is not included in the subset of cache lines for which space has already been allocated at L1 cache 120.

[0059] At a subsequent time T10, cache controller 130 receives a request for cache line -5 210 from compute unit 115. L1 cache 120 does not contain cache line -5 210 because cache controller 130 has excluded cache line -5 210 from L1 cache 120. Therefore, the request results in a miss at L1 cache 120. Cache controller 130 retrieves cache line -5 210 from L2 cache 220 and provides cache line -5 210 directly to compute unit 115, thereby bypassing the storage of cache line -5 210 at L1 cache 120, since cache line -5 210 is not included in the subset of cache lines for which space has already been allocated at L1 cache 120. Therefore, for the repeated 5-cached-line data pattern of cache lines 202, 204, 206, 208, and 210, selective caching using the LRU replacement policy results in a 60% hit rate for L1 cache 120 during cycles T6-T10, which is consistent with... Figure 3 The hit rate is significantly increased compared to the 0% hit rate in the conventional cache case shown. Furthermore, additional cycles of the same repetitive data patterns can be expected to result in a similarly increased hit rate for the L1 cache 120 using selective caching.

[0060] Figure 8 This is a flowchart illustrating a method 800 for selectively caching cache lines of repetitive data patterns, according to some embodiments. In some embodiments, method 800 is performed by a processing system (such as...) Figure 1 The processing system 100 shown executes this. At block 802, the pattern recognition unit 135 identifies the duplicate data pattern 420. At block 804, the pattern recognition unit 135 compares the reuse distance of the cache lines of the duplicate data pattern 420 with the maximum number of cache lines that the L1 cache 120 can store at one time, and determines whether the reuse distance exceeds the storage capacity of the L1 cache 120.

[0061] If the pattern recognition unit 135 determines at box 804 that the reuse distance of the cache lines of the duplicated data pattern 420 does not exceed the storage capacity of the L1 cache 120, the process continues to box 806. At box 806, the cache controller 130 caches all cache lines of the duplicated data pattern in the L1 cache 120.

[0062] If at block 804 the pattern recognition unit 135 determines that the reuse distance of the cache lines for duplicate data pattern 420 exceeds the storage capacity of L1 cache 120, the method flow continues to block 808. At block 808, cache controller 130 allocates storage at L1 cache 120 only for a subset 140 of cache lines for duplicate data pattern 420, and excludes the remaining 415 of cache lines for duplicate data pattern 420 from L1 cache 120. In some embodiments, pattern recognition unit 135 randomly selects a subset of cache lines from duplicate data patterns during iteration to store at L1 cache 120. Cache controller 130 measures the hit rate of each selected subset of cache lines and selects a subset 140 of cache lines for duplicate data pattern 420 to exclude from L1 cache 120 based on the hit rate.

[0063] In some implementations, cache controller 130 partitions L1 cache 120 into a first portion 540 and a second portion 545. In response to pattern recognition unit 135 identifying a first duplicate data pattern 520 and determining that the reuse distance of cache lines for the first duplicate data pattern 520 exceeds the storage capacity of the first portion 540 of the L1 cache, cache controller 130 allocates storage in the first portion 540 to store a subset 525 of cache lines for the first duplicate data pattern 520.

[0064] In some implementations, the above-described apparatus and techniques are implemented in systems including one or more integrated circuit (IC) devices (also known as integrated circuit packages or microchips), such as those referenced above. Figures 1 to 8The described processing system. Electronic design automation (EDA) and computer-aided design (CAD) software tools can be used in the design and manufacture of these IC devices. These design tools are typically represented as one or more software programs. One or more software programs include code executable by a computer system to manipulate the computer system to operate on code representing a circuit of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system to manufacture the circuit. The code may include instructions, data, or a combination of instructions and data. Software instructions representing design or manufacturing tools are typically stored in a computer-readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of an IC device may be stored in or accessed from the same computer-readable storage medium or different computer-readable storage media.

[0065] Computer-readable storage media can include any non-transitory storage medium or a combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable storage media can be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or a flash memory based on a universal serial bus (USB)), or coupled to a computer system via a wired or wireless network (e.g., a network accessible storage device (NAS)).

[0066] In some implementations, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing the software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, disk or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0067] It should be noted that not all activities or elements described above in the general description are essential. A particular activity or part of the apparatus may not be essential, and one or more additional activities may be performed, or elements may be included in addition to those described. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Additionally, these concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes may be made without departing from the scope of this disclosure as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this disclosure.

[0068] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, the benefits, advantages, solutions to problems, and any features that may lead to or make any benefit, advantage, or solution appear or become more significant should not be construed as key, essential, or fundamental features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and practiced in different but equivalent ways that will be apparent to those skilled in the art who benefit from the teachings herein. No limitation is intended on the details of the constructions or designs shown herein, except as described in the following claims. Therefore, it will be apparent that the specific embodiments disclosed above can be altered or modified, and all such changes are considered to be within the scope of the disclosed subject matter. Therefore, the protection sought herein is set forth in the following claims.

Claims

1. A method, the method comprising: In response to a duplicate data pattern, wherein a single instance of the duplicate data pattern has a number of cache lines exceeding the storage capacity of the cache, wherein the duplicate data pattern will be stored in the cache, and storage will be allocated at the cache only for a subset of the cache lines of the single instance.

2. The method according to claim 1, further comprising: Bypassing the storage of the remaining portion of the cache line for the single instance at the cache location.

3. The method of claim 2, wherein allocating storage only for the subset comprises: Select the cache lines in the duplicate data pattern that exceed the maximum number of cache lines that the cache can store as the remaining portion.

4. The method according to claim 1, comprising: The reuse distance of the cache lines of the individual instance is determined to be greater than the number of cache lines that can be stored in the cache. as well as The subset is restricted to the number of cache lines that can be stored in the cache.

5. The method according to claim 4, further comprising: The cache lines representing the duplicate data pattern are iteratively selected to be excluded from the cache in the following manner: Determine the hit rate of the subset of cache lines for the duplicate data pattern that was not excluded from the cache; as well as The cache lines of the duplicate data pattern are selected based on the hit rate to be excluded from the cache.

6. The method according to claim 4, further comprising: The cache is partitioned into two or more parts; as well as Assigning the first portion of the cache to the duplicate data pattern, and wherein determining that the reuse distance of the cache lines of the individual instance is greater than the number of cache lines that can be stored in the cache includes determining that the reuse distance of the cache lines of the individual instance is greater than the number of cache lines that can be stored in the first portion of the cache.

7. The method of claim 1, wherein the repeating data pattern includes a texture.

8. A method, the method comprising: The number of cache lines in a single instance that determines a duplicate data pattern exceeds the storage capacity of the cache, and the duplicate data pattern will be stored in the cache line of the cache. and In response to the determination, cache lines of the cache are allocated only to a subset of cache lines of the single instance.

9. The method according to claim 8, further comprising: Bypassing the storage of the remaining portion of the cache line for the single instance at the cache location.

10. The method of claim 9, wherein selecting the subset comprises: Select the cache lines in the duplicate data pattern that exceed the maximum number of cache lines that the cache can store as the remaining portion.

11. The method according to claim 8, further comprising: The reuse distance of the cache lines of the individual instance is determined to be greater than the number of cache lines that can be stored in the cache. as well as The subset is restricted to the number of cache lines that can be stored in the cache.

12. The method according to claim 11, further comprising: The cache lines representing the duplicate data pattern are iteratively selected to be excluded from the cache in the following manner: Determine the hit rate of cache lines containing the duplicate data pattern that has not been excluded from the cache; and The cache lines of the duplicate data pattern are selected based on the hit rate to be excluded from the cache.

13. The method according to claim 11, further comprising: The cache is partitioned into two or more parts; as well as The first portion of the cache is assigned to the duplicate data pattern; Determining that the reuse distance of a cache line of a single instance is greater than the number of cache lines that can be stored in the cache includes determining that the reuse distance of a cache line of a single instance is greater than the number of cache lines that can be stored in the first portion of the cache.

14. The method of claim 8, wherein the repeating data pattern includes a texture.

15. An apparatus, the apparatus comprising: A parallel processing unit, the parallel processing unit including a cache in which duplicate data patterns are stored; as well as A cache controller, wherein the cache controller is used for: In response to a single instance of the duplicate data pattern having a number of cache lines exceeding the storage capacity of the cache, storage is allocated at the cache only for a subset of the cache lines of that single instance.

16. The device of claim 15, wherein the cache controller is configured to: Bypassing the storage of the remaining portion of the cache line of the duplicate data pattern at the cache location.

17. The device of claim 16, wherein the cache controller is configured to: Select the cache lines in the duplicate data pattern that exceed the maximum number of cache lines that the cache can store as the remaining portion.

18. The device of claim 15, wherein the cache controller is configured to: Determine whether the reuse distance of a cache line in a single instance is greater than the number of cache lines that can be stored in the cache; and The subset is restricted to the number of cache lines that can be stored in the cache.

19. The device of claim 18, wherein the cache controller is configured to: Determine the hit rate of cache lines containing the duplicate data pattern that has not been excluded from the cache; and The cache lines of the duplicate data pattern are selected based on the hit rate to be excluded from the cache.

20. The device of claim 18, wherein the cache controller is configured to: The cache is partitioned into two or more parts; The first portion of the cache is assigned to the duplicate data pattern; and the reuse distance of the cache lines of the individual instance is determined to be greater than the number of cache lines that can be stored in the first portion of the cache by determining that the reuse distance of the cache lines of the individual instance is greater than the number of cache lines that can be stored in the cache.

Images