Cache for the area where data is stored

Abstract

Disclosed are systems, devices, and methods for efficiently performing memory accesses in a computing system. The computing system includes one or more clients, a communication fabric, and a last-level cache implemented using low-latency, high-bandwidth memory. A cache controller for the last-level cache determines an address range corresponding to a first region of system memory, the first region having a copy of data stored in a second region of the last-level cache. When the cache controller determines that a request address of a memory access request is not within the address range, the cache controller sends a selected memory access request to the system memory. When the cache controller determines that the request address is within the address range, the cache controller services the selected memory request by accessing data from the last-level cache.

Description

Background Art

[0001] Description of Related Technology

[0002] As semiconductor manufacturing processes advance and die geometries decrease, semiconductor chips offer increased functionality and performance. However, modern techniques in processing and integrated circuit design still present design challenges that limit potential benefits. One issue is the increasing interconnect delay per unit length in successive generations of two-dimensional planar layout chips. Furthermore, the high electrical impedance between individual chips increases latency. Furthermore, signals that traverse off-chip to another die increase their power consumption due to the increased parasitic capacitance on these longer signal paths.

[0003] Another design issue is that most software applications that access large amounts of data are typically memory-bound, as computation time is generally determined by memory bandwidth. Memory access latency for off-chip dynamic random access memory (DRAM) is hundreds of clock cycles to over a thousand, and the increasing number of cores in processor designs has exacerbated the memory bandwidth problem. In recent years, advances have been made in memory technology for implementing in-package memory, which provides access to large, low-latency, high-bandwidth memories before accessing off-package DRAM and main memory.

[0004] One example of such memory technology is a three-dimensional integrated circuit (3D IC), which is used to include two or more layers of active electronic components integrated vertically and horizontally into a single circuit. 3D packaging, also known as a system-in-package (SiP) or a die-stacked multi-chip module (MCM), saves space by stacking separate chips in a single package. Components within these layers communicate vertically or horizontally using on-chip signaling. This signaling provides reduced interconnect signal delay compared to known two-dimensional planar layout circuits.

[0005] The manufacturing trends described above have led to gigabytes of integrated memory within a single package. In some cases, computing systems use additional on-chip storage as a last-level cache before accessing off-chip memory. The reduced miss rate achieved by the additional memory helps hide the latency gap between the processor and its off-chip memory. However, the cache access mechanism of row-based memory is inefficient for this additional integrated memory. Large tagged data arrays, such as hundreds of megabytes of a multi-gigabyte cache, are expensive to place on the microprocessor die and provide high latency for looking up the large tag arrays. Because the tags and data are read out in a serial manner, the lookup and data retrieval consumes too much time.

[0006] Increasing the size of the data cache lines of the additional integrated memory, such as from a 64-byte line to a 4-kilobyte (KB) line, reduces the number of cache lines in the integrated memory and the corresponding tag size. However, dirty bits and coherence information are still maintained at the granularity of the original cache line size (64-byte line). Thus, on-package DRAM provides a large amount of additional data storage, but the cache and DRAM access mechanisms are inefficient.

[0007] In view of the foregoing, there exists a need for efficient methods and systems for efficiently performing memory accesses in computing systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0009] Figure 1 is a block diagram of one embodiment of a data store.

[0010] Figure 2 is a flow chart of one embodiment of a method for performing efficient memory access in a computing system.

[0011] Figure 3 is a block diagram of one embodiment of a computing system.

[0012] Figure 4 is a block diagram of one embodiment of a system in a package (SiP).

[0013] Figure 5 is a block diagram of one embodiment of a data store.

[0014] Figure 6 is a block diagram of one embodiment of a data store.

[0015] Figure 7 is a block diagram of one embodiment of a data store.

[0016] Figure 8 is a block diagram of one embodiment of a data store.

[0017] Figure 9 is a block diagram of one embodiment of a data store.

[0018] Figure 10 is a flow chart of one embodiment of a method for performing efficient memory access in a computing system.

[0019] Figure 11 is a block diagram of one embodiment of a data store.

[0020] Figure 12 is a block diagram of one embodiment of a data store.

[0021] Figure 13 is a block diagram of one embodiment of a data store.

[0022] Figure 14 is a flow chart of one embodiment of a method for performing efficient memory access in a computing system.

[0023] While the invention is susceptible to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and are herein described in detail. However, it should be understood that the drawings and detailed description thereof are not intended to limit the invention to the specific forms disclosed, but on the contrary, the invention is intended to cover all modifications, equivalents, and alternatives falling within the scope of the invention as defined by the appended claims. DETAILED DESCRIPTION

[0024] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, one of ordinary skill in the art will recognize that various embodiments may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid obscuring the methods described herein. It will be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, some dimensions of an element may be exaggerated relative to other elements.

[0025] Various systems, devices, methods, and computer-readable media for efficiently performing memory access in a computing system are disclosed. One or more clients in the computing system process applications. Examples of such clients include general-purpose central processing units (CPUs), graphics processing units (GPUs), accelerated processing units (APUs), input / output (I / O) devices, and the like. The computing system also includes multiple link interfaces for transferring data between the clients. In addition, each of the one or more clients accesses data from a last-level cache via a communication fabric.

[0026] In various embodiments, a low-latency, high-bandwidth memory separate from the system memory is used to implement the cache. In some embodiments, the cache is used as a last-level cache in the cache subsystem. In other embodiments, the cache is another level within the cache subsystem. The system memory includes a variety of off-package dynamic random access memories (DRAM) and one of the main memories such as hard disk drives (HDDs) and solid-state drives (SSDs). In some embodiments, the computing system uses an integrated DRAM (such as a three-dimensional (3D) DRAM) included in a system in package (SiP) together with a processing unit of one of the clients to implement the cache. In other embodiments, the computing system includes one of the other memory technologies for implementing the cache, such as synchronous RAM (SRAM), embedded DRAM (eDRAM), flash memory such as a solid-state drive, and one of the various non-volatile memories. Examples of non-volatile memories are phase change memory, memristors, and spin-transfer torque (STT) magnetoresistive random access memories (MRAM).

[0027] A cache controller for caching includes one or more queues. Each queue stores memory access requests of a corresponding type. For example, in some designs, a first queue stores memory read requests and a second queue stores memory write requests. Logic within the cache controller selects a queue from the one or more queues and selects a memory access request from the selected queue. The logic determines an address range corresponding to a first area of ​​contiguous data stored in system memory, the first area having a copy of the contiguous data stored in a second area of ​​cache. As used herein, "contiguous data" refers to one or more bits of data that are located next to each other in a data storage. In some embodiments, the size of the contiguous data ranges between the size of a cache line (e.g., 64 bytes) and the size of a page (e.g., 4 kilobytes) to provide a size granularity corresponding to the area of ​​predicted upcoming data access by the software application being executed. In other embodiments, contiguous data of another size is used.

[0028] At an earlier point in time, when the logic determines that the region of the predicted upcoming data access is defined, the logic stores a copy of contiguous data from this region (in this example, a first region of system memory) in a second region of the cache. The contiguous data in the first region includes data corresponding to the predicted upcoming data access. The logic also initializes a plurality of parameters characterizing the region. For example, the logic maintains a first starting address that points to a memory location at the beginning of the system memory where data is stored. Additionally, the logic maintains a second starting address that points to a memory location at the beginning of the second region of the cache where data is stored. Furthermore, the logic maintains a size of the second region.

[0029] In one embodiment, the logic predicts an area of ​​system memory to which an upcoming memory access will access by monitoring received memory access requests and identifying patterns. The logic identifies this area. In response, the logic performs the above steps, such as storing a copy of the contiguous data from this area and updating corresponding parameters. In another embodiment, the logic receives one or more hints from software that identify or are used to identify the area of ​​the predicted upcoming data access.

[0030] When the logic detects a change in the size of the second area, the logic determines an address range that starts at the first starting address and ends at an address that is the sum of the first starting address and the new size of the second area. In some embodiments, when data is updated in the second area, an update to one or more of the first starting address and the size of the second area occurs. The update to the second area includes one or more of the following: adding data, removing data, and overwriting existing data in the second area. When the logic selects a memory access request from one of the multiple queues, the logic of the cache controller compares the request address of the selected memory access request with the address range. The logic determines whether the request address is within this range. Therefore, to determine whether there is a cache hit or a cache miss in the last level cache, the logic compares the request address with this maintained address range instead of performing a set associative lookup or a fully associative lookup of the tag array in the cache. The comparison is a faster operation than the cache index and tag lookup operation.

[0031] When the logic determines that the request address of the selected memory access request is not within the address range, the logic sends the selected memory access request to the system memory for servicing. However, when the logic determines that the request address is within the address range, the logic services the memory access request by accessing data from the cache. To do so, the logic determines an offset based on the difference between the request address and the first starting address. Thereafter, the logic determines a translated address based on the offset and the second starting address. The logic then services the memory access request by accessing data from the cache starting at the translated address.

[0032] See Figure 1, shows a general block diagram of one embodiment of data storage 100. As shown, system memory 110 and last-level cache 130 each store data. Although the description describes cache 130 as a last-level cache, in other embodiments, cache 130 is another level within a cache subsystem. For simplicity of illustration, processing units, communication interfaces, etc. are not shown. Data 126 is contiguous data stored in region 120 of system memory 110. Last-level cache 130 stores a copy of contiguous data 126 from region 120 in region 140. Region parameters 150 characterize regions 120 and 140.

[0033] In various designs, system memory 110 includes one or more of off-package DRAM, a hard disk drive (HDD), and a solid-state drive (SSD). In some designs, final-level cache 130 includes on-package low-latency, high-bandwidth memory separate from system memory 110. In some designs, final-level cache 130 includes 3D DRAM. In other designs, final-level cache 130 includes synchronous RAM (SRAM), embedded DRAM (eDRAM), flash memory (such as a solid-state drive), and one of a variety of non-volatile memories. Examples of non-volatile memories are phase change memory, memristors, and spin-transfer torque (STT) magnetoresistive random access memory (MRAM).

[0034] In the illustrated embodiment, address 122 (also referred to as "x") points to a memory location at the beginning of region 120 where data is stored. Here, the generic value "x" is any value represented in any manner, such as an integer, hexadecimal, etc. Region 120 has a size 124, also referred to as "S bytes." In a similar manner, address 142 (also referred to as "a") points to a memory location at the beginning of region 140 where data is stored. Region 140 has a size 144, also referred to as "S bytes," and is equal to size 124 of region 120. The values ​​"x," "S," and "a" are positive integers.

[0035] In some embodiments, a contiguous element in a cache controller for last level cache 130 stores region parameters 150. Examples of contiguous elements are registers, flip-flop circuits, and latches. In one embodiment, region parameters 150 include state information 152, such as a valid bit and metadata. Examples of metadata are an identifier of the producer of data 126, an identifier of the consumer of data 126, cache coherency information for data 126, clean / dirty information for data 126, and the like. The identifiers of the producer and consumer include one or more of a processing unit identifier, a process identifier, and a thread identifier. In other embodiments, region parameters 150 do not include state information 152, as this information is stored in other queues and contiguous elements of the cache controller.

[0036] In one embodiment, region parameters 150 include two addresses. First address 154 is a copy of address 122 and points to the memory location where data is stored at the start of region 120. Second address 156 is a copy of address 142 and points to the memory location where data is stored at the start of region 140. Thus, region parameters 150 include a memory mapping between the start of region 120 and the start of region 140. For example, region parameters 150 currently stores a memory mapping between address 122 ("x") and address 142 ("a"). In some embodiments, region parameters 150 also include a size 158 of region 140. In one embodiment, logic in the cache controller uses a size value of zero bytes in size field 158 to indicate that no valid region is stored in the last level cache, rather than using the valid bit in status field 152 to indicate this.

[0037] Using region parameter 150, logic in the cache controller uses a comparison operation, which is faster than a lookup operation in the large tag array, to determine a cache hit or cache miss in the last-level cache 130. In one example, the logic determines whether a valid region is stored in the last-level cache 130. If status field 152 is used and the valid bit is cleared, no valid region is stored in the last-level cache 130. If status field 152 is not used and size field 158 stores a value of zero bytes, no valid region is stored in the last-level cache 130. In such a case, the logic in the cache controller determines that there is a cache miss and sends a memory access request with the request address to system memory 110 for servicing. As a result, the logic skips performing a set-associative lookup operation in a set of large tag arrays selected by the index of the request address, which reduces the latency of processing the memory access request.

[0038] If the status field 152 is used and the valid bit is asserted, or if the status field 152 is not used but the size field 158 stores a positive non-zero integer, the logic of the cache controller determines that there is a valid region stored in the last-level cache 130. In this case, when the logic in the cache controller determines a change in one or more of the address 122 (“x”) and the size 158 (“S”) of the region 140, the logic determines the address range. The logic determines the address range to start at the address 122 (“x”) and end at the address that is the sum of the address 122 and the size 158 (“S”) of the region 140. Using the symbols in the illustrated embodiment, the address range is “x + S”. The logic determines whether the requested address is within this range. For example, if the requested address is represented as “y”, the logic determines whether the expression x < y < (x + S) is true. Thus, to determine whether there is a cache hit or a cache miss within the last-level cache 130, the logic compares the requested address with this address range. The comparison operation is a faster operation than the index and tag look-up operations of the last-level cache 130.

[0039] If the logic determines that the access to the last-level cache 130 is a cache miss, the cache controller sends a memory access request with the requested address to the system memory 110 for servicing. However, if the logic determines that the access to the last-level cache 130 is a cache hit, the logic services the memory access request by retrieving data from the last-level cache 130. To do this, the logic determines an offset based on the difference between the requested address (“y”) and the address 122 (“x”), which is expressed as (y – x). The logic determines the translated address based on the offset (y – x) and the address 142 (“a”), which is the sum of two values and is expressed as (a + (y – x)). The logic services the memory access request by accessing data from the last-level cache 130 starting at the translated address or the address represented by (a + (y – x)). The logic skips performing a set-associative look-up operation in a set of large tag arrays selected by the index of the requested address. Instead, after the comparison operation for determining a cache hit, simple arithmetic operations are used to identify the location where the requested data is stored in the last-level cache 130.

[0040] Now referring to Figure 2 , an embodiment of a method 200 for efficiently performing memory access in a computing system is shown. For purposes of discussion, it is shown in this embodiment (and in Figure 10 and Figure 14). However, it should be noted that in various embodiments of the described methods, one or more of the described elements are performed simultaneously, in an order different from the order shown, or omitted entirely. Other additional elements are also performed as needed. Any of the various systems or devices described herein can be configured to implement methods 200, 1000, and 1400.

[0041] One or more processing units execute one or more computer programs or software applications. Examples of processing units are processor cores within a general-purpose central processing unit (CPU), a graphics processing unit (GPU), or others. In some embodiments, a system-in-package (SiP) includes a processing unit and low-latency, high-bandwidth memory on the package. An example of such memory is 3D integrated memory, such as 3D DRAM. In one embodiment, the processing unit utilizes at least a portion of the 3D DRAM as a cache. In one embodiment, the cache is a last-level cache. Although the following description describes low-latency, high-bandwidth memory as a last-level cache, in other embodiments, high-bandwidth memory is used as a level 1 (L1), level 2 (L2), or other level in the cache hierarchy other than the last level. The processing unit determines a memory request miss within the cache subsystem at a level below the last-level cache (block 202).

[0042] The processing unit sends a request address corresponding to the memory request to the last level cache (block 204). In one embodiment, logic in a cache controller for the last level cache maintains an identification of a first region of contiguous data in system memory, the first region having a copy of the contiguous data stored in a second region of the last level cache. In some embodiments, the identification includes a first starting address that identifies a starting point of the first region. Additionally, the identification includes a size of the second region. The logic in the cache controller for the last level cache determines an address range for this first region, the address range being a range of addresses within the system memory address space that point to memory locations where the contiguous data stored in the system memory is stored (block 206). This contiguous data has a copy stored in the last level cache. In some designs, the logic uses the address range described earlier in ( Figure 1 ) The expression described in the description of data storage 100.

[0043] If the request address is not within the range (the "no" branch of conditional block 208), the logic sends a memory request including the request address to the system memory (block 210). The access to the last-level cache for the memory request is considered a cache miss, and accordingly, the memory request is sent to a lower level of the memory subsystem (such as system memory). If the request address is within the selected range (the "yes" branch of conditional block 208), the logic determines an offset based on the difference between the request address and the start address of the range in the system memory (block 212). The logic determines a translated address based on the offset and the start address of the range in the last-level cache (block 214). For example, the translated address is the sum of the offset and the start address of the range in the last-level cache. The logic services the memory request by accessing data from the last-level cache starting at the translated address (block 216).

[0044] Now turn Figure 3 , shows a general block diagram of one embodiment of a computing system 300 utilizing a low-latency, high-bandwidth cache. In various embodiments, computing system 300 utilizes three-dimensional (3D) packaging, such as a system-in-package (SiP) 310. SiP 310 connects to memory 362 and off-package DRAM 370 via a memory bus 350. In one embodiment, computing system 300 is a standalone system within: a mobile computer, smartphone, or tablet computer; a desktop computer; a server; or other. SiP 310 utilizes a processing unit 320 and a low-latency, high-bandwidth cache 330. Processing unit 310 and cache 330 communicate via a low-latency interconnect 348. The on-package low-latency interconnect 348 utilizes one or more of horizontal and / or vertical routes that are shorter than the long off-chip interconnects that would be used without a SiP.

[0045] While in some embodiments, SiP 310 utilizes DRAM memory technology, such as 3D DRAM, it is possible and contemplated to use other memory technologies with low latency, high bandwidth, and row-based access schemes including one or more row buffers or other equivalent structures. Examples of other memory technologies include phase change memory, spin torque transfer resistive memory, memristors, embedded DRAM (eDRAM), and the like. In some designs, processing unit 320 is a general-purpose microprocessor, while in other designs, processing unit 320 is another type of processing unit. Other types of processing units include graphics processing units (GPUs), field programmable gate arrays (FPGAs), and accelerated processing units (APUs), which are chips that include additional processing capabilities. This additional processing capability accelerates one or more types of computations beyond those of a general-purpose CPU. In one embodiment, the APU includes a general-purpose CPU integrated on the same die as the GPU, FPGA, or other processing unit, thereby increasing the data transfer rate between these units while reducing power consumption. In other embodiments, the APU includes video processing and other specialized accelerators.

[0046] The execution engine 322 utilizes one or more processor cores based on the type of processing unit 320. Additionally, in some designs, the execution engine 322 utilizes a communication fabric (or "fabric") for passing communication messages. Examples of communication messages are coherence probes, interrupts, and read and write access commands and corresponding data. Examples of interconnects in the fabric are bus architectures, crossbar-based architectures, network-on-chip (NoC) communication subsystems, inter-die communication channels, silicon interposers, and through-silicon vias (TSVs). In many designs, the processing unit 320 incorporates a system bus controller in the interface logic 326 that connects the processor core of the execution engine 322 to memory 362, DRAM 370, peripheral input / output (I / O) devices, and other processing units using one of a variety of protocols.

[0047] Computing system 300 uses off-package memory 362 as main memory or system memory. Memory 362 is one of a hard disk drive (HDD) and a solid-state drive (SSD). Off-package DRAM 370 is one of several types of DRAM. Computing system 300 populates off-chip DRAM 370 with data from off-package memory 362 via I / O controller and bus 360 and memory bus 350. Interface logic 360 supports communication protocols, address formats, and packet formats for each of off-package memory 362 and off-package DRAM 370.

[0048] Each of the processor cores within the execution engine 322 uses one or more levels of the cache subsystem to reduce memory latency for the processor core. In some designs, the processor cores additionally access a shared cache within the execution engine 322. When the cache subsystem within the execution engine 322 does not include the data requested by the processor core, the execution engine 322 sends a memory access request to the in-package cache 330. Interface logic 340 supports the communication protocol, address format, and packet format for transferring information between the in-package cache 330 and the processing unit 320.

[0049] Similar to other DRAM topologies, in some designs, the in-package cache 330 uses multiple memory arrays 332 that are partitioned into multiple groups. In these cases, each of the groups includes a corresponding row buffer. Each of the row buffers stores data from accessed rows of multiple rows within the corresponding memory array group. In some embodiments, the functionality of the queue 342, the region parameters 344, and part of the logic 346 that uses the region parameters 344 is located in the logic 336. For example, this functionality is included in the cache controller for the in-package cache 330. In other embodiments, this functionality is located in the interface logic 340 shown. Each of the logic 336 and the logic 346 is implemented by software, hardware such as circuits for combinational logic and serial elements, or a combination of software and hardware.

[0050] When the interface logic 340 receives a memory access request from the execution engine 322, the logic 346 stores the received memory access request in one of the plurality of queues 342 based on the access type. For example, a first queue in the plurality of queues 342 stores memory read requests, and a second queue in the plurality of queues 342 stores memory write requests. Arbitration logic within the logic 346 selects a queue in the plurality of queues 342 and selects a memory access request from the selected queue. For the selected memory access request, the logic 346 determines an address range corresponding to a first region of system memory (such as region 372), where a copy of the data is stored in a second region of the in-package cache 330 (such as region 338). The system memory is implemented by a combination of off-package memory 362 and off-package DRAM 370.

[0051] When the logic 346 determines that the request address of the memory access request is not within the address range of the region 372, the logic 346 sends the selected memory access request to the system memory. When the logic 346 determines that the request address is within the address range of the region 372, the cache controller services the selected memory request by accessing data from the memory array 332 within the in-package cache 330. The logic 346 uses the region parameter 344 for the above determination. In various embodiments, the region parameter 344 is equivalent to ( Figure 1) area parameter 150.

[0052] Logic 346 uses one of a variety of techniques to determine when to store a copy of region 372 in off-package system memory as region 338 in in-package cache 330. In some embodiments, logic 346 monitors memory accesses from execution engine 322 to detect streaming or continuous memory access patterns. Logic 346 uses one of a variety of techniques to detect streaming access patterns, such as those used by at least a hardware prefetcher. When logic 346 detects streaming pattern, logic 346 defines a new region. In some embodiments, when a memory request accesses an address within L bytes of the end of region 338, one of logic 336 and logic 346 expands the size of region 338 by P bytes, where L and P are positive, non-zero integers. In one embodiment, the values ​​of L and P are stored in programmable registers in control and status register (CSR) 347. In some embodiments, the initial region size is also stored in a programmable register in CSR 347.

[0053] In other embodiments, logic 346 uses software hints to determine when to define and create region 338 in the in-package cache 330. The software uses specific instructions to update certain registers accessed by the application or operating system. In addition, the software can update one or more control and status registers (CSRs) 347 in the interface logic 340. When processing a deep neural network, the software application knows when it has finished processing a layer of the multi-layer neural network and when to move to the next layer of the multi-layer network. As each layer of the multi-layer network is traversed (whether forward or backward), the software application uses software hints to inform one or more of logic 346 and CSRs 347 of the current region in system memory being processed. In some embodiments, the software application provides hints to indicate when to increase or decrease the size of regions 372 and 338. The hints also indicate changes in the size from the left or right of regions 372 and 338.

[0054] In addition, the software hint indicates that the entire contents of regions 372 and 338 are to be changed by moving them to another region in system memory. The limits stored in CSR 347 prevent region 338 from exceeding the size of in-package cache 330. In some embodiments, if logic 346 has already defined regions, logic 346 selects between supporting existing regions and new regions based on criteria. Examples of such criteria are the size of the region, the priority level of the application accessing the data in the region, the age of the existing region, etc.

[0055] During execution of one or more software applications, the applications modify the contents of region 338. If logic 346 adjusts the size or portion of region 338 such that the modified portion of region 338 is no longer valid, one of logic 336 and logic 346 sends the modified data to off-package DRAM 370. In some designs, one of logic 336 and logic 346 controls in-package cache 330 using a write-through cache policy or a write-back cache policy. The write-through cache policy propagates write operations to off-package DRAM 370 over time. In contrast, the write-back cache policy delays write operations until the size of region 338 is reduced. At this time, logic 336 or logic 346 sends write operations for the modified data to off-package DRAM 370 in a burst of write traffic. In other designs, one of logic 336 and logic 346 controls in-package cache 330 using a combination of write-through and write-back cache policies to balance the benefits and costs of the two policies.

[0056] As described earlier, in some designs, the in-package cache 330 uses low-latency, high-bandwidth memory technology such as SRAM, phase-change memory, spin-torque transfer resistive memory, memristors, embedded DRAM (eDRAM), etc. In other designs, the in-package cache 330 uses low-latency, high-bandwidth 3D DRAM. Turning now to Figure 4 , a generalized block diagram of an embodiment of a system-in-package (SiP) 400 and 440 is shown. The illustrated SiP includes one or more three-dimensional integrated circuits (3D ICs). A 3D IC includes two or more layers of active electronic components integrated vertically and / or horizontally into a single circuit. In some designs, manufacturing techniques use interposer-based integration, whereby the manufacturing techniques place the 3D IC next to the processing unit 420. Alternatively, the manufacturing techniques stack the 3D IC directly on top of another IC.

[0057] Die stacking technology is a manufacturing process that enables the use of high-bandwidth and low-latency interconnects to physically stack multiple pieces of silicon (integrated chips) together in the same package. The dies are stacked side by side on a silicon interposer, or directly stacked vertically on top of each other. One configuration of SiP is to stack one or more DRAM chips next to and / or on top of a processing unit. The stacked DRAM chips provide a very large cache for the processing unit. In some designs, this large cache has a size of about hundreds of MB (or more).

[0058] As shown, in one embodiment, SiP 400 includes a processing unit 420 and one or more three-dimensional (3D) DRAMs 430 and 432 that communicate with the processing unit 420 via a horizontal, low-latency interconnect 410. Again, the processing unit 420 is one of a general-purpose CPU, a graphics processing unit (GPU), an accelerated processing unit (APU), a field-programmable gate array (FPGA), or other data processing device that utilizes row-based memory, such as a cache.

[0059] The intra-package low-latency interconnect 410 provides a reduced length for interconnect signals relative to the long off-chip interconnects that would be required without a SiP. The intra-package low-latency interconnect 410 uses specific signals and protocols as if the chips, such as the processing unit 420 and the 3D DRAMs 430 and 432, were mounted in separate packages on a circuit board. The SiP 400 also includes backside vias or through-silicon vias 412 that access package-external connections 414. The package-external connections 414 are used for input / output (I / O) signals and power signals.

[0060] In another embodiment, SiP 440 includes 3D DRAM 450 stacked directly on top of processing unit 420. Although not shown, for each of SiP 400 and SiP 440, multiple chips or device layers are stacked on top of each other with direct vertical interconnects 416 tunneling through them. The size and density of vertical interconnects 416 that can be tunneled between different device layers varies based on the underlying technology used to manufacture the 3D IC.

[0061] See Figure 5 , shows a general block diagram of one embodiment of data storage 500. Previously described circuits and logic are numbered identically. As shown, each of system memory 110 and last-level cache 130 stores data. Similarly, while the description describes cache 330 as a last-level cache, in other embodiments, cache 330 is another level within the cache subsystem. Data 126 is contiguous data stored in system memory 110. Data 526 is contiguous data added to a region in system memory 110 to create region 520. The size of the contiguous data in system memory 110 that is copied as a region increases from size 124 to size 524.

[0062] Last level cache 130 stores a copy of contiguous data 126 and data 526 for region 520 in region 540. Consequently, the size of the contiguous data maintained as a region in last level cache 130 increases from size 144 to size 544. Region parameters 150 characterize regions 520 and 540. Starting addresses 122 and 142 remain the same. Therefore, fields 154 and 156 remain unchanged in region parameters 150. However, the logic updates size field 158 to the increased amount. In this example, the logic updates size field 158 from S bytes to S+T bytes, where S and T are positive, non-zero integers.

[0063] Now turn Figure 6 , a general block diagram of one embodiment of data storage 600 is shown. Previously described circuits and logic are numbered identically. Data 126 is contiguous data stored in system memory 110. Data 626 is contiguous data added to the region in system memory 110 to create region 620. The size of the contiguous data in system memory 110 that is copied as a region increases from size 124 to size 624.

[0064] Last level cache 130 stores a copy of contiguous data 126 and data 626 for region 620 in region 640. Consequently, the size of the contiguous data maintained as a region in last level cache 130 increases from size 144 to the sum of size 644 and size 646. The contiguous data wraps around last level cache 130. Region parameters 150 characterize regions 620 and 640. Starting addresses 122 and 142 remain the same. Therefore, fields 154 and 156 remain unchanged in region parameters 150. However, the logic updates size field 158 to the increased amount. In this example, the logic updates size field 158 from S bytes to S+T+U bytes, where S, T, and U are positive, non-zero integers.

[0065] Accessing the wraparound region changes the calculation of the translated address for the last level cache 130. In one example, region 620 uses an address space of 2,000 to 2,700, where addresses are expressed as numbers. The entire last level cache 130 uses an address space of 5,000 to 6,000, where addresses are also expressed as numbers. Region 640 uses an address space of 5,800 that wraps around to 5,500. When a received memory request uses a request address of 2,400, the logic determines that the offset is (2,400–2,000), or 400. The logic adds the offset to the region start address of 5,800 to obtain (5,800+400), or 6,200. This value exceeds the boundaries of region 640. In response, the logic determines a difference, which is (6,200–6,000), or 200. The logic adds the difference to the start address to obtain (5,000+200), or 5,200. The translated address is 5,200, and the logic uses the translated address 5,200 to access data from the last level cache 130 in order to service the memory request.

[0066] See Figure 7 , shows a general block diagram of one embodiment of data storage 700. Similar to data storages 500 and 600 and upcoming data storages 800-900 and 1300, previously described circuits and logic are numbered identically. As shown, each of system memory 110 and last-level cache 130 stores data. While the description describes cache 330 as a last-level cache, in other embodiments, cache 330 is another level within the cache subsystem. Data 126 is contiguous data stored in system memory 110. Data 726 is contiguous data added to the region in system memory 110 to create region 720. The size of the contiguous data in system memory 110 copied as the region increases from size 124 to size 724. This increase occurs in the left direction, not the right direction. Therefore, the address pointing to the memory location storing the data at the beginning of region 720 is address 722 ("x2"), not address 122 ("x1").

[0067] Last level cache 130 stores a copy of contiguous data 126 and data 726 for region 720 in region 740. Thus, the size of the contiguous data maintained as a region in last level cache 130 increases from size 144 to size 744. The increase is in the left direction rather than the right direction. Thus, the address pointing to the memory location where the data is stored at the beginning of region 740 is address 742 ("a2") rather than address 142 ("a1"). Region parameters 170 characterize regions 720 and 740. Starting addresses 122 and 142 change, and fields 154 and 156 indicate the change in region parameters 150. Logic also updates size field 158 to the amount of the increase. In the example, the logic updates size field 158 from S bytes to S+V bytes, where S and V are positive, non-zero integers.

[0068] See Figure 8 , a general block diagram of one embodiment of data storage 800 is shown. Data 126 is contiguous data stored in system memory 110. Data 826 is contiguous data added to the region in system memory 110 to create region 820. The size of the contiguous data in system memory 110 that is copied as a region increases from size 124 to size 824. The increase is in the left direction rather than the right direction. Therefore, the address pointing to the memory location where the data is stored at the beginning of region 820 is address 822 ("x2") rather than address 122 ("x1").

[0069] Last-level cache 130 stores a copy of contiguous data 126 and data 826 for region 820 in region 840. Consequently, the size of the contiguous data maintained as a region in last-level cache 130 increases from size 144 to the sum of size 844 and size 846. The contiguous data wraps around last-level cache 130. This increase is in the left direction, not the right direction. Therefore, the address pointing to the memory location storing the data at the beginning of region 840 is address 842 ("a2"), not address 142 ("a1"). Region parameters 150 characterize regions 820 and 840. Starting addresses 122 and 142 change, and fields 154 and 156 indicate the change in region parameters 150. Logic also updates size field 158 by the amount of the increase. In this example, the logic updates size field 158 from S bytes to S+V+W bytes, where S, V, and W are positive, non-zero integers. Accessing the wraparound region changes the calculation of the translated address for the last level cache 130. The logic uses the calculations described earlier for the data store 600.

[0070] See Figure 9, a general block diagram of one embodiment of data storage 900 is shown. Data 126 is contiguous data stored in system memory 110. Data 926 is contiguous data removed from the region in system memory 110 to create region 920. The size of the contiguous data in system memory 110 that is copied as the region is reduced from size 124 to size 924. The reduction is in the right direction rather than the left direction. Therefore, the address pointing to the memory location where the data is stored at the beginning of region 920 is address 922 ("x2") rather than address 122 ("x1").

[0071] Last-level cache 130 stores a copy of contiguous data 126 for region 920 in region 940. Consequently, the size of the contiguous data maintained as a region in last-level cache 130 is reduced from size 144 to size 944. This reduction occurs in the right direction, not the left direction. Therefore, the address pointing to the memory location storing the data at the beginning of region 940 is address 942 ("a2"), not address 142 ("a1"). Region parameters 150 characterize regions 920 and 940. Starting addresses 122 and 142 change, and fields 154 and 156 indicate the change in region parameters 150. The logic also updates size field 158 by the amount of the reduction. In this example, the logic updates size field 158 from S bytes to SU bytes, where S and U are positive, non-zero integers. Note that if the reduction in the size of the region occurs at the end of the region and in the left direction, rather than the right direction, size field 158 is still updated while addresses 122 and 142 remain the same.

[0072] See now Figure 10 , one embodiment of a method 1000 for performing memory access in a computing system is shown. Logic monitors memory access patterns and / or receives software hints for data access (block 1002). As described earlier, software techniques with specific instructions acting as hints, hardware techniques such as those used by hardware prefetchers, or a combination thereof are used to determine when to begin defining a region of memory.

[0073] If the logic does not predict a region for the upcoming data access (the "No" branch of conditional block 1004), the control flow of method 1000 returns to block 1002, where the logic monitors memory access patterns and / or receives software hints. If the logic predicts a region for the upcoming data access (the "Yes" branch of conditional block 1004), the logic initializes parameters characterizing the region for the predicted upcoming data access (block 1006). For example, the logic stores the starting address of the region for the upcoming data access in system memory and stores the starting address of this region in the last-level cache. In addition, the logic stores the region size of this region. In some embodiments, the initial region size is provided in a programmable register in a plurality of control registers and status registers. In some designs, the initial size is between the granularity of a cache line size (e.g., 64 bytes) and a page size (e.g., 4 kilobytes or more).

[0074] The logic stores a copy of contiguous data from a region of system memory for an upcoming data access in the last-level cache. For example, the logic stores a copy of data from a first region of system memory in a second region of the last-level cache (block 1008), where each of the first region and the second region corresponds to a region of system memory for a predicted upcoming data access. The logic services a memory request targeted at the first region by accessing data from the second region (block 1010). If the logic determines that the second region has changed size ("yes" branch of conditional block 1012), the logic updates a parameter characterizing the region to indicate the size change (block 1014).

[0075] If the logic determines that the size of the second region has not changed (the "No" branch of conditional block 1012) and that access to the second region has not yet completed (the "No" branch of conditional block 1016), the control flow of method 1000 returns to block 1010. In block 1010, the logic services the memory request targeted at the first region by accessing data from the second region. If access to the second region has completed (the "Yes" branch of conditional block 1016), the logic updates the parameters characterizing the region to indicate that the region does not exist (block 1018). Thereafter, the control flow of method 1000 returns to block 1002, where the logic monitors memory access patterns and / or receives software hints.

[0076] See Figure 11, shows a general block diagram of one embodiment of data storage 1100. System memory 1110 and last-level cache 1130 each store data. For simplicity of illustration, processing units, communication interfaces, etc. are not shown. Although the description describes cache 1130 as a last-level cache, in other embodiments, cache 1130 is another level within a cache subsystem. Data 1120-1128 is contiguous data stored in system memory 1110. Last-level cache 1130 stores copies of portions of contiguous data 1120-1128 at different points in time.

[0077] In one design example, the weights of a large (deep) neural network are stored in system memory 1110 (such as off-package DRAM). The weights (such as data 1120-1128) are too large to fit in final-level cache 1130 (such as in-package 3D DRAM). During training of the neural network, the weights are evaluated by a processing unit executing a software application. From time point t1 to 11t7 (or from time t1 to t7), the size and contents of the region stored in final-level cache 1130 change. Data 1120 corresponds to the first layer weights of a multi-layer neural network. Data 1122 corresponds to the second layer weights of a multi-layer neural network, and so on.

[0078] Initially, at time t0, data 1120 is copied to the last level cache 1130 (not shown). At a later time t1, data 1122 is added to the area stored in the last level cache 1130. Similarly, at times t2 and t3, data 1124 and data 1126 are added to the area stored in the last level cache 1130. As the evaluation of the neural network proceeds through inference or forward propagation, the area in the last level cache 1130 is expanded to store the weights. Access to the weights of the neural network proceeds in a regular, predictable manner. Therefore, the area in the last level cache 1130 is fully populated before the weights are evaluated. As described earlier, ( Figure 3 Programmable registers of the CSR 347 store parameters L and P to indicate when and by how much to change the size of the region stored in the last-level cache. Thus, the processing unit accesses weights in the in-package last-level cache rather than in the off-package DRAM.

[0079] At time t3, the entire last level cache 1130 is filled. At this point, logic in the cache controller or processing unit resizes the region by reducing the size from the left. At time t4, the logic removes data 1120 from the region in the last level cache 1130. The logic updates the region parameters accordingly. Due to the nature of the software application that performs training on weights, once the processing unit evaluates a given layer, the corresponding weight is no longer needed for the current inference or forward propagation. Therefore, in some designs, a given layer weight is removed from the region of the last level cache 1130. At time t5, data 1126 is added to the region of the last level cache 1130. The region wraps around the last level cache 1130. Accessing the wrap-around region changes the calculation of the translated address for the last level cache 130. The logic uses the calculations described earlier for the data store 600.

[0080] At time t6, the logic removes data 1122 from the region in the last-level cache 1130. The logic updates the region parameters accordingly. At time t7, the logic adds data 1128 to the region in the last-level cache 1130. After the processing unit has processed the last layer of the neural network, the processing unit generates a final output. The processing unit typically compares this final output to an expected value to calculate an error or loss. Training of the neural network then continues with the backpropagation phase. During backpropagation, the processing unit processes the layers of the multi-layer neural network in reverse order. The logic allocates and deallocates regions of the last-level cache 1130 in a manner that supports the reverse order.

[0081] See Figure 12 , shows a general block diagram of one embodiment of data storage 1200. System memory 1210 stores data. For simplicity of illustration, processing units, communication interfaces, etc. are not shown. System memory 1210 stores data in multiple regions. Examples of these regions include a first region 1220 pointed to by address 1212 ("w0"), data in a second region 1222 pointed to by address 1214 ("x0"), data in a third region 1224 pointed to by address 1216 ("y0"), and data in a fourth region 1226 pointed to by address 1218 ("z0").

[0082] In this example, the software application performs a class template calculation in which each element in the output vector stored in the region pointed to by address 1218 ("z0") is the sum of the elements in the other vectors pointed to by addresses 1212-1216 ("w0"-"y0"). For example, if the output vector is represented as vector "d", and each of the vectors in the other regions is represented as "a" to "c", then the value of element d[i] of vector d is [i-1]+a[i]+a[i+1]+b[i-1]+b[i]+b[i+1]+c[i-1]+c[i]+c[i+1]. Adder 1230 sums the values ​​of the elements of the input vector to generate the elements in the output vector. In many cases, no input vectors can be accommodated within the in-package cache. However, each region within the in-package cache is capable of storing the valid portion of each corresponding vector. As the calculation proceeds, each of the regions is updated to maintain the valid portion of each vector. An example of this scheme is shown in the following description.

[0083] See Figure 13 , shows a general block diagram of one embodiment of data storage 1300. Previously described circuits and logic are numbered identically. Last-level cache (LLC) 1330 stores a copy of data stored in system memory 1210. Although the description describes cache 1330 as a last-level cache, in other embodiments, cache 1330 is another level within the cache subsystem. For simplicity of illustration, processing units, communication interfaces, etc. are not shown. Last-level cache 1330 stores data in multiple regions. Examples of these regions include a first region with a size of 1334 ("S bytes") pointed to by address 1332 ("a0"), a second region with a size of 1338 ("T bytes") pointed to by address 1336 ("b0"), a third region 1224 with a size of 1342 ("U bytes") pointed to by address 1340 ("c0"), and a fourth region with a size of 1346 ("V bytes") pointed to by address 1344 ("d0").

[0084] Table 1350 stores region parameters for regions stored in last level cache 1330. In many designs, fields 1352-1358 are equivalent to ( Figure 1) region parameter 150. Here, table 1350 supports multiple separate regions, rather than a single region. In the illustrated embodiment, table 1350 includes four valid rows (entries) for supporting four regions in final level cache 1330. Although four regions and entries are shown, any number of entries and regions are used in other examples. To support multiple regions, logic maintains information in table 1350 to ensure that multiple regions grow, shrink, and wrap around final level cache 1330 without exceeding another region. For each memory access to final level cache 1330, logic compares the request address with each valid supported region in final level cache 1330. In various designs, table 1350 stores information in a fully associative manner. The requested address now checks all N sets of region definition registers (similar to a fully associative cache structure).

[0085] See now Figure 14 , illustrates one embodiment of a method 1400 for performing memory access in a computing system. One or more processing units execute one or more computer programs or software applications. Examples of processing units are processor cores within a CPU, GPU, or other processors. In some embodiments, a system-in-package (SiP) includes a processing unit and a high-bandwidth memory. An example of a high-bandwidth memory is a 3D integrated memory, such as a 3D DRAM. In one embodiment, the processing unit utilizes at least a portion of the 3D DRAM as a cache. The processing unit determines a memory request miss within a cache subsystem at a level below a last-level cache (block 1402). In various embodiments, the processing unit utilizes at least a portion of the high-bandwidth memory as a last-level cache. The processing unit sends an address corresponding to the memory request to the last-level cache (block 1404).

[0086] The logic selects a range from one or more address ranges within the system memory address space that has data stored in the last-level cache (block 1406). If the logic determines that the request address is not within the selected range (the "no" branch of conditional block 1408) and the last range has not been reached (the "no" branch of conditional block 1410), control flow of method 1400 returns to block 1406. In block 1406, the logic selects another range from the one or more address ranges. If the logic determines that the request address is not within the selected range (the "no" branch of conditional block 1408) and the last range has been reached (the "yes" branch of conditional block 1410), the logic sends a memory request including the request address to the system memory (block 1412).

[0087] If the logic determines that the request address is within the selected range ("yes" branch of conditional block 1408), the logic determines an offset based on the difference between the address and the start address of the range in system memory (block 1414). The logic determines a translated address based on the offset and the start address of the range in the last-level cache (block 1416). The logic services the memory request by accessing data from the last-level cache starting at the translated address (block 1418).

[0088] In various embodiments, the previously described methods and / or mechanisms are implemented using program instructions of a software application. The program instructions describe the behavior of the hardware in a high-level programming language (such as C). Alternatively, a hardware design language (HDL) such as Verilog is used. The program instructions are stored on a non-transitory computer-readable storage medium. Numerous types of storage media are available. The storage medium can be accessed by a computing system during use to provide the program instructions and accompanying data to the computing system for program execution. The computing system includes at least one or more memories and one or more processors that execute the program instructions.

[0089] It should be emphasized that the above embodiments are only non-limiting examples of implementations. Once the above disclosure is fully understood, numerous changes and modifications will become apparent to those skilled in the art. It is expected that the following claims will be understood to include all such changes and modifications.

Claims

1. A device, comprising: an interface configured to receive a plurality of memory access requests including request addresses; as well as A logic circuit configured to perform the following operations: maintaining an identification of a first region of contiguous data in a system memory, the first region having a copy of the contiguous data stored in a second region of a cache, wherein the first region represents an address range and the identification comprises: a first start address identifying a starting point of the first region; and the size of the second area; and selecting a first memory access request among the plurality of received memory access requests; comparing a first request address of the first memory access request with each of the start address and a sum of the start address and the size of the second region to determine whether the cache stores requested data of the first memory access request; sending the first memory access request to the system memory in response to determining that the first request address is not within the address range; and The first memory access request is serviced by accessing data from the cache in response to determining that the first request address is within the address range.

2. The apparatus of claim 1 , wherein to service the selected first memory access request by accessing data from the cache, the logic circuit is further configured to: determining an offset based on a difference between the first request address and the first start address; determining a translated address based on the offset and a second start address pointing to a start point of the second region of the cache; as well as The memory access request is serviced by accessing data from the cache starting from the translated address.

3. The apparatus of claim 2 , wherein in response to determining that the translated address points to a memory location outside of the second region of the cache, the logic circuitry is further configured to: determining a wraparound address by subtracting a maximum address of the second region from the translated address; and The memory access request is serviced by accessing data from the cache starting at the wrap-around address.

4. The apparatus of claim 2 , wherein the logic circuit is further configured to: maintaining the second starting address; and The translated address is determined as a sum of the offset and the second starting address.

5. The apparatus of claim 4 , wherein in response to determining that the size of the second region has changed, the logic circuit is further configured to: updating one or more of the first start address, the second start address, and the size of the second area; and The address range is updated based at least in part on the updated size of the second region.

6. The apparatus of claim 4 , wherein in response to predicting the region of upcoming data access, the logic circuit is further configured to: initializing one or more of the first start address, the second start address, and the size of the second area; and A copy of contiguous data from the region of the upcoming data access in the system memory is stored in the cache.

7. The apparatus of claim 6 , wherein predicting the area of ​​the upcoming data access comprises one or more of: The logic circuitry monitors received memory access requests and identifies a pattern identifying a region of the upcoming data access; and The logic circuitry receives a hint from software identifying the region of the upcoming data access.

8. The apparatus of claim 6 , wherein in response to determining that there are no more upcoming data accesses for the region of the upcoming data access, the logic circuit is further configured to: An indication specifying that there is no area stored in the cache by updating the size of the second area to a value of zero bytes is stored.

9. The apparatus of claim 1 , wherein the logic circuitry is further configured to maintain identification of a plurality of regions of contiguous data in the system memory, each region having a copy of corresponding contiguous data stored in the cache, wherein the logic circuitry is further configured to: determining a plurality of address ranges, one address range for each of the plurality of regions; selecting a second memory access request among the received plurality of memory access requests; comparing a second request address of the second memory access request with one or more of the start address and a sum of the start address and a corresponding size of the copy stored in the cache to determine whether the cache stores the requested data of the second memory access request; in response to determining that the second request address is not within any of the plurality of address ranges, sending the second memory access request to a system memory; as well as In response to determining that the second request address is within one of the plurality of address ranges, the second memory access request is serviced by accessing data from the cache.

10. A method comprising: receiving a plurality of memory access requests including request addresses; maintaining an identification of a first region of contiguous data in a system memory, the first region having a copy of the contiguous data stored in a second region of a cache, wherein the first region represents an address range and the identification comprises: a first starting address pointing to a memory location storing data at the beginning of the first region; and the size of the second area; and selecting a first memory access request among the plurality of received memory access requests; comparing a first request address of the first memory access request with each of the start address and a sum of the start address and the size of the second region to determine whether the cache stores requested data of the first memory access request; sending the memory access request to the system memory in response to determining that the request address is not within the address range; and The memory access request is serviced by accessing data from the cache in response to determining that the request address is within the address range.

11. The method of claim 10, wherein to service the first memory access request by accessing data from the cache, the method further comprises: determining an offset based on a difference between the first request address and the first start address; determining a translated address based on the offset and a second start address pointing to a start point of the second region of the cache; as well as The memory access request is serviced by accessing data from the cache starting from the translated address.

12. The method of claim 11 , wherein in response to determining that the translated address points to a memory location outside of the second region of the cache, the method further comprises: determining a wraparound address by subtracting a maximum address of the second region from the translated address; as well as The memory access request is serviced by accessing data from the cache starting at the wrap-around address.

13. The method of claim 11, further comprising: maintaining the second starting address; as well as The translated address is determined as a sum of the offset and the second starting address.

14. The method of claim 13, wherein in response to determining that the size of the second area has changed, the method further comprises: updating one or more of the first start address, the second start address, and the size of the second area; as well as The address range is updated based at least in part on the updated size of the second region.

15. The method of claim 13, wherein in response to predicting the area of ​​upcoming data access, the method further comprises: initializing one or more of the first start address, the second start address, and the size of the second area; as well as A copy of contiguous data from the region of the upcoming data access in the system memory is stored in the cache.

16. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are executable by a processor to perform the following operations: receiving a plurality of memory access requests including request addresses; maintaining an identification of a first region of contiguous data in a system memory, the first region having a copy of the contiguous data stored in a second region of a cache, wherein the first region represents an address range and the identification comprises: a first starting address pointing to a memory location storing data at a beginning of the first region; as well as the size of the second area; as well as selecting a first memory access request among the plurality of received memory access requests; comparing a first request address of the first memory access request with each of the start address and a sum of the start address and the size of the second region to determine whether the cache stores requested data of the first memory access request; sending the first memory access request to the system memory in response to determining that the first request address is not within the address range; as well as The first memory access request is serviced by accessing data from the cache in response to determining that the first request address is within the address range.

17. The non-transitory computer-readable storage medium of claim 16, wherein, to service the first memory access request by accessing data from the cache, the program instructions are further executable by a processor to: determining an offset based on a difference between the first request address and the first start address; determining a translated address based on the offset and a second start address pointing to a start point of the second region of the cache; as well as The memory access request is serviced by accessing data from the cache starting from the translated address.

18. The non-transitory computer-readable storage medium of claim 17 , wherein in response to determining that the translated address points to a memory location outside the second region of the cache, the program instructions are further executable by a processor to: determining a wraparound address by subtracting a maximum address of the second region from the translated address; and The memory access request is serviced by accessing data from the cache starting at the wrap-around address.

19. The non-transitory computer-readable storage medium of claim 17, wherein the program instructions are further executable by a processor to: maintaining the second starting address; and The translated address is determined as a sum of the offset and the second starting address.

20. The non-transitory computer-readable storage medium of claim 19, wherein in response to determining that the size of the second area has changed, the program instructions are further executable by the processor to: updating one or more of the first start address, the second start address, and the size of the second area; and The address range is updated based at least in part on the updated size of the second region.

Images