Enhanced cash usage

Abstract

A method, system, and apparatus for performing a dynamic front-end sacrifice process, including a computer program encoded on a computer storage medium. In one embodiment, the system includes a cache consisting of a plurality of sets, each set having a plurality of ways and configured to receive cache requests having addresses. The cache is configured to perform a front-end sacrifice process to identify the set to which the addresses are mapped and to identify the sacrifice ways of the identified set. The cache is configured to determine that a sacrifice way is dirty and, in response to determining that a sacrifice way is dirty, to write the sacrifice way downstream while processing the retrieval of fill data for the cache request. The cache is configured to write the fill data to the cache when the fill data for the cache request arrives.

Description

Background Art

[0001] This specification relates to a technique for enhancing the utilization of a cache.

[0002] A cache is a device that stores data retrieved from memory or data to be written to memory for one or more different hardware devices of a system. The hardware devices can be different components integrated in a system-on-chip (SOC). A cache is generally organized into multiple sets having multiple ways. A cache can receive a cache request, and the memory address of the cache request (e.g., a fill request) is used to identify a particular set seeking the requested data.

[0003] In the case of a cache miss, the cache selects a victim way that will be overwritten by fill data when acquired. Next, the cache requests the fill data that can be found in memory or a lower-level cache where the victim way is located. However, the cache issuing the fill request may take many cycles to complete, during which the victim way is reserved for the requested fill data.

Summary of the Invention

[0004] The system described in this specification enhances the utilization of a cache by configuring the cache to select either to perform a front-end victim process or to perform a back-end victim process. As used herein, a victim process refers to a process that defines selecting a victim line of the cache to be replaced by newly acquired data. The victim process can be a front-end victim process, in which case the victim line is selected before data is fetched from a downstream source, e.g., a lower-level cache in the cache hierarchy or memory, or a back-end victim process, in which case the victim line is selected after the data is fetched from the downstream source.

[0005] Specifically, the cache can initially perform a front-end sacrifice process. However, if the sacrifice way has been accessed before the fill data arrives, the cache can switch to executing a back-end sacrifice process to select a different sacrifice way. This allows the sacrifice way to be utilized while the fill request is being processed, improving efficient cache utilization.

[0006] Certain embodiments of the subject matter described herein can be implemented to achieve one or more of the following advantages: A cache that chooses to use either a front-end sacrifice process or a back-end sacrifice process improves cache utilization and brings the advantages of both approaches. Specifically, using a front-end sacrifice process helps to reduce the burden on the cache. Furthermore, by dynamically selecting a back-end sacrifice process when the sacrifice way is accessed, the main drawback of the front-end sacrifice, that cache resources are unavailable while the fill request is being processed, is addressed. With the techniques described herein, not only are cache resources kept available while the fill request is being processed, but a completely different sacrifice is selected through the back-end sacrifice. This tends to keep the most frequently used data in the cache rather than unnecessarily evicted.

[0007] Details of one or more embodiments of the subject matter of this specification are described in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from this specification, the drawings, and the claims. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram of an example system. [Figure 2] This is an illustrative process flowchart for deciding whether to take action using the sacrifice method. [Figure 3] This is an illustrative process flowchart for performing a backend sacrifice process. [Modes for carrying out the invention]

[0009] Similar reference numbers and symbols in various drawings indicate the same elements.

[0010] Figure 1 shows an exemplary system 100. System 100 is an example of a system in which the systems, components, and technologies described herein may be implemented. System 100 includes a system-on-a-chip (SOC) 102 coupled to memory 120. SOC 102 is an example of a device that can be installed in or integrated into any suitable computing device.

[0011] The SOC102 includes a processor 106 and a cache system 104 that caches data between the processor 106 and the memory 120. For example, the cache system 104 can store in the SOC102 data retrieved from the memory 120, or data written to memory for the processor 106 or one or more other hardware devices. The other hardware devices may be different components integrated into the SOC102.

[0012] The cache in the cache system 104 is an example of a cache that, as will be further detailed below, can choose to perform either a front-end sacrifice process or a back-end sacrifice process for a single cache request. For clarity, the following example will describe the process as being performed by the L1 cache 108. However, the same technique can also be performed by any other suitable standalone cache or cache within a cache hierarchy.

[0013] The cache system 104 includes three cache levels, including a Level 1 (L1) cache 108 and lower-level caches, such as the L2 cache 110 and the L3 cache 112. Each cache in the cache system 104 can be a set-associative structure having a specific number of sets that can store addresses or cache lines. A set-associative cache structure has a fixed number of ways and a fixed number of address indices within each set. The caches in the cache system 104 can be organized into sets, each having multiple ways, for example, 2, 4, 8, or 16 ways. Thus, there are multiple possible locations in a single set on which a cache line can be written.

[0014] Processor 106 can generate a cache request to the cache system 104 associated with memory operations in memory 120. For example, the L1 cache 108 of the cache system 104 can receive a cache request 122 from processor 106, and the L1 cache 108 can look up the index of the address of the cache line included in the cache line request. The L1 cache 108 can map the index to a specific set of cache 108. The L1 cache 108 can then determine whether the cache line exists by comparing the bits of the address for each way in the specific set. If the cache line exists, the L1 cache 108 identifies that the cache request is a cache hit, and the L1 cache 108 can return the requested data to processor 106.

[0015] Alternatively, if no cache line exists, the L1 cache 108 sends a fill request 114 to a lower cache level (or memory 120). The cache 108 can then perform a front-end sacrifice process to select a sacrifice way to which the data will be stored after it has been retrieved from memory 120 or a lower-level cache. The L1 cache 108 can perform a front-end sacrifice by selecting a sacrifice way using any suitable replacement algorithm (e.g., a replacement policy), such as the Longest Used (LRU) policy.

[0016] The front-end sacrifice process involves the L1 cache 108 selecting a sacrifice way before the fill data 126 for the cache request is retrieved. In this case, the L1 cache 108 can select the sacrifice before or simultaneously with sending the fill request 114 to replace the sacrifice way. The fill request 114 itself may take many cycles to complete, e.g., 100, 1,000, or 10,000 cycles.

[0017] If the sacrifice way selected by the front-end sacrifice process holds dirty data, the L1 cache 108 may optionally send the dirty sacrifice way data 116 while waiting for the fill request 114 to complete. Since the write-back of the dirty sacrifice way data 116 can be done simultaneously with the fill request 114, this process can improve the performance and efficiency of the cache system 104.

[0018] Once the fill request 114 is complete and the fill data 126 arrives from the lower-level cache or memory 120, the L1 cache 108 can choose to use the sacrifice way previously selected by the front-end sacrifice process, or to select a new sacrifice way. Since the selection of a new sacrifice way occurs after the fill data 126 arrives, the new sacrifice way is selected by the back-end sacrifice process.

[0019] Cache 108 can use any appropriate substitution policy to select a sacrifice by the backend sacrifice process. In some embodiments, cache 108 explicitly excludes from consideration the sacrifice way selected by the frontend sacrifice process so as to ensure that the sacrifice way selected by the backend sacrifice process is different.

[0020] The L1 cache 108 can decide which sacrifice process to use based on whether it has received other cache requests for the sacrifice way while the fill request 114 is being processed. For example, after the fill request 114 has started, the cache 108 may receive a second cache request 124 for the sacrifice way. Instead of treating that request 124 as a cache miss, the cache 108 can maintain the sacrifice way and respond to the second request 124 using the sacrifice way data. When the cache 108 is writing back dirty sacrifice way data 116, the cache 108 is essentially maintaining a clean copy of the sacrifice way data.

[0021] This also means that the L1 cache 108 can utilize all the ways of the identified set while the fill request 114 is being processed, thereby improving the efficiency and utilization of the cache 104.

[0022] Alternatively, if the sacrifice way selected by the front-end sacrifice process is not accessed while the fill request 114 is being processed, the L1 cache 108 can write the fill data 126 to the original sacrifice way.

[0023] FIG. 2 is a flowchart of an exemplary process for determining whether to perform an action in a sacrificial way. For convenience, process 200 is described as being performed by a cache. For example, process 200 can be performed by a cache system suitably configured in accordance with this specification, such as cache system 104 of FIG. 1.

[0024] The cache system is configured to receive a cache request having an address for a set of caches (202). For example, the cache system can receive a request to address A having index "1" and tag "T4". [Table 1]

[0025] The cache of the cache system identifies the set that is the mapping destination of the address (204). The cache can search for the index of the cache line included in the cache request. The cache determines whether a cache line exists by comparing the bits of the address for each way of a particular set. If a cache line exists, the cache identifies that the cache request is a cache hit, and the cache performs an operation associated with the cache line at the requested index.

[0026] Alternatively, if no cache line exists, the cache identifies that the cache request is a cache miss. For example, referring to Table 1, the cache can determine that address A having index "1" and tag "T4" does not exist. The cache determines a tag mismatch between the requested tag "T1" and the actual tag "T1" at the requested index, and the cache can identify a cache miss.

[0027] If the cache identifies a cache miss, the cache performs a front-end sacrifice process to identify the identified set of sacrifice ways that will be replaced by the requested cache line (206). [Table 2]

[0028] In conventional embodiments, the cache determines whether a sacrificial way contains dirty data, and if so, sends a write-back request to a lower-level cache downstream. For example, referring to Table 1, the L1 cache determines that a sacrificial way contains dirty data. The L1 cache sends a write-back request to the L2 cache for tag "T1" by sending the sacrificial way to L2. Referring to Table 2, the L1 cache installs tag "T4" with a fill indication and an invalid data indication. The L1 cache may increase latency and decrease cache efficiency by sending a fill request after installing the tag.

[0029] In this embodiment, the cache refrains from clearing the sacrificial way and sends a fill request to the processor requesting the fill request data. For example, referring to Table 1, the L1 cache can select the line at index "1" as the sacrificial way, and the L1 cache can send a fill request to the processor to replace the line of the sacrificial way.

[0030] After sending a fill request, the cache determines whether the sacrifice way is dirty (for example, whether the sacrifice way contains dirty data) before writing the sacrifice way to a lower-level cache (208). For example, the L1 cache may determine whether the sacrifice way contains dirty data before sending it to the L2 cache.

[0031] If the cache determines that a sacrificial way is dirty, the cache writes the sacrificial way downstream and maintains a clean copy of the data corresponding to the sacrificial way while the fill request data is being processed for retrieval from the processor (210). The cache determines that the data in the sacrificial way is dirty with respect to the corresponding data in a lower-level cache (e.g., the L2 cache). The cache can write the data in the sacrificial way (e.g., dirty data) to a lower-level cache downstream. Since the dirty data corresponds to the clean data stored in a lower-level cache (e.g., a lower level in the cache hierarchy), the cache can convert the data in the sacrificial way from dirty to clean by writing the dirty data to a lower-level cache downstream. The cache can then provide the clean data of the sacrificial way to subsequent cache requests until the sacrificial way is replaced.

[0032] During this time, the cache can access the clean data of the sacrificial way, so the cache is configured to utilize all ways while the fill data for the cache request is being processed. The cache can hold the tags and data of the sacrificial way, fulfill the cache request, and transfer the data of the selected set of ways to the downstream cache, thereby increasing the cache's efficiency. Furthermore, the cache can send write-back requests (e.g., write-clean requests) to lower-level caches, which can transition the sacrificial way from dirty to clean and mark the sacrificial way as clean. [Table 3]

[0033] For example, referring to Table 3, the L1 cache transitions the status of the sacrifice way from dirty to clean while waiting for fill request data by maintaining a clean copy. The L1 cache replaces the dirty data with the newly cached data. [Table 4]

[0034] Furthermore, while waiting for the fill request data, the L1 cache writes the data for the sacrifice way by holding the index and tag "T1" of the sacrifice way, as well as the tag of the requested cache line (e.g., insertion tag "T4"). [Table 5]

[0035] When fill data arrives for a cache request, or when the cache determines that the sacrifice way is clean (e.g., not dirty), the cache may write the fill data to the sacrifice way according to a front-end sacrifice process or a back-end sacrifice process (212). The cache decides whether to perform a front-end sacrifice process or a back-end sacrifice process based on whether the cache has accessed the sacrifice way, as will be described in detail below with reference to Figure 3. After performing a front-end sacrifice process or a back-end sacrifice process, the cache may perform a clean evict procedure on the sacrifice way, and the cache sends the sacrifice way to a lower-level cache. The cache fills the index of the evicted sacrifice way with the fill request data. For example, the L1 cache sends the sacrifice way to the L2 cache, and the L1 cache installs the corresponding fill request data "D4" in addition to the insertion tag "T4" at index "1".

[0036] Figure 3 is an exemplary process flowchart for performing a backend sacrificial process. For convenience, process 300 is described as being performed by a cache. Process 300 can be performed by a cache system preferably configured according to this specification, such as the cache system 104 in Figure 1.

[0037] The cache then receives fill data for the first cache request (304). For example, the L1 cache may receive fill request data from the processor.

[0038] The cache then determines whether the sacrifice way has been accessed (306). In some cases, the cache may receive a second cache request for the sacrifice way while it is processing the retrieval of fill data for the first cache request.

[0039] Next, when fill data arrives, the cache can determine if any process is requesting data from the sacrifice way. Rather than clearing the sacrifice way once identified by a front-end sacrifice process, as described above, the cache can maintain a clean copy of the sacrifice way while the fill request is being processed. Meanwhile, other processes can request data from the sacrifice way, and in response, the cache can provide clean sacrifice way cache data.

[0040] If some process requests data for a sacrifice way, the cache can perform a backend sacrifice process. In this case, the cache performs a backend sacrifice process to select a different second sacrifice way in the set (308). For example, referring to Figure 2, the L1 cache receives fill request data from a first cache request, and the L1 cache can select a different second sacrifice way.

[0041] The cache can then write the fill data to a different second sacrifice way (312). For example, the L1 cache writes the fill request data from the first cache request to a different second sacrifice way. If other processes still need the data in the sacrifice way identified by the front-end sacrifice process, these processes are likely to access that data again in the future, thus improving cache utilization. Therefore, it is often more efficient for the back-end sacrifice process to select a different way to keep the accessed data in the cache.

[0042] If the sacrificial way is not accessed, the cache writes the fill data to the sacrificial way identified by the front-end sacrificial process (310). For example, referring to Figure 2, the L1 cache writes the requested fill data to the originally selected sacrificial way.

[0043] This specification uses the term “configured” in relation to systems and computer program components. When one or more computer systems are configured to perform a particular operation or action, it means that, while in operation, software, firmware, hardware, or a combination thereof is installed on the system that causes the system to perform that operation or action. When one or more computer programs are configured to perform a particular operation or action, it means that one or more programs, when executed by a data processing device, contain instructions that cause the device to perform that operation or action.

[0044] The subject matter and functional embodiments described herein can be implemented in digital electronic circuits, tangibly embodied computer software or firmware, or computer hardware, including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded in a tangible, non-temporary storage medium, which are executed by or control the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random-access memory device, or a serial-access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions may be encoded in artificially generated transmitted signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0045] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, machines, and equipment for data processing, including, for example, programmable processors, computers, or multiple processors or multiple computers. A device may also be, or further include, special-purpose logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). Optionally, in addition to hardware, a device may include code that generates an execution environment for computer programs (e.g., code that constitutes processor firmware, protocol stacks, database management systems, operating systems, or one or more combinations thereof).

[0046] Computer programs, sometimes called or described as programs, software, software applications, apps, modules, software modules, scripts, or code, can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A program may, but does not have to, correspond to a file in a file system. A program may be stored in a single file dedicated to the program in question, in a file containing other programs or data, such as one or more scripts stored in a markup language document, or in multiple collaborative files, such as files containing one or more modules, subprograms, or parts of code. A computer program can be deployed to run on one computer, or it can be deployed to run on multiple computers located in one place or distributed across multiple locations and interconnected by a data communication network.

[0047] In this specification, the term “engine” is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Generally, an engine is implemented as one or more software modules or components and installed in one or more computers at one or more locations. In some cases, one or more computers are dedicated to a particular engine, while in other cases, multiple engines may be installed and run on the same one or more computers.

[0048] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to act on input data and generate outputs. Alternatively, the processes and logic flows can be performed by special-purpose logic circuits, such as FPGAs or ASICs, or by a combination of special-purpose logic circuits and one or more programmed computers.

[0049] A computer suitable for running computer programs can be based on a general-purpose or dedicated microprocessor, or both, or any other type of central processing unit. Generally, the central processing unit receives instructions and data from read-only memory or random-access memory, or both. The basic elements of a computer are a central processing unit for issuing or executing instructions, and one or more memory devices for storing instructions and data. The central processing unit and memory may be complemented by or integrated into special-purpose logic circuits. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operablely coupled for receiving data from them, transferring data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be integrated into other devices, for example, a mobile phone, a personal digital assistant (PDA), a mobile audio player or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, such as a Universal Serial Bus (USB) flash drive.

[0050] Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks, as well as all forms of non-volatile memory, media, and memory devices.

[0051] To provide user interaction, embodiments of the subject matter described herein can be implemented in a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal screen) monitor, and a keyboard and pointing device, such as a mouse or trackball, on which the user can input to the computer. Other types of devices can also be used to interact with the user. For example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or haptic feedback, and input from the user can be received in any form, including acoustic, voice, or haptic input. Furthermore, the computer can interact with the user by sending and receiving documents to and from the device used by the user, for example, by sending a web page to a web browser on the user's device in response to a request received from a web browser. The computer can also interact with the user by sending text messages or other forms of messages to a personal device (for example, a smartphone running a messaging application) and then receiving a response message from the user.

[0052] Data processing equipment for implementing machine learning models may include, for example, dedicated hardware accelerator units for handling the general and computationally intensive parts of machine learning training or production, i.e., inference workloads.

[0053] Machine learning models can be implemented and deployed using machine learning frameworks, such as the TensorFlow framework.

[0054] Embodiments of the subject matter described herein can be implemented in a computing system that includes, for example, backend components such as a data server, or middleware components such as an application server, or frontend components such as a client computer having a graphical user interface, a web browser, or an application that allows a user to interact with the implementation of the subject matter described herein, or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by digital data communications of any form or medium, such as a communication network. Examples of communication networks include local area networks (LANs), wide area networks (WANs), and, for example, the Internet.

[0055] A computing system can include clients and servers. Clients and servers are generally separate from each other and typically interact through a communication network. The client-server relationship arises from computer programs that run on each computer and have a client-server relationship with each other. In some embodiments, the server sends data (e.g., an HTML page) to a user device for the purpose of displaying data to a user interacting with a device acting as a client and receiving user input from that user. Data generated on the user device (e.g., the results of user interactions) can be received by the server from the device.

[0056] In addition to the embodiments described above, the following embodiments are also innovative.

[0057] Embodiment 1 is a cache comprising multiple sets, each set having multiple ways, and the cache is Receiving a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, Initiating a request to obtain fill data for the aforementioned cache request, The cache is configured to perform an operation that, upon arrival of fill data for the cache request, includes choosing to write the fill data to the cache according to the front-end sacrifice process, or to write the fill data to the cache according to the back-end sacrifice process.

[0058] Embodiment 2 is, Determining that the aforementioned sacrifice way is dirty, The cache according to Embodiment 1 further includes writing the sacrificial way downstream in response to determining that the sacrificial way is dirty, and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

[0059] Embodiment 3 allows for selection between the front-end sacrificial process and the back-end sacrificial process, The second cache request determines that it has been received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, A cache according to any one of embodiments 1 to 2, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

[0060] Embodiment 4 allows for selection between the front-end sacrificial process and the back-end sacrificial process, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. A cache according to any one of embodiments 1 to 3, comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

[0061] Embodiment 5 is the cache according to Embodiment 2, wherein the operation includes writing data from the dirty sacrifice way before the fill data for the cache request arrives.

[0062] Embodiment 6 is the cache described in Embodiment 5, wherein the operation includes marking the sacrificial way as clean.

[0063] Embodiment 7 is a cache according to any one of Embodiments 1 to 6, wherein the cache is configured to utilize the sacrifice way selected by the front-end sacrifice process while the fill data for the cache request is being processed.

[0064] Embodiment 8 is a cache according to any one of Embodiments 1 to 7, wherein the cache is configured to utilize all of the identified set of ways while the fill data for the cache request is being processed.

[0065] Embodiment 9 is a cache according to any one of Embodiments 1 to 8, wherein the cache is configured to use both a front-end sacrificial process and a back-end sacrificial process.

[0066] Embodiment 10 is the cache according to Embodiment 9, wherein the cache is configured to perform a front-end sacrifice process and a back-end sacrifice process for a single cache request.

[0067] Embodiment 11 is, Receiving a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, Initiating a request to obtain fill data for the aforementioned cache request, A method comprising, upon arrival of fill data for the cache request, selecting to write the fill data to the cache according to the front-end sacrifice process, or to write the fill data to the cache according to the back-end sacrifice process.

[0068] Embodiment 12 is, Determining that the aforementioned sacrifice way is dirty, The method of Embodiment 11 further includes, in response to determining that the sacrificial way is dirty, writing the sacrificial way downstream and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

[0069] Embodiment 13 allows for selection between the front-end sacrificial process and the back-end sacrificial process, The second cache request determines that it has been received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, The method according to any one of embodiments 11 to 12, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

[0070] Embodiment 14 allows for selection between the front-end sacrificial process and the back-end sacrificial process, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. The method according to any one of embodiments 11 to 13, comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

[0071] Embodiment 15 is the method of Embodiment 12, further comprising writing data from the dirty sacrifice way before the fill data for the cache request arrives.

[0072] Embodiment 16 is a system for performing a front-end sacrificial process, Memory configured to store data, The cache includes multiple sets, each set having multiple ways, and the cache includes, Receiving a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, Initiating a request to obtain fill data for the aforementioned cache request, The system is configured to repeatedly perform an operation that, upon arrival of fill data for the cache request, includes either writing the fill data to the cache according to the front-end sacrifice process, or writing the fill data to the cache according to the back-end sacrifice process.

[0073] Embodiment 17 is, Determining that the aforementioned sacrifice way is dirty, The system according to Embodiment 16 further includes writing the sacrificial way downstream in response to determining that the sacrificial way is dirty, and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

[0074] Embodiment 18 allows for selection between the front-end sacrificial process and the back-end sacrificial process, The second cache request determines that it has been received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, The system according to any one of embodiments 16 to 17, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

[0075] Embodiment 19 allows for selection between the front-end sacrificial process and the back-end sacrificial process, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. The system according to any one of embodiments 16 to 18, further comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

[0076] Embodiment 20 is the system according to Embodiment 17, wherein the operation includes writing data from the dirty sacrifice way before the fill data for the cache request arrives.

[0077] While this specification includes details of many specific embodiments, these should not be construed as limiting the scope of any invention or claimable content, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described as a single embodiment may also be implemented in multiple embodiments, individually or in any preferred secondary combination. Furthermore, features may be described above as functioning in a certain combination, and even if initially claimed as such, one or more features from the claimed combination may be removed from the combination, and the claimed combination may cover subcombinations or variations of subcombinations.

[0078] Similarly, while operations are shown in the drawings and described in a specific order in the claims, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all shown operations be performed, in order to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described can generally be integrated into a single software product or packaged into multiple software products.

[0079] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order, and this may still yield desirable results. As an example, the process shown in the accompanying drawings does not necessarily require the actions to be performed in the specific order or sequence shown to obtain the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A cache comprising multiple sets, each set having multiple ways, the cache is Receiving a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, Initiating a request to obtain fill data for the aforementioned cache request, A cache configured to perform the following actions upon arrival of fill data for the cache request: writing the fill data to the cache according to the front-end sacrifice process, or writing the fill data to the cache according to the back-end sacrifice process.

2. The aforementioned operation is, Determining that the aforementioned sacrifice way is dirty, The cache according to claim 1, further comprising: writing the sacrificial way downstream in response to determining that the sacrificial way is dirty; and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

3. The choice between the front-end sacrifice process and the back-end sacrifice process is, The second cache request determines that it was received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, The cache according to any one of claims 1 to 2, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

4. The choice between the front-end sacrifice process and the back-end sacrifice process is, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. The cache according to any one of claims 1 to 3, comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

5. The cache according to claim 2, wherein the operation includes writing data from the dirty sacrifice way before the fill data for the cache request arrives.

6. The cache according to claim 5, wherein the operation includes marking the sacrificial way as clean.

7. The cache according to any one of claims 1 to 6, wherein the cache is configured to utilize the sacrifice way selected by the front-end sacrifice process while the fill data for the cache request is being processed.

8. The cache according to any one of claims 1 to 7, wherein the cache is configured to utilize all of the identified set of ways while the fill data for the cache request is being processed.

9. The cache according to any one of claims 1 to 8, wherein the cache is configured to use both a front-end sacrifice process and a back-end sacrifice process.

10. The cache according to claim 9, wherein the cache is configured to perform a front-end sacrifice process and a back-end sacrifice process for a single cache request.

11. A cache with multiple sets can receive a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, Initiating a request to obtain fill data for the aforementioned cache request, A method comprising, upon arrival of fill data for the cache request, selecting to write the fill data to the cache according to the front-end sacrifice process, or to write the fill data to the cache according to the back-end sacrifice process.

12. Determining that the aforementioned sacrifice way is dirty, The method of claim 11, further comprising: writing the sacrificial way downstream in response to determining that the sacrificial way is dirty, and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

13. The choice between the front-end sacrifice process and the back-end sacrifice process is, The second cache request determines that it was received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, The method according to any one of claims 11 to 12, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

14. The choice between the front-end sacrifice process and the back-end sacrifice process is, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. The method according to any one of claims 11 to 13, comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

15. The method of claim 12, further comprising writing data from the dirty sacrifice way before the fill data for the cache request arrives.

16. A system for performing a front-end sacrifice process, wherein the system is Memory configured to store data, The cache includes multiple sets, each set has multiple ways, and the cache includes, Receiving a cache request that has an address, Identifying the set to which the aforementioned address is mapped, A front-end sacrifice process is performed to identify the sacrifice way of the identified set, In response to determining that the sacrifice way is dirty, the sacrifice way is written downstream while the fill data for the cache request is being processed. A system configured to repeatedly perform an operation including writing the fill data for the cache request to the cache when fill data for the cache request arrives.

17. Determining that the aforementioned sacrifice way is dirty, The system according to claim 16, further comprising: writing the sacrificial way downstream in response to determining that the sacrificial way is dirty; and maintaining a clean copy of the data corresponding to the sacrificial way while processing the retrieval of fill data for the cache request.

18. The choice between the front-end sacrifice process and the back-end sacrifice process is, The second cache request determines that it was received for an address cached in the sacrifice way while the fill data for the cache request is being processed. In response, the backend sacrifice process is used to identify a different second sacrifice way, The system according to any one of claims 16 to 17, comprising writing the fill data to the different second sacrifice way identified by the backend sacrifice process.

19. The choice between the front-end sacrifice process and the back-end sacrifice process is, While the fill data for the cache request is being processed, it is determined that no cache request has been received for the sacrifice way. The system according to any one of claims 16 to 18, comprising writing the fill data to the sacrifice way identified by the front-end sacrifice process.

20. The system according to claim 17, wherein the operation includes writing data from the dirty sacrifice way before the fill data for the cache request arrives.

Images