Method and apparatus for adaptive cache write gathering

Abstract

A method, a cache or a non- transitory storage medium comprising identifying, in the cache (112, fig.1), a first cache line to be cleaned to a row of the backing memory (804). When write gathering is

Description

BACKGROUND

[0001] Computer applications often access memory with a degree of temporal or spatial locality. Performance can be improved by storing recently accessed data in a small, fast memory called a cache. Data in a cache of data processing system is referred as being “clean” when it matches corresponding data in a backing memory, and “dirty” when it does not match the corresponding data. When data is evicted from a last level cache of a data processing system, it must be written back to a backing memory if it is “dirty.” The process of writing data to the backing memory is not time-critical compared to the process of reading data from the memory (which can delay execution of the computer application). However, in the memory itself, the writeback process occurs in series with the read process and can interfere with read operations.

[0002] In order to access a row of a dynamic random-access memory (DRAM), the row must be activated and placed into a row buffer before a column can be accessed for reading or writing. The row must then be closed by pre-charging it. Associated with these operations are a number of timing constraints which dictate, for example, the number of cycles between a write request and a read request, or the number of cycles between a write to one row and a write to a different row. One approach to minimizing the interference between read and write operations is to minimize the time spent writing. This may be achieved, for example, by grouping together a number of write requests to the same DRAM row.

[0003] A disadvantage of these approaches is that the clean line may become dirty again before eviction from the cache - resulting in an additional write operation when it is eventually evicted. This is referred to as excess cleaning, and can negatively impact DRAM power efficiency and read latency if too many excess writes are generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The accompanying drawings provide visual representations which will be used to describe various representative embodiments more fully and can be used by those skilled in the art to understand better the representative embodiments disclosed and their inherent advantages. In these drawings, like reference numerals identify corresponding or analogous elements.

[0005] FIG. 1 is a block diagram of a data processing system, in accordance with various representative embodiments.

[0006] FIG. 2 is a further block diagram of a data processing system, in accordance with various representative embodiments.

[0007] FIG. 3 is a further block diagram of a data processing system, in accordance with various representative embodiments.

[0008] FIG. 4 is a further block diagram of a data processing system, in accordance with various representative embodiments.

[0009] FIG. 5 is a block diagram of a last level cache, in accordance with various representative embodiments.

[0010] FIG. 6 is a flow chart of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments.

[0011] FIG. 7 is a flow chart of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments.

[0012] FIG. 8 is a flow chart of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments.

[0013] FIG. 9 is a flow chart of a method of cleaning a cache to a backing memory in accordance with various representative embodiments. DETAILED DESCRIPTION

[0014] The various apparatus and devices described herein provide mechanisms for reducing the amount of excess cleaning when write gathering is used in a last level cache of a data processing system.

[0015] While this present disclosure is susceptible of embodiment in many different forms, there is shown in the drawings and will herein be described in detail specific embodiments, with the understanding that the embodiments shown and described herein should be considered as providing examples of the principles of the present disclosure and are not intended to limit the present disclosure to the specific embodiments shown and described. In the description below, like reference numerals are used to describe the same, similar, or corresponding parts in the several views of the drawings. For simplicity and clarity of illustration, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.

[0016] Data in a cache of data processing system is referred as being “clean” when it matches corresponding data in a backing memory, and “dirty” when it does not match the corresponding data. A cache line may be “cleaned” by copying data in the cache line back to the backing memory so that data the cache line matches corresponding data in the backing memory. When data in a cache line is evicted from the cache, it must be written back to the backing memory if it is “dirty.” If the cache line is “clean,” it can be overwritten with new data without needing to be written back.

[0017] Write gathering is a process of grouping together a number of write requests to the same DRAM row. The write requests are for dirty cache lines in a last level cache that have not yet been evicted. Write gathering saves power by generating consecutive write operations to the same DRAM row. However, excess writes may be generated due to lines being written and then cleaned multiple times before eviction. Write gathering can negatively impact DRAM power efficiency if there are too many excess writes, since the power saving of consecutive writes in the same row is offset, or negated, by the power cost of the excess writes.

[0018] In accordance with embodiments of the present disclosure, dynamic feedback from the cache where the gathering is being performed is used to determine if a gathering operation should go ahead or not.

[0019] In accordance with a first embodiment, a reduction in excess writes is obtained by using a per-line state from the replacement policy. This indicates how close a line is to being evicted. Lines that are not close to being evicted are excluded from the write gather attempt as there is a strong likelihood that they will be rewritten again before being evicted.

[0020] In a second embodiment, a cache-wide state is used that compares the number of lookup requests to the count of dirty evictions. A ratio of dirty evictions to lookups can be calculated. When this value is below some threshold, write gather is disabled as this indicates that the cache is not under pressure. Regardless of the per-line state, cache lines are unlikely to be evicted in the near future and thus they are likely to be written again before eviction.

[0021] In a third embodiment, the number of gathered lines is compared to a threshold number. If below the threshold number, the gathered lines are not cleaned.

[0022] These embodiments may be used separately or in combination to control when write gathering is performed.

[0023] These mechanisms may be implemented using software-controlled thresholds, such as an eviction priority threshold below which lines should not be gathered, or a ratio threshold of dirty evictions to lookup requests. These thresholds could be obtained through performance and power sweeps for content of interest.

[0024] The mechanism could be implemented such that the effect of different threshold settings in addition to the effect of the currently selected thresholds are monitored. This approach enables the system to react to changes in behavior and dynamically switch between different settings.

[0025] FIG. 1 is a block diagram of a data processing system, in accordance with various representative embodiments. The diagram shows an example data processing system 100 with multiple initiators including central processing units (CPUs) 102 in a CPU cluster, CPU 104, media accelerator 106 and artificial intelligence (AI) accelerator 108. Some of the initiators have their own private cache (e.g., level one and / or level cache (L1 / L2) cache 110) or a shared cache (e.g., level three (L3) cache 112). System 100 also includes interconnect 114 to which all initiators are connected, a single Last Level Cache (LLC) 116 to which a single Dynamic Memory Controller (DMC) 118 is connected which in turn is connected to a single Dynamic Random-Access Memory (DRAM) 120. All accesses to DRAM 120 from the initiators go through the single LLC and DMC / DRAM instances.

[0026] In such a system, write gathering can be done across all accesses, since all accesses go through a single LLC.

[0027] FIG. 2 is a further block diagram of a data processing system, in accordance with various representative embodiments. The diagram shows an example data processing system 200. Multiple initiators (not shown) are connected via interconnect 114 to a single Last Level Cache (LLC) 116 to which a multiple Dynamic Memory Controllers (DMCs) 118 are connected. In turn, each DMC 118 is connected to a single DRAM 120. All accesses to a DRAM from the initiators will go through the single LLC. Determining which DMC, and thus DRAM, access will go through is determined by the LLC. This may be done may mapping the physical address using a hashing function, for example.

[0028] In such a system, write gathering can be done across all accesses as all accesses go through a single LLC. The LLC needs to consider the impact of hashing as contiguous physical addresses may target different DMCs and DRAMs and thus would not increase the number of in-row hits of the target DMC / DRAM.

[0029] FIG. 3 is a further block diagram of a data processing system, in accordance with various representative embodiments. The diagram shows an example data processing system 300. Multiple initiators (not shown) are connected via interconnect 114 to multiple Last Level Caches (LLCs) 116 each of which is connected to a single Dynamic Memory Controller (DMC) 118 which, in turn, is connected to a single DRAM 120. Interconnect 114 determines which LLC instance (and thus DMC and DRAM) (e.g., via hashing) the access will be directed to.

[0030] In such a system, write gathering would typically happen only within a single LLC instance as each LLC instance only observes transactions sent to it and adding support for cross LLC instance communication would introduce a lot of complexity (possibly for little or no gain). Also, as different LLCs are targeting different DMCs / DRAMs, gathering across LLC instances may not increase in-row hits of the target DMC / DRAM.

[0031] FIG. 4 is a further block diagram of a data processing system, in accordance with various representative embodiments. The diagram shows an example data processing system 400. Again, multiple initiators (not shown) are connected to interconnect 114. System 400 includes multiple Last Level Caches (LLCs) 116. Pairs of LLCs are connected to a single Dynamic Memory Controller (DMC) 118 which in turn is connected to a single DRAM 120. Interconnect 114 determines which LLC instance (and thus DMC and DRAM) (e.g., via hashing) the access will be directed to.

[0032] In such a system, write gathering would typically happen only within a single LLC instance as each LLC instance only observes transactions sent to it and adding support for cross LLC instance communication would introduce a lot of complexity. Only LLCs connected to the same DMC (and thus DRAM) would benefit from cross LLC communication as only these would increase in-row hits of the target DMC / DRAM.

[0033] Various embodiments of the present disclosure relate to a cache that includes a data memory array, a metadata memory array, and a cache controller. In operation, the memory array stores cache lines, and the metadata memory array stores address tags and status bits associated with the stored cache lines. The cache controller is configured to move the cache lines between the data memory array and a backing memory and to move cache line data between the data memory array and a data processor. The cache controller includes cleaning logic circuitry configured to identify, in the cache, a first cache line to be cleaned to a row of the backing memory. When write gathering is enabled in the cache, the cleaning logic circuitry identifies, in the cache, additional cache lines to be cleaned to the row of the backing memory. When the number of additional cache lines exceeds a threshold number, the cleaning logic circuitry cleans the first cache line and the additional cache lines to the backing memory, otherwise, when the number of additional cache lines does not exceed the threshold number, only the first cache line is cleaned to the backing memory. When write gathering is not enabled, only the first cache line is cleaned to the backing memory.

[0034] In one embodiment, write gathering is enabled when the memory has sufficient bandwidth, such as when the memory bus is idle or operating at less than full capacity.

[0035] Additional cache lines to be cleaned to the row of the backing memory may be identified by searching the metadata memory array cache line addresses in one or more address regions adjacent an address of the first cache line.

[0036] In one embodiment, the cleaning logic circuitry is further configured to determine a ratio of dirty cache evictions to cache lookup requests for the cache. Write gathering is enabled when the ratio of dirty cache evictions to cache lookup requests exceeds a threshold ratio and disabled when the ratio of dirty cache evictions to cache lookup requests does not exceed the threshold ratio.

[0037] Determining the ratio of dirty cache evictions to cache lookup requests for the cache may include determining a number of dirty cache evictions in a time period from a total number of cache evictions in the time period. This may be done, for example, by scaling the total number of evictions in the time period by a factor to estimate the number of dirty evictions. The ratio of dirty cache evictions to cache lookup requests can then be determined by dividing the estimated number of dirty evictions by the number of cache lookup request. Equivalently, the ratio may be determined by scaling the ratio of total evictions to cache lookup requests. The scaling factor may be predetermined or estimated during operation.

[0038] In one embodiment, identifying, in the cache, the additional cache lines to be cleaned to the row of the backing memory includes identifying, from an address tag of the first cache line, the row of the backing memory to which the first cache line will be cleaned, and searching the metadata memory array of the cache, based on address tags and cache line status bits, to find valid dirty cache lines to be cleaned to the identified row of the backing memory.

[0039] In one embodiment, the cache controller includes eviction control circuitry configured to store an eviction priority value for each cache line and update the stored eviction priority values in response to cache installations and cache hits. Searching of the cache is restricted to valid dirty cache lines for which the eviction priority value of the cache line exceeds a threshold value. The eviction control circuitry may be configured, for example, to determine a Re-Reference Prediction Value (RRPV) for each cache line of the cache, in which case searching the cache is restricted to valid dirty cache lines for which the RRPV exceeds a threshold value. Alternatively, the eviction control circuitry may be configured to store a use classification and age for each cache line in a cache set and update the stored use classification and age for each cache line in response to installations and hits to the cache set. An eviction priority table is then accessed, based on the stored use classifications and ages for cache lines in a cache set, to determine an eviction priority value and searching the cache is restricted to cache lines for which the eviction priority value exceeds a threshold value.

[0040] FIG. 5 is a block diagram of a last level cache, in accordance with various representative embodiments. Last level cache 116 includes data array 502, metadata data array 504, and cache controller 506. Data array 502 is used for storing data in cache lines, in anticipation of the data being reused by a processor. Metadata array 504 stores information relating to the data in data array 502. This may include a validity bit (“V”) that indicates if the data in the cache line is valid, status bits (“S”) that indicate, for example, if the data is dirty or clean, unique (to an initiator) or shared (between multiple initiators), and address tag (“TAG”) that indicates the location of the data in a backing memory.

[0041] Cache controller 506 is configured to access a backing memory (via a memory controller) via bus 508 to read cache lines for storage in data array 502 or writeback data. In addition, cache controller 506 is coupled via link 510 to an interconnect of a data processing system.

[0042] Cache controller 506 includes cleaning logic circuitry 512, which controls writing data in cache lines back to the backing memory. Writeback may occur when a dirty cache line is evicted from the cache (to provide space for a new cache line) or in response to an instruction from an initiator, for example. In accordance with embodiment of the present disclosure, cache controller 506 may perform write gathering. Operation of cache controller 506 is discussed in more detail below.

[0043] Eviction control circuitry 514 is used to determine which cache line is to be evicted from the cache when space is needed in the cache for a new cache line. Eviction control circuitry 514 may maintain information, such as access counts associated with data in data array 502, in storage array “COUNT” 516 for use in an eviction policy. The eviction policy determines eviction priorities indicating which cache line should be evicted from the cache. For example, the eviction priority may be an estimation of which cache line is least likely to be accessed at some future time period.

[0044] FIG. 6 is a flow chart 600 of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments. The method includes identifying, in the cache, a first cache line to be cleaned to a row of the backing memory at block 602. When write gathering is not enabled, as depicted by the negative branch from decision block 604, the first cache line is cleaned at block 606 by writing back to the backing memory. When write gathering is enabled, as depicted by the positive branch from decision block 604, the row in the backing memory that the line will be written to is identified at block 608. Additional cache lines in the cache to be cleaned to the same row of the backing memory are identified at block 610. When the number of additional cache lines exceeds a threshold number, as depicted by the positive branch from decision block 612, the first valid dirty cache and the additional cache lines are clean to the backing memory at block 614. Otherwise, when the number of cache lines in the additional cache lines does not exceed the threshold number as depicted by the negative branch from decision block 612, only the first cache line is cleaned to the backing memory at block 606. The threshold number of lines may be predetermined, set by user software, or determined automatically in response to monitored cache operation.

[0045] FIG. 7 is a flow chart 700 of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments. The method follows that described above with reference FIG. 6, but in this embodiment a ratio of dirty cache evictions to cache lookup requests for the cache is determined at block 702. When the ratio of dirty cache evictions to cache lookup requests exceeds a threshold ratio, write gathering is enabled, as depicted by the positive branch from decision block 704. When the ratio of dirty cache evictions to cache lookup requests does not exceed the threshold ratio, write gathering is disabled, as depicted by the negative branch from decision block 704.

[0046] In one embodiment, the threshold ratio for the ratio of dirty evictions to lookups is found though experimentation. A range of values (in 10% increments, for example) is tested and the effect on DRAM power efficiency monitored - either by direct measurement or by monitoring proxies of DRAM power efficiency such as DRAM bandwidth and sample hit length. The threshold ratio that has the lowest DRAM power usage when running workloads of interest can then be set as the threshold ratio. Analysis of workloads on a graphics processing unit (GPU) has shown that disabling write gathering when the dirty eviction ratio to lookup count is less than 10% provides good results.

[0047] Similarly, a preferred value for the lookup counter threshold can be determined via experimentation or can be set directly, based on the cache size.

[0048] In further embodiments, the threshold ratio is determined automatically. In one approach, specific feedback from the cache, such as monitoring the eviction of dirty lines that have been cleaned, is used to adjust the threshold. Optionally, a cache line has an extended sideband or metadata to store an indicator of whether the line had been gather-cleaned. If an evicted the line was gather-cleaned and the evicted line is dirty, the setting for the threshold ratio is reduced by some amount. If, after some period, no dirty evicted lines have been gather-cleaned, the threshold ratio is increased by some amount.

[0049] In another approach, direct or indirect feedback from DRAM power utilization is used and the impact of variations in the threshold ratio is monitored. Periodically, the threshold ratio is randomly adjusted (up or down) and DRAM power usage (or proxies thereof) are used to determine if the previous or new setting is more power efficient. If the previous setting was more power efficient then it is restored and the probability of adjusting the setting in the direction of the regressing setting reduced, thus reducing the likelihood of selecting it again.

[0050] In one embodiment, determining the ratio of dirty cache evictions to cache lookup requests for the cache includes determining a number of dirty cache evictions in a time period from a total number of cache evictions in the time period.

[0051] In one embodiment, identifying, in the cache, the additional cache lines to be cleaned to the row of the backing memory includes identifying, from an address tag of the first cache line, the row of the backing memory to which the first cache line will be cleaned, and searching the cache, based on address tags and cache line status bits, to find valid dirty cache lines to be cleaned to the identified row of the backing memory.

[0052] FIG. 8 is a flow chart 800 of a method of cleaning a cache to a backing memory, in accordance with various representative embodiments. In these embodiments, an eviction priority value is determined for each cache line. In one embodiment, replacement policy values are updated in response to cache line operations (e.g., installs, hits and accesses to a cache set). The eviction priority can be determined directly or indirectly from the replacement policy values . Write gathering is performed dependent upon the eviction priority values. Searching of the cache is restricted to valid dirty cache lines for which the eviction priority value of the cache line exceeds a threshold value. The method includes identifying, in the cache, a first cache line to be cleaned to a row of the backing memory at block 802. The first cache line is cleaned at block 804 by writing back to the backing memory. When write gathering is not enabled, as depicted by the negative branch from decision block 806, no write gathering is performed, and flow returns to block 802.

[0053] When write gathering is enabled, as depicted by the positive branch from decision block 806, the row in the backing memory that the line will be written to is identified at block 808. An additional cache line in the cache to be cleaned to the same row of the backing memory is identified at block 810. When the eviction priority value of the identified line exceeds a threshold value, as depicted by the positive branch from decision block 812, the identified line is cleaned to the backing memory at block 814. Alternatively, an identifier of the list is added to list of gathered lines to be written back at a later time. When the eviction priority value of the identified line does not exceed the threshold value, as depicted by the negative branch from decision block 812, the line is not written back. If enough lines have been gathered, searched, or written back, as depicted by the negative branch from decision block 816, flow returns to block 802. Otherwise, as depicted by the positive branch from decision block 816, flow returns to block 810 and search of the cache continues.

[0054] In one embodiment of the method, the eviction priority value is a Re-Reference Prediction Value (RRPV) for each cache line of the cache and searching the cache is restricted to valid dirty cache lines for which the RRPV exceeds a threshold value.

[0055] In a further embodiment of the method, a use classification and age are stored for each cache line in a cache set. The stored use classification and age for each cache line are updated in response to installs, hits and accesses to the cache set. An eviction priority table is accessed, based on the stored use classifications and ages for cache lines in a cache set, to determine an eviction priority value. Searching the cache is restricted to cache lines for which the eviction priority value exceeds a threshold value.

[0056] In some embodiments, the first cache line to be cleaned is a valid dirty cache line that has been selected to be evicted from the cache.

[0057] In some embodiments the first cache line to be cleaned may be identified by determining when a memory bus is idle and identifying a valid dirty cache line with a highest eviction priority.

[0058] In some embodiments, identifying, in the cache, additional cache lines to be cleaned to the row of the backing memory includes identifying the backing memory from a plurality of backing memories based on an address tag of the first address tag.

[0059] The various embodiments described above may be used separately or in combination without departing from the present disclosure.

[0060] FIG. 9 is a flow chart 900 of a method of cleaning a cache to a backing memory in accordance with various representative embodiments. The method includes identifying, in the cache, a first cache line to be cleaned to a row of the backing memory at block 902. When write gathering is not enabled, as depicted by the negative branch from decision block 904, the first cache line is cleaned at block 906 by writing back to the backing memory. When write gathering is enabled, as depicted by the positive branch from decision block 904, the row in the backing memory that the line will be written to is identified at block 908. An additional cache line in the cache to be cleaned to the same row of the backing memory is identified at block 910.

[0061] When the eviction priority value of the identified line exceeds a threshold value, as depicted by the positive branch from decision block 912, the identified line is added at block 914 to list of candidate lines to be written back at a later time. When the eviction priority value of the identified line does not exceed the threshold value, as depicted by the negative branch from decision block 912, the line is not written back. If more cache lines are to be gathered, as depicted by the positive branch from decision block 916, flow returns to block 910 and search of the cache continues. If enough lines have been gathered or searched, as depicted by the negative branch from decision block 916, flow continues to decision block 918. If the number of gathered lines in the candidate list exceeds a threshold number, as depicted by the positive branch from decision block 918, the first cache line and the gathered lines in the candidate list are cleaned at block 920 by writing to the backing memory. Otherwise, as depicted by the negative branch from decision block 918, flow continues to block 906 and only the first cache line is cleaned to memory.

[0062] In this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by “comprises .. .a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0063] Reference throughout this document to “one embodiment,” “certain embodiments,” “an embodiment,” “implementation(s),” “aspect(s),” or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of such phrases or in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0064] The term “or,” as used herein, is to be interpreted as an inclusive or meaning any one or any combination. Therefore, “A, B or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B and C ” An exception to this definition will occur only when a combination of elements, functions, steps, or acts are in some way inherently mutually exclusive.

[0065] As used herein, the term “configured to,” when applied to an element, means that the element may be designed or constructed to perform a designated function, or that is has the required structure to enable it to be reconfigured or adapted to perform that function.

[0066] Numerous details have been set forth to provide an understanding of the embodiments described herein. The embodiments may be practiced without these details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid obscuring the embodiments described. The disclosure is not to be considered as limited to the scope of the embodiments described herein.

[0067] Those skilled in the art will recognize that the present disclosure has been described by means of examples. The present disclosure could be implemented using hardware component equivalents such as special purpose hardware and / or dedicated processors which are equivalents to the present disclosure as described and claimed. Similarly, dedicated processors and / or dedicated hard wired logic may be used to construct alternative equivalent embodiments of the present disclosure.

[0068] Dedicated or reconfigurable hardware components used to implement the disclosed mechanisms may be described, for example, by instructions of a hardware description language (HDL), such as VHDL, Verilog or RTL (Register Transfer Language), or by a netlist of components and connectivity. The instructions may be at a functional level or a logical level or a combination thereof. The instructions or netlist may be input to an automated design or fabrication process (sometimes referred to as high-level synthesis) that interprets the instructions and creates digital hardware that implements the described functionality or logic.

[0069] The HDL instructions or the netlist may be stored on non-transitory computer readable medium such as Electrically Erasable Programmable Read Only Memory (EEPROM); non-volatile memory (NVM); mass storage such as a hard disc drive, floppy disc drive, optical disc drive; optical storage elements, magnetic storage elements, magnetooptical storage elements, flash memory, core memory and / or other equivalent storage technologies without departing from the present disclosure. Such alternative storage devices should be considered equivalents.

[0070] Concepts described herein may be embodied in computer-readable code for fabrication of an apparatus that embodies the described concepts. For example, the computer-readable code can be used at one or more stages of a semiconductor design and fabrication process, including an electronic design automation (EDA) stage, to fabricate an integrated circuit comprising the apparatus embodying the concepts. The above computer-readable code may additionally or alternatively enable the definition, modelling, simulation, verification and / or testing of an apparatus embodying the concepts described herein.

[0071] For example, the computer-readable code for fabrication of an apparatus embodying the concepts described herein can be embodied in code defining a hardware description language (HDL) representation of the concepts. For example, the code may define a register-transfer-level (RTL) abstraction of one or more logic circuits for defining an apparatus embodying the concepts. The code may define an HDL representation of the one or more logic circuits embodying the apparatus in Verilog, Sy stem Verilog, Chisel, or VHDL (Very High-Speed Integrated Circuit Hardware Description Language) as well as intermediate representations such as FIRRTL. Computer-readable code may provide definitions embodying the concept using system-level modelling languages such as SystemC and SystemVerilog or other behavioral representations of the concepts that can be interpreted by a computer to enable simulation, functional and / or formal verification, and testing of the concepts.

[0072] Additionally, or alternatively, the computer-readable code may define a low-level description of integrated circuit components that embody concepts described herein, such as one or more netlists or integrated circuit layout definitions, including representations such as GDSII. The one or more netlists or other computer-readable representation of integrated circuit components may be generated by applying one or more logic synthesis processes to an RTL representation to generate definitions for use in fabrication of an apparatus embodying the invention. Alternatively, or additionally, the one or more logic synthesis processes can generate from the computer-readable code a bitstream to be loaded into a field programmable gate array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for the purposes of verification and test of the concepts prior to fabrication in an integrated circuit or the FPGA may be deployed in a product directly.

[0073] The computer-readable code may comprise a mix of code representations for fabrication of an apparatus, for example including a mix of one or more of an RTL representation, a netlist representation, or another computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus embodying the invention. Alternatively, or additionally, the concept may be defined in a combination of a computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus and computer-readable code defining instructions which are to be executed by the defined apparatus once fabricated.

[0074] Such computer-readable code can be disposed in any known transitory computer-readable medium (such as wired or wireless transmission of code over a network) or non-transitory computer-readable medium such as semiconductor, magnetic disk, or optical disc. An integrated circuit fabricated using the computer-readable code may comprise components such as one or more of a central processing unit, graphics processing unit, neural processing unit, digital signal processor or other components that individually or collectively embody the concept.

[0075] Various embodiments described herein are implemented using dedicated hardware, configurable hardware or programmed processors executing programming instructions that are broadly described in flow chart form that can be stored on any suitable electronic storage medium or transmitted over any suitable electronic communication medium. A combination of these elements may be used. Those skilled in the art will appreciate that the processes and mechanisms described above can be implemented in any number of variations without departing from the present disclosure. For example, the order of certain operations carried out can often be varied, additional operations can be added, or operations can be deleted, without departing from the present disclosure. Such variations are contemplated and considered equivalent.

[0076] The various representative embodiments, which have been described in detail herein, have been presented by way of example and not by way of limitation. It will be understood by those skilled in the art that various changes may be made in the form and details of the described embodiments resulting in equivalent embodiments that remain within the scope of the appended claims.

Claims

1. A method of cleaning a cache to a backing memory comprising:identifying, in the cache, a first cache line to be cleaned to a row of the backing memory;when write gathering is enabled:identifying, in the cache, additional cache lines to be cleaned to the row of the backing memory;when a number of additional cache lines exceeds a threshold number:cleaning the first cache line and the additional cache lines to the backing memory; andwhen the number of additional cache lines does not exceed the threshold number:cleaning the first cache line to the backing memory; andwhen write gathering is not enabled:cleaning the first cache line to the backing memory.

2. The method of claim 1, further comprising:determining a ratio of dirty cache evictions to cache lookup requests for the cache;enabling write gathering when the ratio of dirty cache evictions to cache lookup requests exceeds a threshold ratio; anddisabling write gathering when the ratio of dirty cache evictions to cache lookup requests does not exceed the threshold ratio.

3. The method of claim 2, where determining the ratio of dirty cache evictions to cache lookup requests for the cache includes determining a number of dirty cache evictions in a time period from a total number of cache evictions in the time period.

4. The method of claim 1, where identifying, in the cache, the additional cache lines to be cleaned to the row of the backing memory includes:identifying, from an address tag of the first cache line, the row of the backing memory to which the first cache line will be cleaned; andsearching the cache, based on address tags and cache line status bits, to find valid dirty cache lines to be cleaned to the identified row of the backing memory.

5. The method of claim 4, further comprising:determining an eviction priority value for each cache line; andupdating the eviction priority values in response to cache installations and cache hits;where said searching the cache is restricted to valid dirty cache lines for which the eviction priority value of the cache line exceeds a threshold value.

6. The method of claim 4, further comprising determining a Re-Reference Prediction Value (RRPV) for each cache line of the cache, where said searching the cache is restricted to valid dirty cache lines for which the RRPV exceeds a threshold value.

7. The method of claim 4, further comprising:determining a use classification and age for each cache line in a cache set;updating the use classification and age for each cache line in response to installations and hits to the cache set; andaccessing an eviction priority table, based on the use classifications and ages for cache lines in a cache set, to determine an eviction priority value,where said searching the cache is restricted to cache lines for which the eviction priority value exceeds a threshold value.

8. The method of claim 1, where identifying the first cache line to be cleaned includes identifying a valid dirty cache line to be evicted from the cache.

9. The method of claim 1, where identifying the first cache line to be cleaned includes: determining when a memory bus is idle or operating at less than full capacity; and identifying a valid dirty cache line with a highest eviction priority.

10. The method of claim 1, identifying, in the cache, additional cache lines to be cleaned to the row of the backing memory includes identifying the backing memory from a plurality of backing memories based on an address tag of the first address tag.

11. A cache comprising:a data memory array for storing a plurality of cache lines;a metadata memory array for storing address tags and status bits associated with the plurality of cache lines;a cache controller configured to move the cache lines between the data memory array and a backing memory and to move cache line data between the data memory array and a data processor, the cache controller including:cleaning logic circuitry configured to:identify, in the cache, a first cache line to be cleaned to a row of the backing memory;when write gathering is enabled:identify, in the cache, additional cache lines to be cleaned to the row of the backing memory;when a number of additional cache lines exceeds a threshold number:clean the first valid dirty cache and the additional cache lines to the backing memory; andwhen the number of additional cache lines does not exceed the threshold number:clean the first cache line to the backing memory; andwhen write gathering is not enabled:clean the first cache line to the backing memory.

12. The cache of claim 11, where identifying additional cache lines to be cleaned to the row of the backing memory includes searching the metadata memory array cache line addresses in one or more address regions adjacent to an address of the first cache line.

13. The cache of claim 11, where the cleaning logic circuitry is further configured to:determine a ratio of cache evictions to cache lookup requests for the cache;enable write gathering when the ratio of cache evictions to cache lookup requests exceeds a threshold ratio; anddisable write gathering when the ratio of cache evictions to cache lookup requests does not exceed the threshold ratio.

14. The cache of claim 11, where determining the ratio of dirty cache evictions to cache lookup requests for the cache includes determining a number of dirty cache evictions in a time period from a total number of cache evictions in the time period.

15. The cache of claim 11, where identifying, in the cache, the additional cache lines to be cleaned to the row of the backing memory includes:identifying, from an address tag of the first cache line, the row of the backing memory to which the first cache line will be cleaned; andsearching the metadata memory array of the cache, based on address tags and cache line status bits, to find valid dirty cache lines to be cleaned to the identified row of the backing memory.

16. The cache of claim 15, where the cache controller includes eviction control circuitry configured to:determine an eviction priority value for each cache line; andupdate the eviction priority values in response to cache installations and cache hits, where said searching the cache is restricted to valid dirty cache lines for which the eviction priority value of the cache line exceeds a threshold value.

17. The cache of claim 15, where the cache controller includes eviction control circuitry configured to:determine a Re-Reference Prediction Value (RRPV) for each cache line of the cache,where said searching the cache is restricted to valid dirty cache lines for which the RRPV exceeds a threshold value.

18. The cache of claim 15, where the cache controller includes eviction control circuitry configured to:determine a use classification and age for each cache line in a cache set;update the use classification and age for each cache line in response to installations and hits to the cache set; andaccess an eviction priority table, based on the use classifications and ages for cache lines in a cache set, to determine an eviction priority value;where said searching the cache is restricted to cache lines for which the eviction priority value exceeds a threshold value.

19. A non-transitory computer-readable medium to store computer-readable code for fabrication of a cache comprising:a data memory array for storing a plurality of cache lines;a metadata memory array for storing address tags and status bits associated with the plurality of cache lines;a cache controller configured to move the cache lines between the data memory array and a backing memory and to move cache line data between the data memory array and a data processor, the cache controller including:cleaning logic circuitry configured to:identify, in the cache, a first cache line to be cleaned to a row of the backing memory;when write gathering is enabled:identify, in the cache, additional cache lines to be cleaned to the row of the backing memory;when a number of additional cache lines exceeds a threshold number:clean the first valid dirty cache and the additional cache lines to the backing memory; andwhen the number of additional cache lines does not exceed the threshold number:clean the first cache line to the backing memory; andwhen write gathering is not enabled:clean the first cache line to the backing memory.

Images