System and method for cache management

Abstract

The invention discloses a cache management system and method. In some embodiments, a system includes: a lower level cache; a first superior cache; and a second superior cache, the subordinate cache, the first superior cache, and the second superior cache being configured to store first metadata of a cache line in the first superior cache, the first metadata indicating that the cache line is in a partial state, the partial state indicates that at least two words in the cache line are in different permission states.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Application No. 63 / 650,775, filed May 22, 2024, entitled “WORD GRANULARITY CACHECOHERENCE FOR EFFICIENT FALSE SHARING,” and U.S. Non-Provisional Application No. 18 / 967,595, filed December 3, 2024, entitled “SYSTEM AND METHOD FOR CACHE MANAGEMENT,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] One or more aspects of embodiments of this disclosure relate to computing systems, and more specifically to cache managers. Background Technology

[0004] In computing systems, caching can be used to improve performance. For example, a computing system may include an upper-level cache closest to the system core, and lower-level caches. Lower-level caches can act as backing stores for the upper-level cache and can use another lower-level cache or main memory as their backing store. One or more caches can be shared; for example, a lower-level cache can also act as a backing store for an upper-level cache in another core of the system.

[0005] The aspects of this disclosure are relevant to this general technical environment. Summary of the Invention

[0006] According to embodiments of this disclosure, a system is provided, the system comprising: a lower-level cache; a first upper-level cache; and a second upper-level cache, the lower-level cache, the first upper-level cache, and the second upper-level cache being configured to: store first metadata of a cache line in the first upper-level cache, the first metadata indicating that the cache line is in a partial state, the partial state indicating that at least two words in the cache line are in different permission states, the first metadata further indicating that a first word in the cache line is in a readable or writable state in the first upper-level cache; store second metadata of the cache line in the second upper-level cache, the second metadata indicating that the cache line is in a partial state, the second metadata further indicating that a second word in the cache line is in a writable state in the second upper-level cache; and store third metadata of the cache line in the lower-level cache, the third metadata indicating that the cache line is in a partial state.

[0007] In some embodiments, the first metadata indicates that the first word in the cache line is in a readable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being cleared.

[0008] In some embodiments, the first metadata indicates that the first word in the cache line is writable, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being set.

[0009] In some embodiments, the third metadata also includes a list of writers for the second word, the list of writers including the second parent cache.

[0010] In some embodiments, the first metadata indicates that the first word in the cache line is readable, and the third metadata also includes a read-only parent cache list for the cache line, the read-only parent cache list including the first parent cache.

[0011] In some embodiments, the read-only parent cache list includes a first parent cache indication: the first parent cache has read access to each word of the cache line that is not writable to another parent cache.

[0012] In some embodiments, the lower-level cache, the first upper-level cache, and the second upper-level cache are further configured to modify the data of the second word in the cache line in the second upper-level cache.

[0013] In some embodiments, the lower-level cache, the first upper-level cache, and the second upper-level cache are further configured to modify the second metadata to indicate that the second word in the cache line is in a modified state.

[0014] In some embodiments, modifying the second metadata includes clearing the read bit corresponding to the second word in the cache line and setting the write bit corresponding to the second word in the cache line.

[0015] In some embodiments, the first metadata also indicates that the second word in the cache line is invalid in the first parent cache.

[0016] In some embodiments, the first metadata includes a read bit corresponding to the second word in the cache line, the read bit being cleared, and the first metadata includes a write bit corresponding to the second word in the cache line, the write bit being cleared.

[0017] In some embodiments, the lower-level cache is configured to store a copy of the second word.

[0018] In some embodiments, the third metadata is stored in the directory of the lower-level cache.

[0019] According to embodiments of this disclosure, a method is provided, comprising: storing first metadata of a cache line in a first parent cache, the first metadata indicating that the cache line is in a partial state, the partial state indicating that at least two words in the cache line are in different permission states, the first metadata further indicating that a first word in the cache line is in a readable state or a writable state in the first parent cache; storing second metadata of the cache line in a second parent cache, the second metadata indicating that the cache line is in a partial state, the second metadata further indicating that a second word in the cache line is in a writable state in the second parent cache; and storing third metadata of the cache line in a lower cache, the third metadata indicating that the cache line is in a partial state.

[0020] In some embodiments, the first metadata indicates that the first word in the cache line is in a readable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being cleared.

[0021] In some embodiments, the first metadata indicates that the first word in the cache line is writable, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being set.

[0022] In some embodiments, the third metadata also includes a list of writers for the second word, the list of writers including the second parent cache.

[0023] In some embodiments, the first metadata indicates that the first word in the cache line is readable, and the third metadata also includes a read-only parent cache list for the cache line, the read-only parent cache list including the first parent cache.

[0024] In some embodiments, the read-only parent cache list includes a first parent cache indication: the first parent cache has read access to each word of the cache line that is not writable to another parent cache.

[0025] According to embodiments of this disclosure, a method is provided, comprising: receiving, from a first upper-level cache, a request for ownership of a data word at a first address, the first address being within a cache line, the cache line being shared in the lower-level cache, the cache line having a set of read-only sharers including a second upper-level cache, the cache line being shared in the second upper-level cache; invalidating the first address in the second upper-level cache; and maintaining a read-only state in the second upper-level cache for a second address in the cache line that is different from the first address. Attached Figure Description

[0026] These and other features and advantages of this disclosure will be appreciated and understood by referring to the specification, claims and drawings, wherein:

[0027] Figure 1A This is a block diagram of a computing system according to an embodiment of the present disclosure;

[0028] Figure 1B This is a block diagram of a host according to an embodiment of the present disclosure;

[0029] Figure 1C This is a block diagram of a central processing unit according to an embodiment of the present disclosure;

[0030] Figure 2A This is a sequence diagram of cache operations according to embodiments of the present disclosure;

[0031] Figure 2B This is a sequence diagram of cache operations according to embodiments of the present disclosure;

[0032] Figure 2C This is a sequence diagram of cache operations according to embodiments of the present disclosure;

[0033] Figure 3 This is a flowchart based on an embodiment of the present disclosure; and

[0034] Figure 4 This is a flowchart of an embodiment according to the present disclosure. Detailed Implementation

[0035] The detailed description set forth below with reference to the accompanying drawings is intended as a description of exemplary embodiments of the systems and methods for cache management provided in accordance with this disclosure, and is not intended to represent the only forms in which this disclosure can be constructed or utilized. This description illustrates features of the disclosure in conjunction with the illustrated embodiments. However, it should be understood that the same or equivalent functionality and structure may be implemented through different embodiments, which are also intended to be included within the scope of this disclosure. As indicated elsewhere herein, the same element numbers are intended to indicate the same elements or features.

[0036] Figure 1AThis is a system-level block diagram of a computing system in some embodiments. Each of a plurality of hosts 105 is connected to other hosts 105 of the computing system via network 110 and to shared resources such as persistent storage 115. Figure 1B This is a system-level block diagram of host 105 (or "node") in some embodiments. A central processing unit 120 (which may be processing circuitry and may be fabricated on a single silicon chip) is connected to main memory 125, persistent storage 115, and a network interface (e.g., a network interface card (NIC)) 130 serving as an interface to network 110. In embodiments including external persistent storage 115 (such as...) Figure 1A In the embodiment shown, persistent storage 115 may be absent or may have a limited capacity. Figure 1C This is a block diagram of CPU 120 in some embodiments. CPU 120 includes multiple cores 135, each core having a corresponding upper-level cache 140. The upper-level cache 140 shares the lower-level cache 145. The components of CPU 120 can be connected together via a network on-chip (NoC) 150 as shown.

[0037] Data can be stored in and retrieved from the cache in units that may be referred to as "cache lines" or "blocks". Each cache line may include, for example, 64 bytes, or more or fewer bytes. Each cache line may contain multiple words, which can be accessed individually via kernel instructions. Each such word may be smaller than a cache line; for example, a word may be four bytes long.

[0038] However, if caches and directories operate at the granularity of cache lines, disposing of data in larger units of cache lines can lead to inefficiency. For example, some invalid cache consistency schemes (such as Modify, Exclusive, Shared, Invalid (MESI) cache consistency schemes, which use these four states) may allow only one core to modify a cache line at any given time.

[0039] Such a scheme can also invalidate all read-only copies before granting write permission to the requester, and can transfer full ownership of the entire cache line between the reader and the writer as needed.

[0040] Therefore, some cache coherence schemes track modifications at the cache line granularity. In such a scheme, if one core 135 is writing any word in a cache line, other cores 135 are not permitted to read or write other words in that cache line at the same time. A core 135 attempting to modify a different word in a cache line (in what might be called a pseudo-shared approach) can be asked to sniff out a copy of the entire cache line in its peer's cache and invalidate it.

[0041] Two cores that continuously write different words within a cache line can "ping-pong" between them to transfer cache line ownership (e.g., repeatedly transferring cache line ownership back and forth). The process of migrating cache blocks and managing permissions introduces latency and is unnecessary in the case of false sharing, but is required for using a single consistent state for the entire cache line.

[0042] Therefore, some embodiments of the present invention relate to word-level tracking of writeability and readability to mitigate the false-sharing problem in directory-based hardware cache consistent nodes. Word-level tracking of writability and readability enables multiple cores (e.g., for multiple core-level caches) to efficiently access individual words in a cache line, wherein the cache line is maintained in multiple higher-level caches with different permission states for different words.

[0043] In some embodiments, this is achieved by extending the Modified, Exclusive, Shared, Invalid (MESI) cache consistency scheme with a "P" state that indicates partial read / write permission. When a cache line is in the partial (P) state, additional bits of metadata can be used to indicate the state of the word in the cache line. For example, two bits of metadata can be used for this purpose for each word in a cache line in the highest-level cache (e.g., the level closest to core 135).

[0044] The first of these two bits can be called the read bit, and it can be used to indicate whether the word is valid for reading from that position (when the word is writable, this bit can also indicate whether the word has been modified). The second of these two bits can be called the write bit, and it can be used to indicate whether the word is writable. If the line is in any of the following states: modified, exclusive, shared (or "read-only"), or invalid, these bits can be ignored except when the cached line is in the M state, in which case the write bit can be updated.

[0045] When a cache line is in a partial (P) state, the lower-level cache tracks which higher-level caches have write permissions for each word. For example, a lower-level cache may maintain a list of writers for each word in a cache line. For any cache line with multiple writers, cache inclusion can be used (e.g., a lower-level cache may maintain a copy of the data stored in the higher-level cache). On-chip network (NoC) transactions can carry word addresses and word enable to support this functionality.

[0046] In such a system, performance may degrade if a parent cache has cache lines in a shared state and another parent cache attempts to write a series of words, because each storage operation (executed by core 135 and causing that parent cache to send an ownership request transaction to the lower cache) can generate a new ownership request transaction to write the (previously unwritten) word.

[0047] Thus, it may be advantageous to distinguish between (i) performing a conversion from shared to partial and (ii) invalid, thereby allowing the writer (the upper-level cache seeking the data to be written) to convert the entire cache line to a modified state.

[0048] In a read scenario, if the cache line is held in the modified (M) state and core 135 performs a read of an unmodified word, then core 135 receives read access to all unwritten words (in the partial (P) state). If core 135 requests a written word, the writer's copy is invalidated, the modified data is merged into the next level of cache, and if no other writer exists, the cache line can transition to a shared state.

[0049] In other words, a read transaction can always result in the reader gaining permission to read all words that are not kept writable in other caches. Conversely, if cache lines are kept in a shared or modified state in other caches, an ownership request can only result in the writer gaining permission one word at a time.

[0050] In some embodiments, if core 135 attempts to write two consecutive words (e.g., if the upper cache of core 135 attempts to write two consecutive words), the lower cache may assume that the upper cache of core 135 will continue to write more words, and may therefore change the entire cache line to a modified state and invalidate the sharer.

[0051] The determinations made and actions taken by lower-level caches can also depend on the position of the word in the cache line. If the first request is in progress, it may be possible to merge the requests and transition directly to the modified state without transitioning through the partial state. Two separate writes to consecutive words can result in a transition to the M state, regardless of the size of each write, as long as they remain within a single cache line. For example, in the case of independent requests, an eight-byte write can function as a four-byte write.

[0052] In some embodiments, more sophisticated heuristics and schemes may be employed, such as methods for predicting which addresses are likely to be read or written. In some embodiments, a different heuristic is used for each cache line or each set of cache lines.

[0053] In some embodiments, a lower-level cache will only cause a higher-level cache to transition to a partial (P) state if another higher-level cache has a cache line in a shared (S) state or a modified (M) state.

[0054] A transition from the shared (S) state to the partial (P) state can occur when one or more read-only sharers have cache lines in the shared (S) state and the core attempts to write to them.

[0055] A transition from the Modified (M) state to the Partial (P) state can occur when a cache has a cache line in the Modified (M) state and a reader or writer requests words that have not yet been written. In this case, the write bits can be updated even when the entire cache line is in the Modified state to determine which words have been written. A lower-level cache can send a sniff to a higher-level cache that has a cache line in the Modified state, specifying which words it needs in the Modified or Exclusive state. Based on its current state and the transactions received, the higher-level cache controller can update its state and respond to the lower-level cache accordingly. The receiver of the sniff can determine whether to flush its copy or transition to P. In this case, the behavior of the receiver of the sniff can be configurable.

[0056] When core 135 writes consecutive words to other sharers, a transition from the partial (P) state to the modified (M) state can occur. This can be done as part of mitigating the aforementioned performance degradation.

[0057] A transition from the Partial (P) state to the Modified (M) state can also occur when core 135 issues a store instruction to a word already written to another cache and there are no other writers or sharers. In this case, one of two options can be used. In the first option, if no other words have been written to the first cache, it can flush the cache line and give it to the new writer in the Modified state. In the second option, the first cache can remain in the Partial state and can remove writability only for the word to which core 135 issued a store instruction. The choice between these two options can be controlled through configuration.

[0058] A transition from the Partial (P) state to the Shared (S) state can also occur when core 135 requests read access to a word marked as written in another cache, and no other cache has a write bit set for any other word in that cache line. If the writer has written other words (e.g., if it set a write bit for another word in the cache line), one of two options is available. In the first option, the writer can choose to release only the writability of the requested word and keep that cache line in P, thus allowing readability for all unwritten words. In the second option, the writer can write back the data and invalidate its copy.

[0059] If the requested word is not marked as written, the writer can keep the cache line in a partial state, and the reader can be authorized to access all unwritten words in the partial state.

[0060] If multiple writers leave a cache line in a partially state, then lower-level caches that know which upper-level caches have set the write bit for the word can only send probes to those caches. Then, if other written words remain unchanged, the reader can get the cache line in a partially state.

[0061] The specific rules for transitioning to and from partial states can vary. Different cache lines or cache sets may use different transition criteria. This disclosure does not specify any particular conditions, but only provides the above conditions as examples.

[0062] In some embodiments, hybrid cache inclusion is employed, where some writes can be merged at the next lower cache level. If only one writer exists for a cache line, the lower-level cache does not need to save a copy, and the writer has the complete cache line. If multiple writers exist (e.g., if no writer leaves the entire cache line in an exclusive or modified state), the lower level can save a copy for merging.

[0063] In some embodiments, word-granularity tracking is hierarchical. Each lower cache level (farther from the core) tracks the sharers and writers of words in the cache lines of the higher-level cache (closer to the core).

[0064] For example, an L3 cache can track which L2 caches share a cache line and which have write permissions for a specific word. A lower-level cache (e.g., L3) can maintain a list of readers (read-only sharers) for the entire cache line—excluding any words for which the upper-level cache has write permissions. For each upper-level cache, a single bit can be used to indicate the reader. Lower-level caches can also track which upper-level caches have write permissions for each word in the cache line. For each word in the cache line, that permission can be tracked using a single field that identifies the upper-level cache.

[0065] Word granularity can be a design-time decision. Granularity can be as small as 1 byte (to match the minimum write size) or as large as half a cache line. Granularity can be chosen based on a trade-off between pseudo-share efficiency and cache overhead. In one embodiment, granularity can be four bytes or eight bytes to match the most common load and store instruction sizes. Pseudo-share can be performed within a word, but this may result in a reversion to normal MESI efficiency.

[0066] Some implementations may use a network on-chip (NoC) that supports transactions with granularity smaller than cache lines.

[0067] Figure 2A This illustrates word-granular cache consistency operations in one example. Figure 2AIn the middle, the first core 135 (core B) executes the store (ST) instruction, and thus issues a claim of ownership (RFO) transaction at address 202 to the first address (address X+1).

[0068] The address (X+1) is in the same cache line as the second address (address X) to which the second core (core A) previously issued a read (Rd) transaction at 204 (as a result of executing a load (LD) instruction). The issuance of the corresponding read transaction by the second core resulted in a cache miss (when the second upper-level cache (L1-A) determines that the cache line containing address X is in an invalid (I) state), which causes the second upper-level cache (L1-A) to send a transaction to the lower-level cache (L2) at 206. In response, the lower-level cache (L2) (i) sends the transaction containing the cache line at address X to the second upper-level cache (L1-A) at 208, sets the state of the cache line in the lower-level cache (L2) to shared, and adds the second upper-level cache (L1-A) to the list of read-only sharers (RO sharers) of the cache line. As a result of receiving the transaction sent at 208, the state of the cache line in the second upper-level cache (L1-A) is changed to shared (S).

[0069] exist Figure 2A In China (and in Figure 2B and 2C (In the middle), the status indicator immediately adjacent to the vertical line indicates the cache status information maintained in that cache line.

[0070] At position 202, a Request for Ownership (RFO) transaction sent by the first core results in a cache miss in the first upper-level cache (L1-B), and at position 210, an RFO transaction is sent from the first upper-level cache (L1-B) to the lower-level cache (L2). The lower-level cache (L2) then modifies the state of that cache line in its lower-level cache (L2) to a partially state and adds the first upper-level cache (L1-B) to the writer list maintained by the lower-level cache (L2) for that cache line, where the first upper-level cache (L1-B) is identified as the writer of only the word at address X+1. This... Figure 2A The label “writer: 1:B” indicates that the first parent cache (cache L1-B) is the writer of word 1 of the cache line (where the word at address X+n is the nth word of the cache line).

[0071] The lower-level cache (L2) then sends a transaction at 212 to the second-level cache (L1-A) that invalidates the first address (i.e., address X+1) in the second-level cache (L1-A), and the second-level cache replies with an acknowledgment (Ack). Invalidating the first address in the second-level cache has the following effects: (i) it transitions the cache line from a shared (S) state to a partial (P) state in the second-level cache; (ii) it clears (e.g., sets to 0) the read bits corresponding to the first address (X+1) in the second-level cache (making the read bit vector 1101, as shown); and (iii) it clears the write bit vector corresponding to each word in the cache line in the second-level cache (making the write bit vector 0000, as shown).

[0072] Before receiving the transaction sent at 212, the shared (S) state of this cache line in the second parent cache can be considered equivalent to the partial (P) state, in which the vector of read bits is 1111 and the vector of write bits is 0000. Therefore, this can be the initial state from which the clearing of read bits and the setting of write bits to 1 are performed.

[0073] As used herein, a “set” bit means (i) changing its value to 1 (or keeping its value at 1 if it is already 1) or (ii) changing its value to 0 (or keeping its value at 0 if it is already 0), and a “clear” bit means the opposite of a set bit (e.g., if a set bit means changing its value to 1 (or keeping its value at 1 if it is already 1), then a clear bit means changing its value to 0 (or keeping its value at 0 if it is already 0). The examples in this disclosure treat a “set” bit as meaning changing its value to 1 (or keeping its value at 1 if it is already 1), and a “clear” bit as meaning changing its value to 0 (or keeping its value at 0 if it is already 0)).

[0074] In response to receiving an acknowledgment (Ack), the lower-level cache can send a cache line transaction with write enable to the first upper-level cache at 214, which causes the first upper-level cache to set its read bit vector to 1111 and its write bit vector to 0010 (indicating that the first upper-level cache has permission to write word 1 of the cache line and permission to read all words).

[0075] Then, the second core executes a store (ST) instruction at the third address (address X+2), causing a request for ownership (RFO) transaction for the word at address X+2 to be sent at address 216. This causes the lower-level cache to invalidate the word at address X+2 in the first-level cache at address 218. Invalidating the third address in the first-level cache has the effect of (i) clearing the read bits of the third address in the first-level cache (making the read bit vector 1011, as shown in the figure) and clearing the write bits of the third address in the first-level cache (making the write bit vector 0010, as shown in the figure). Then, an acknowledgment is sent from the first-level cache to the lower-level cache, and an acknowledgment is sent from the lower-level cache to the second-level cache, causing the second-level cache to set the write bits of the second word, so that the write bit vector in the second-level cache is 0100.

[0076] Figure 2B It shows the relationship with Figure 2A In a similar example, besides executing the store instruction instead of the second core, the first core writes the new value of the first word (equivalent to word 1 mentioned above) to the first upper cache at position 220. This causes the first upper cache to set the first bit of its read bit vector to 0, indicating that the first word has been modified in the first upper cache. Figure 2A and Figure 2B The examples illustrate four possible combinations of read and write bit values ​​corresponding to the following states: (i) a read bit with a value of 1 and a write bit with a value of 0 correspond to a shared state; (ii) a read bit with a value of 0 and a write bit with a value of 0 correspond to an invalid state; (iii) a read bit with a value of 1 and a write bit with a value of 1 correspond to an exclusive state; and (iv) a read bit with a value of 0 and a write bit with a value of 1 correspond to a modified state. As used herein, the "writable" state is either an exclusive or modified state, and the "readable" state is a shared state.

[0077] Figure 2C It shows the relationship with Figure 2A A similar example, except that the second core does not initially execute read instructions (as it does in...). Figure 2A In the example, this is performed at 204, and as a result of the action of the third upper-level cache (L1-C), which is not shown, the cache line is initially in a shared state in the lower-level cache. In this example, the lower-level cache sends an invalid (Inv) transaction to the third upper-level cache at 222, and after the lower-level cache invalidates the word at address X+2 in the first upper-level cache at 218 and receives an acknowledgment, the lower-level cache sends a read bit vector (1101) to the second upper-level cache at 224. Figure 2CThe cache line for "Read Enable" causes the second-level cache to use the received read bit vector and set its read bit vector to 1101. The lower-level cache sends the read bit vector to the second-level cache because without such a send, the second-level cache would not know which words it has read permissions for. In this example, each higher-level cache assumes its write bit vector has been cleared except for any words for which it sent ownership requests.

[0078] Figure 3 Methods for cache management in some embodiments are illustrated. Although Figure 3 Various operations in such a method are illustrated, but the embodiments according to this disclosure are not limited thereto. For example, according to some embodiments, such a method may include additional or fewer operations, or the order of operations may be varied (unless expressly stated or implied otherwise), without departing from the spirit and scope of the embodiments according to this disclosure.

[0079] The method includes receiving, at 305, a request for ownership of a data word at a first address from a first upper-level cache, the first address being within a cache line, the cache line being shared in the lower-level cache, the cache line having a set of read-only sharers in the lower-level cache including a second upper-level cache, the cache line being shared in the second upper-level cache.

[0080] The method further includes: invalidating the first address in the second parent cache at 310; and maintaining a read-only state in the second parent cache for the second address in the cache line that is different from the first address at 315.

[0081] Invalidating the first address in the second parent cache may include: changing the state of the cache line to a partial state in the second parent cache, clearing the read bit corresponding to the first address in the second parent cache, and clearing the write bit corresponding to the first address in the second parent cache.

[0082] Maintaining a read-only state for the second address in the second upper-level cache can include keeping the read bits corresponding to the second address unchanged in the second upper-level cache.

[0083] The method further includes: at 320, adding a first upper-level cache to the list of writers for the first address in the lower-level cache; at 325, changing the state of the cache line to a partial state in the first upper-level cache; and at 330, setting the write bit corresponding to the first address in the first upper-level cache.

[0084] The method also includes modifying the value / data stored at the first address in the first parent cache at position 335.

[0085] The method further includes: at 340, clearing the read bit corresponding to the first address in the first upper-level cache, and at 345, setting the write bit corresponding to the first address in the first upper-level cache.

[0086] Figure 4 Methods for cache management in some embodiments are illustrated. Although Figure 4 Various operations in such a method are illustrated, but the embodiments according to this disclosure are not limited thereto. For example, according to some embodiments, such a method may include additional or fewer operations, or the order of operations may be varied (unless expressly stated or implied otherwise), without departing from the spirit and scope of the embodiments according to this disclosure.

[0087] The method includes: at 405, storing first metadata of a cache line in a first parent cache, the first metadata indicating that the cache line is in a partial state, the partial state indicating that at least two words in the cache line are in different permission states, and the first metadata also indicating that the first word in the cache line is in a readable or writable state in the first parent cache. For example, as Figure 2A As shown, after the lower-level cache sends a cache line transaction with write enable to the first upper-level cache at position 214 (which causes the first upper-level cache to set its read bit vector to 1111 and its write bit vector to 0010 (indicating that the first upper-level cache has permission to write the first word of the cache line and permission to read all words)), the first word of the first upper-level cache (the word at X+1) is in a writable state (exclusive state), and each other word is in a different permission state (shared state).

[0088] The method further includes: at 410, storing second metadata of the cache line in a second parent cache, the second metadata indicating that the cache line is in a partial state, and the second metadata also indicating that the second word in the cache line is writable in the second parent cache. For example, as Figure 2A As shown, the second upper-level cache transitions to a partial state in which the word at address X+2 (which may be referred to as the second word) is writable (e.g., in an exclusive state), and the word at X+1 is invalid. The method also includes storing third metadata of the cache line in the lower-level cache at address 415, the third metadata indicating that the cache line is in a partial state. For example, as... Figure 2A As shown, the lower-level cache has transitioned to a partial state.

[0089] In some embodiments, the first metadata indicates that the first word in the cache line is readable. The first metadata includes a read bit corresponding to the first word in the cache line, which is set, and a write bit corresponding to the first word in the cache line, which is cleared. This is, for example, through... Figure 2AThe corresponding bit settings of the word at address X, which is in a shared state in the first upper-level cache, are used to illustrate this.

[0090] In some embodiments, the first metadata alternatively indicates that a first word in a cache line is writable. The first metadata includes a read bit corresponding to the first word in the cache line, which is set, and a write bit corresponding to the first word in the cache line, which is also set. This is, for example, through... Figure 2A The corresponding bit settings of the word at address X+1, which is in a writable (exclusive) state in the first upper-level cache, are used to illustrate this.

[0091] In some embodiments, the third metadata also includes a list of writers for the second word, the list of writers including a second parent cache. For example, such as Figure 2A As shown, the writer list can include "2:A", which can mean that the word at address X+2 has writer L1-A (the second upper-level cache, cache L1-A, in...). Figure 2A The list of read-only sharers is abbreviated as "A". In some embodiments, the first metadata indicates that the first word in the cache line is readable, and the third metadata also includes a list of read-only parent caches for the cache line, which includes a first parent cache (in embodiments where the first parent cache has read-only access). Including a second parent cache in the list of read-only parent caches can indicate that the second parent cache has read access to each word of the cache line that is not writable by the other parent cache.

[0092] In some embodiments, for example, if multiple parent caches have modification access to different words in a cache line, a lower cache can (e.g., when evicting the cache line or flushing it to memory) merge the modified words into a single modified cache line. To make this possible, the lower cache can store a copy of each modified word. In some embodiments, the metadata of the lower cache is stored in a directory. In other embodiments, the metadata of the lower cache is stored along with a copy of the data from the parent cache.

[0093] As used herein, "a part" of something means "at least some" of that thing, and therefore can mean less than or all of that thing. Thus, "a part" of something includes the whole thing as a special case, i.e., an example where the whole thing is a part of something. As used herein, when a second quantity is "within Y" of a first quantity X, it means that the second quantity is at least XY and at most X+Y. As used herein, when a second number is "within Y%" of a first number, it means that the second number is at least (1-Y / 100) times the first number and at most (1+Y / 100) times the first number. As used herein, the term "or" should be interpreted as "and / or," such that, for example, "A or B" means either "A" or "B" or "A and B."

[0094] The terms “processing circuit” and “means for processing” are used herein to refer to any combination of hardware, firmware, and software employed to process data or digital signals. Processing circuit hardware may include, for example, application-specific integrated circuits (ASICs), general-purpose or special-purpose central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), and programmable logic devices such as field-programmable gate arrays (FPGAs). In processing circuitry, as used herein, each function is performed by hardware configured (i.e., hardwired) to perform said function or by more general-purpose hardware (e.g., a CPU) configured to execute instructions stored in a non-transitory storage medium. Processing circuitry may be fabricated on a single printed circuit board (PCB) or distributed across several interconnected PCBs. Processing circuitry may include other processing circuitry; for example, processing circuitry may include two processing circuits, an FPGA and a CPU, interconnected on a PCB.

[0095] As used herein, when a method (e.g., adjustment) or a first quantity (e.g., a first variable) is referred to as “based on” a second quantity (e.g., a second variable), it means that the second quantity is an input to the method or affects the first quantity. For example, the second quantity may be an input to a function that computes the first quantity (e.g., a unique input or one of several inputs), or the first quantity may be equal to the second quantity, or the first quantity may be the same as the second quantity (e.g., stored in one or more locations that are the same as the second quantity).

[0096] It should be understood that although the terms "first," "second," "third," etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used only to distinguish one element, component, region, layer, or portion from another element, component, region, layer, or portion. Therefore, without departing from the spirit and scope of the inventive concept, the first element, component, region, layer, or portion discussed herein may be referred to as the second element, component, region, layer, or portion.

[0097] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the inventive concept. As used herein, the terms “substantially,” “about,” and similar terms are used as approximate terms rather than terms of degree and are intended to explain the inherent biases of measured or calculated values ​​that will be recognized by one of ordinary skill in the art.

[0098] Expressions such as "at least one of..." modify the entire list of elements when following it, but do not modify individual elements in the list. Furthermore, the use of "may" when describing embodiments of the inventive concept means "one or more embodiments of this disclosure." Additionally, the term "exemplary" is intended to refer to an example or illustration. As used herein, the terms "using," "being used," and "utilizing" can be considered synonymous with the terms "exploiting," "being exploited," and "being used," respectively.

[0099] Any numerical range described herein is intended to include all subranges containing the same numerical precision within the range. For example, a range “1.0 to 10.0” or “between 1.0 and 10.0” is intended to include all subranges between the minimum value 1.0 and the maximum value 10.0 (and including the endpoints), i.e., all subranges between a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0 (and including the endpoints), such as, for example, 2.4 to 7.6. Similarly, a range described as “within 35% of 10” is intended to include all subranges between the minimum value 6.5 (i.e., (1-35 / 100) multiplied by 10) and the maximum value 13.5 (i.e., (1+35 / 100) multiplied by 10) (and including the endpoints), i.e., all subranges between a minimum value equal to or greater than 6.5 and a maximum value equal to or less than 13.5 (and including the endpoints), such as 7.4 to 10.6. Any maximum numerical limit described herein is intended to include all upper limit numerical limits contained herein, and any minimum numerical limit described herein is intended to include all upper limit numerical limits contained herein.

[0100] Although exemplary embodiments of systems and methods for cache management have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it should be understood that systems and methods for cache management constructed in accordance with the principles of this disclosure may be implemented in ways other than those specifically described herein. The invention is also defined in the appended claims and their equivalents.

Claims

1. A system for cache management, the system comprising: a lower level cache; a first upper level cache; and a second upper level cache, the lower level cache, the first upper level cache, and the second upper level cache configured to: store first metadata for a cache line in the first upper level cache, the first metadata indicating that the cache line is in a partial state, the partial state indicating that at least two words in the cache line are in different permission states, the first metadata further indicating that a first word in the cache line is in a readable state or a writable state in the first upper level cache; store second metadata for the cache line in the second upper level cache, the second metadata indicating that the cache line is in the partial state, the second metadata further indicating that a second word in the cache line is in a writable state in the second upper level cache; and store third metadata for the cache line in the lower level cache, the third metadata indicating that the cache line is in the partial state.

2. The system of claim 1, wherein: the first metadata indicates that the first word in the cache line is in the readable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit cleared.

3. The system of claim 1, wherein: the first metadata indicates that the first word in the cache line is in the writable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit set. the third metadata further includes a writer list for the second word, the writer list including the second upper level cache.

5. The system of claim 1, wherein:

4. The system of claim 1, wherein, the first metadata indicates that the first word in the cache line is in the readable state, and the third metadata further includes a read-only upper level cache list for the cache line, the read-only upper level cache list including the first upper level cache. the read-only upper level cache list including the first upper level cache indicates that the first upper level cache has read access to each word of the cache line that is not writable by another upper level cache. the lower level cache, the first upper level cache, and the second upper level cache are further configured to modify data for the second word in the cache line in the second upper level cache.

6. The system of claim 5, wherein, the lower level cache, the first upper level cache, and the second upper level cache are further configured to modify the second metadata to indicate that the second word in the cache line is in a modified state.

7. The system of claim 1, wherein, modifying the second metadata includes clearing a read bit corresponding to the second word in the cache line and setting a write bit corresponding to the second word in the cache line.

8. The system of claim 7, wherein, the first metadata further indicates that the second word in the cache line is in an invalid state in the first upper level cache.

9. The system of claim 8, wherein, 11. The system of claim 10, wherein:

10. The system of claim 1, wherein, ​ ​ the first metadata includes a read bit corresponding to the second word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the second word in the cache line, the write bit being cleared.

12. The system of claim 1, wherein, the lower cache is configured to store a copy of the second word.

13. The system of claim 1, wherein, the third metadata is stored in a directory of the lower cache.

14. A method for cache management, the method comprising: storing first metadata of a cache line in a first upper cache, the first metadata indicating that the cache line is in a partial state, the partial state indicating that at least two words in the cache line are in different permission states, the first metadata further indicating that a first word in the cache line is in a readable state or a writable state in the first upper cache; storing second metadata of the cache line in a second upper cache, the second metadata indicating that the cache line is in a partial state, the second metadata further indicating that a second word in the cache line is in a writable state in the second upper cache; and storing third metadata of the cache line in a lower cache, the third metadata indicating that the cache line is in a partial state.

15. The method of claim 14, wherein: the first metadata indicates that the first word in the cache line is in a readable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being cleared.

16. The method of claim 14, wherein: the first metadata indicates that the first word in the cache line is in a writable state, the first metadata includes a read bit corresponding to the first word in the cache line, the read bit being set, and the first metadata includes a write bit corresponding to the first word in the cache line, the write bit being set.

17. The method of claim 14, wherein, the third metadata further includes a writer list for the second word, the writer list including the second upper cache.

18. The method of claim 14, wherein: the first metadata indicates that the first word in the cache line is in a readable state, and the third metadata further includes a read-only upper cache list for the cache line, the read-only upper cache list including the first upper cache.

19. The method of claim 18, wherein, the read-only upper cache list including the first upper cache indicates that the first upper cache has read access to every word of the cache line that is not writable by another upper cache.

20. A method for cache management, the method comprising: receiving, by a lower cache from a first upper cache, a request for ownership of a data word at a first address, the first address being within a cache line, the cache line being in a shared state in the lower cache, the cache line having a read-only sharer set in the lower cache including a second upper cache, the cache line being in a shared state in the second upper cache; invalidating the first address in the second upper cache; and For a second address in the cache line different from the first address, a read-only state is maintained in the second upper level cache.

Images