Cache management method and device, cache device, electronic device and medium

Abstract

A cache management method, a cache device, a cache management device, an electronic device and a computer readable storage medium. The cache management method is used for a shared cache shared by a plurality of processor cores. The cache management method comprises: allocating a respective near memory bank and a respective far memory bank to each processor core; and for a memory access request of each processor core, preferentially accessing the corresponding near memory bank, and then accessing the corresponding far memory bank. The method divides the shared cache into a near memory bank and a far memory bank, reduces the delay caused by the increase in physical delay due to the size of the shared cache, and improves performance.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a cache management method, a cache device, a cache management apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In multi-core processor design, memory access is a major factor affecting performance. To improve processor performance, cache technology is currently widely used to reduce latency and enhance performance. However, due to chip size limitations, the capacity of the cache within the processor core is limited, only meeting the needs of a portion of memory access operations. Therefore, the idea of ​​adding a larger-capacity cache outside the core has been proposed as a shared storage unit among multiple processing cores, i.e., a shared cache, thereby reducing memory access latency and improving performance. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a cache management method for a shared cache shared by multiple processor cores. The cache management method includes: allocating a local memory bank and a remote memory bank to each processor core; and for memory access requests from each processor core, prioritizing access to the corresponding local memory bank and then accessing the corresponding remote memory bank.

[0004] At least one embodiment of this disclosure also provides a caching device, including: a shared cache for multiple processor cores to share, the shared cache including multiple storage banks, and a cache management unit configured to allocate a local storage bank and a distant storage bank to each processor core, such that memory access requests for each processor core first access the corresponding local storage bank and then access the corresponding distant storage bank.

[0005] At least one embodiment of this disclosure also provides a cache management apparatus, including: a processor; and a memory storing computer-executable instructions, which, when executed by the processor, implement the cache management method provided in at least one embodiment of this disclosure.

[0006] At least one embodiment of this disclosure also provides an electronic device, including a cache and a cache device provided in at least one embodiment of this disclosure, as well as a plurality of processor cores.

[0007] At least one embodiment of this disclosure also provides a computer-readable storage medium for non-transitory storage of computer-executable instructions, which, when executed by a processor, implement the cache management method provided in at least one embodiment of this disclosure.

[0008] The cache management method provided in the embodiments of this disclosure divides the shared cache into near storage and far storage, which can reduce the latency caused by the increased physical latency due to the size of the shared cache and improve performance. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0010] Figure 1 A schematic diagram of a multi-core processor architecture is shown.

[0011] Figure 2 A schematic diagram illustrating the mapping relationship between memory and cache in direct associativity, fully associativity, and set associativity is shown.

[0012] Figure 3 A schematic diagram illustrating the set-associative organization and addressing mode of the cache is shown;

[0013] Figure 4 A schematic flowchart of a cache management method provided in at least one embodiment of this disclosure is shown;

[0014] Figure 5 A schematic diagram illustrating the mapping relationship between near and far memory in a private cache and a shared cache according to an embodiment is shown.

[0015] Figure 6 It shows Figure 4 A schematic flowchart of an example of step S402;

[0016] Figure 7 It shows Figure 4 A schematic flowchart of another example of step S402;

[0017] Figure 8A A flowchart illustrating a cache management method for read requests is shown.

[0018] Figure 8B A flowchart illustrating a cache management method for write-back requests is shown.

[0019] Figure 8C A schematic block diagram illustrating an example of cache line migration is shown;

[0020] Figure 9A A schematic block diagram of a caching device provided in at least one embodiment of the present disclosure is shown;

[0021] Figure 9B A schematic diagram of the structure of a caching device provided in at least one embodiment of the present disclosure is shown;

[0022] Figure 10 A schematic diagram of a cache management apparatus according to an embodiment of the present disclosure is shown;

[0023] Figure 11A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0025] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0026] Figure 1 This paper illustrates a multi-core processor architecture that is a centralized shared memory architecture. Processing cores core0, core1, core2, and core3 each have their own dedicated caches (private caches), and possess one or more levels of shared caches (typically the last level cache (LLC)). They share the same main memory and input / output (I / O) systems. Each processing core's dedicated cache may include a Level 1 cache or a Level 2 cache, etc.

[0027] For example, caches are typically small, storing only a subset of main memory, and data exchange between the cache and main memory occurs in blocks. To cache data from main memory, a function must be applied to map main memory addresses to the cache; this is called address mapping. After data from main memory is cached according to this mapping, the central processing unit (CPU) translates main memory addresses into cache addresses when executing a program. Common cache address mapping methods include direct mapping, fully associative mapping, and set-associative mapping.

[0028] Although cache capacity is smaller than main memory, its speed is much faster. Therefore, the primary function of cache is to store data that the processor may frequently access in the near future. This allows the processor to directly read data from the cache without frequently accessing the slower main memory, thus improving the processor's access speed to main memory. The basic unit of cache is a cache block or cache line. Similar to how cache is divided into multiple cache lines, the data stored in main memory is also divided in a similar way. The data blocks divided in main memory are called main memory blocks. Typically, the size of a main memory block can be 4KB, and the size of a cache line can also be 4KB. Understandably, in practical applications, the sizes of main memory blocks and cache lines can be set to other values, as long as the size of the main memory block is the same as the size of the cache line.

[0029] There is a certain mapping relationship between main memory and cache, which can be direct associative, fully associative, or set-associative. The principle of the mapping relationship between main memory and cache in direct associative, fully associative, and set-associative methods is as follows: Figure 2 As shown. Both main memory and cache are divided into blocks of the same size. Assume main memory has 32 entries and cache has 8 entries. In direct-associative cache, each main memory block can only be placed in one cache line. For example, if we want to put the 12th main memory block into the cache, since the cache only has 8 entries, it can only be placed in the (12 mod 8 = 4)th entry; it cannot be placed anywhere else. Therefore, the 4th, 12th, 20th, and 28th main memory blocks all correspond to the 4th cache entry. If there is a conflict, they must be replaced. Direct-associative cache requires simple hardware but is inefficient. Figure 2 As shown in (a). In fully associative cache, each main memory block can be placed in any location in the cache, so main memory blocks 4, 12, 20, and 28 can be placed in the cache simultaneously. Fully associative cache requires complex hardware but is highly efficient, such as... Figure 2As shown in (b). Set associativity is a compromise between direct associativity and full associativity. Taking a two-way set associativity as an example, positions 0, 2, 4, and 6 in the cache form one way (referred to as way 0), and positions 1, 3, 5, and 7 form the other way (referred to as way 1), with 4 blocks in each way. For block 12 in main memory, since 12 divided by 4 leaves a remainder of 0, block 12 can be placed either in position 0 of way 0 (i.e., position 0 of the cache) or in position 0 of way 1 (i.e., position 1 of the cache), as shown in (b). Figure 2 As shown in (c).

[0030] Figure 2 The set-associative organization and addressing mode of the cache in (c) can be adopted. Figure 3 Further explanation can be provided through examples. For instance... Figure 3 As shown, the cache is organized as an array of cache lines. A column of cache lines forms the same path, and multiple cache lines at the same position in multiple columns form a set. Therefore, cache lines in the same set are in different paths, i.e., they are ranged through different paths. The location of data or instructions in the cache is obtained by the physical address of the data or instruction to be read. Each physical address (which may include multiple bits depending on the system specifications, such as 32 bits) is divided into three parts:

[0031] • Index: Used to select a group in the cache. All cached lines in the same group are selected using the index.

[0032] • Tags are used to select a specific cache line from a set. The tag of the physical address is compared with the tag of each cache line. If they match, the cache is hit and the cache line is selected. Otherwise, the cache is missed.

[0033] • Offset is used to select the corresponding address in the cache line. It represents the first byte of the physical address in the cache line, and the corresponding data or instruction is read from the position of this byte.

[0034] The working principle of caching requires it to store the latest or most frequently used data as much as possible. When a cache line is sent from main memory to the cache, and the available space in the cache is already full, a cache data replacement problem arises. Solving this problem involves the data replacement mechanism of the caching system. Simply put, the data replacement mechanism of a caching system includes two steps:

[0035] First, filter out data from the cache that is "unimportant" to the application's access;

[0036] Second, remove this data from the cache to make room for new data. For data with a dirty attribute, it also needs to be written back to main memory.

[0037] Existing replacement algorithms may include LRU (Least Recently Used), LFU (Least Frequently Used), MRU (Most Recently Used), NRU (Not Recent Used), SRRIP (Static RRIP), etc.

[0038] A cache consists of a large number of storage cells, each storing one data bit. These cells are physically arranged in an array and accessed via word lines and bit lines. All the cells in each cache are divided and organized into multiple subarrays for easy access; each subarray is called a bank. For example, input buffers and output buffers can be provided for each bank to facilitate access (read, write, etc.); different banks can also be accessed simultaneously in parallel. For example, in the case of associative caches, multiple cache lines in the same path can physically reside in different banks.

[0039] For multi-core processors, when only a single core is working, all resources of the private and shared caches can be used. When a single core's memory access operation accesses the shared cache, if the address is mapped to a remote shared cache, the additional physical latency will reduce the performance improvement brought by the large capacity, thus reducing the performance of the single core. Similarly, when multiple cores are working, resources of the private cache and part of the shared cache can be used. If each core's memory access operation to the shared cache is mapped to a remote physical region (e.g., memory bank), these additional physical latencies will also reduce the performance of the multi-core processor. As the demand for high performance increases, the capacity of off-core caches needs to be larger, which also means that the area of ​​the shared cache needs to be larger. When the size increases to a certain extent, the additional physical latency introduced will offset some of the performance improvement brought by the cache.

[0040] At least one embodiment of this disclosure provides a cache management method for a shared cache shared by multiple processor cores. The cache management method includes: allocating a local memory bank and a remote memory bank to each processor core; and for memory access requests from each processor core, prioritizing access to the corresponding local memory bank and then accessing the corresponding remote memory bank.

[0041] The cache management method provided in the above embodiments of this disclosure divides the shared cache into near storage and far storage, reducing the latency caused by the increased physical latency due to the size of the shared cache and improving performance.

[0042] At least one embodiment of this disclosure also provides a cache device, a cache management device, an electronic device, and a computer-readable storage medium corresponding to the above-described cache management method.

[0043] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0044] Figure 4 A schematic flowchart illustrating a cache management method according to at least one embodiment of this disclosure is shown. This cache management method is used for a shared cache shared by multiple processor cores, the shared cache comprising multiple memory banks.

[0045] like Figure 4 As shown, the cache management method includes the following steps S401 to S402.

[0046] Step S401: Allocate local and far memory for each processor core.

[0047] Step S402: For memory access requests of each processor core, prioritize accessing the corresponding nearby memory bank, and then access the corresponding distant memory bank.

[0048] For example, the access latency of a processor core to a corresponding near memory bank is less than the access latency to a corresponding far memory bank. It should be noted that "near" and "far" here refer to each processor core. Therefore, a memory bank in the cache that is a near memory bank (or a far memory bank) for one processor core may not be a near memory bank (or a far memory bank) for another processor core.

[0049] Figure 5 A schematic diagram illustrating the mapping relationship between near and far memory in a private cache and a shared cache in one embodiment is shown.

[0050] like Figure 5 As shown, the shared cache includes multiple memory banks (near memory bank 0, far memory bank 0, far memory bank 1, etc.). Both the private cache and the shared cache have a "way-set" structure, and the cache lines in the private cache and the cache lines in the memory banks of the shared cache have the same size. For example, a cache line in a certain way within the same group in the private cache of a certain processor core can correspond to a cache line in a certain way within a different group in a different memory bank of the shared cache. For example, the first cache line in the same group and the same way in the private cache can correspond to a cache line in both near memory bank 0 and far memory bank 0. As another example, the second cache line in the same group and the same way in the private cache can correspond to a cache line in both near memory bank 0 and far memory bank 1. The embodiments of this disclosure are not limited to the exemplary correspondences described above.

[0051] For example, for each processor core's memory access request, the nearest memory bank is accessed first, followed by the far memory bank. When a single processor core is working, both near and far memory banks can be accessed. When multiple processor cores are working, each processor core primarily accesses its own near memory bank.

[0052] For memory access requests from a single processor core, latency is minimized, and the access is independent of address mapping, ensuring that all memory accesses from the processor core are concentrated in the nearest memory location. Memory accesses from multiple processor cores are also concentrated in the physically nearest memory location. Whether using a single processor core or multiple processor cores, latency can be reduced and performance improved.

[0053] Figure 6 It shows Figure 4 A schematic flowchart of an example of step S402.

[0054] like Figure 6 As shown, for a read request that is a memory access request, an example of step S402 may include the following steps S601 to S602.

[0055] Step S601: Operate on the corresponding nearby memory bank according to the physical address of the read request.

[0056] For example, in some embodiments of this disclosure, step S601 may include: if a read request hits the corresponding near memory, returning the data in the near memory to the processor core.

[0057] For example, in some embodiments of this disclosure, step S601 may further include: invalidating the copy in the corresponding near memory when the read request hits the corresponding near memory and updating the directory of the shared cache.

[0058] For example, when a processor core issues a read request to the shared cache, it maps the physical address of the read request to a nearby memory location. The processor core accesses this nearby memory location and performs a lookup operation using the tag information of the physical address. It compares the tag of the physical address with the tag of each cache line in the nearby memory location. If a match is found, a cache hit occurs, and the cache line is selected. The data in this cache line is returned to the processor core, the copy in the nearby memory location is invalidated, and the directory in the shared cache is updated. The advantage of this approach is that, since the resources of the nearby memory location are limited, storage space in the nearby memory can be freed up to store other data.

[0059] Step S602: If the read request does not find a match in the nearby storage, the read request is routed to the corresponding far storage based on the physical address of the read request, and the operation is performed on the corresponding far storage.

[0060] For example, if the label of the physical address does not match the labels of all cache lines in the near memory, a cache miss occurs. In this case, the index information of the physical address is used to map to a far memory, the far memory is accessed, and a lookup operation is performed using the label information of the physical address, and the access_farBank_flag (access far memory flag) is marked.

[0061] For example, in some embodiments of this disclosure, step S602 may include: if a read request hits the corresponding remote memory, returning the data in the corresponding remote memory to the processor core.

[0062] For example, in some embodiments of this disclosure, step S602 may further include: retaining a copy in the remote storage, adding aging information of the cache line corresponding to the data stored in the corresponding remote storage, and updating the directory of the shared cache.

[0063] For example, when a read request misses in near memory and accesses far memory, a lookup operation is still required using the tag information of the physical address. The tag of the physical address is compared with the tag of each cache line in the far memory. If a match is found, a cache hit occurs, and this cache line is selected. The data in this cache line is returned to the processor core, and a copy is retained in the far memory. The aging information of this cache line stored in the far memory is added, and the shared cache directory is updated. The reason for retaining the copy is that this far memory may also serve as near memory for other processor cores, and other processor cores have priority access to this far memory. When accessing this far memory, its stored information is not disturbed, maintaining its status as near memory for a particular processor core.

[0064] For example, the cache management method provided in the embodiments of this disclosure may further include step S603: when a read request fails to hit either the corresponding near memory or the corresponding far memory, operations are performed on other processor cores by checking the directory of the shared cache.

[0065] For example, in some embodiments of this disclosure, step S603 may include: if the data to be searched exists in other processor cores, returning the data in the other processor cores to the processor core that issued the read request and updating the directory in the shared cache; if the data to be searched does not exist in other processor cores, sending the read request to memory to obtain the data to be searched.

[0066] For example, if a read request misses in both the corresponding near and far memory, the shared cache directory can be checked to determine if another processor core contains the requested data. If the requested data exists on another processor core, it can be returned to the core that issued the read request via a core-to-core transfer, updating the shared cache directory. If the requested data is also not found on other processor cores, the read request must be sent to memory to retrieve the requested data.

[0067] Figure 7 It shows Figure 4 A schematic flowchart of another example of step S402.

[0068] like Figure 7 As shown, for a write-back request in a memory access request, another example of step S402 may include the following steps S701 to S702.

[0069] Step S701: Perform operations on the corresponding near memory bank based on the physical address of the write-back request.

[0070] For example, in some embodiments of this disclosure, step S701 may include: updating the state stored in the corresponding near memory if the write-back request hits the corresponding near memory; and setting the first sacrificial cache line in the corresponding near memory by means of a replacement algorithm if the write-back request misses the corresponding near memory.

[0071] For example, when a processor core issues a write-back request to the shared cache, it maps the physical address of the write-back request to a nearby memory location. The processor accesses this nearby memory location and performs a lookup operation using the label information of the physical address. The label of the physical address is compared with the label of each cache line in the nearby memory location. If a match is found, a cache hit occurs, and this cache line is called the first victim cache line. The replacement algorithm controls which cache line is sacrificed. If the write-back request hits the corresponding nearby memory location, it means the first victim cache line already exists in that location, so only the state stored in the nearby memory location needs to be updated. If a miss occurs, the replacement algorithm sets the first victim cache line in the corresponding nearby memory location.

[0072] For example, in some embodiments of this disclosure, storing the first sacrificial cache line in the corresponding near memory using a replacement algorithm may include: if there is a free cache line in the corresponding near memory to store the first sacrificial cache line, storing the first sacrificial cache line in the corresponding near memory and updating the shared cache directory; if there is no free cache line in the corresponding near memory to store the first sacrificial cache line, generating a second sacrificial cache line in the corresponding near memory, performing a migration operation on the second sacrificial cache line, marking a flag indicating that it should be written back to the far memory, sending the second sacrificial cache line and the aging information corresponding to the second sacrificial cache line together to the corresponding far memory, and updating the shared cache directory.

[0073] For example, if there is a free cache line in the corresponding near memory to store the first sacrificed cache line, the first sacrificed cache line is directly set in the near memory, and the shared cache directory is updated. If there is no free cache line in the corresponding near memory to store the first sacrificed cache line, a second sacrificed cache line is created in the near memory, the first sacrificed cache line is set at the second sacrificed cache line, and the second sacrificed cache line is migrated within the shared cache. The victim_far_flag signal (write-back to far memory flag) needs to be marked, and the second sacrificed cache line and its corresponding aging information are sent to the far memory. At the same time, the directory update caused by the previous write-back operation generated by the processor core is updated. It should be noted that the parameters of the replacement algorithm are aging information for the Least Recently Used (LRU) algorithm and usage frequency information for the Least Frequently Used (LFU) algorithm. In the embodiments of this disclosure, aging information is used as an example, and this disclosure does not limit this.

[0074] Step S702: In the case of a cache line migration operation generated in the corresponding near memory, the operation is performed on the corresponding far memory according to the address information of the first sacrificed cache line of the corresponding near memory.

[0075] For example, in some embodiments of this disclosure, step S702 may include: if the write-back request hits the corresponding remote memory, updating the state of the cache line in the corresponding remote memory; if the write-back request misses the corresponding remote memory, determining whether to select a third sacrificial cache line in the corresponding remote memory to write back to memory according to the replacement algorithm.

[0076] For example, the index information of the physical address of the write-back request from the near memory corresponds to a far memory bank. This far memory bank is accessed, and a lookup operation is performed using the tag information of the physical address. The tag of the physical address is compared with the tag of each cache line in the far memory bank. If a match is found, a cache hit occurs, indicating that a second sacrificial cache line already exists in the far memory bank. Therefore, only the state stored in the far memory bank needs to be updated. In the case of a cache miss, a replacement algorithm is needed to determine whether a third sacrificial cache line should be generated in the corresponding far memory bank and written back to memory.

[0077] For example, in some embodiments of this disclosure, determining whether to select a third sacrificial cache line in the corresponding remote memory and write it back to memory based on the replacement algorithm may include: storing the second sacrificial cache line in the corresponding remote memory if the replacement algorithm indicates that there is a free cache line available in the corresponding remote memory; and writing the second or third sacrificial cache line back to memory if the replacement algorithm indicates that there is no free cache line available in the corresponding remote memory.

[0078] For example, if there is a free cache line in the corresponding remote memory to store the second sacrificed cache line, the second sacrificed cache line is directly set in the remote memory. If there is no free cache line in the corresponding remote memory to store the second sacrificed cache line, a third sacrificed cache line is generated in the remote memory, and it is determined whether to write the second sacrificed cache line or the third sacrificed cache line into memory. The determination of which sacrificed cache line to write into memory can be achieved in three ways, and the embodiments of this disclosure are not limited to these three methods.

[0079] Method 1: For the LRU replacement algorithm, compare the aging value of the second sacrificial cache line with the aging value of the third sacrificial cache line, and prioritize writing the cache line with the larger aging value between the second and third sacrificial cache lines back to memory. If the aging value of the second sacrificial cache line is equal to the aging value of the third sacrificial cache line, then write the second sacrificial cache line back to memory.

[0080] Method 2: Configure registers to prioritize writing back the second sacrificial cache line or prioritize writing back the third sacrificial cache line.

[0081] Method 3: Check the shared cache directory. If a second sacrificial cache line exists in the near memory corresponding to other processor cores, write the second sacrificial cache line from the near memory corresponding to other processor cores back into memory.

[0082] Figure 8A A flowchart illustrating a cache management method for read requests is shown.

[0083] like Figure 8AAs shown, the processor core first issues a read request. Based on the physical address of the read request, it preferentially accesses the corresponding near memory bank and determines whether the read request hits the corresponding near memory bank. If a hit occurs, the copy in the corresponding near memory bank is invalidated, the data in the near memory bank is returned to the processor core, and the shared cache directory is updated. If a miss occurs, access_farBank_flag is set to 1, and the read request is sent to the corresponding far memory bank. Next, it determines whether the read request hits the corresponding far memory bank. If a hit occurs, the copy in the far memory bank is retained, the aging information of the cache line stored in the far memory bank is added, the data in the corresponding far memory bank is returned to the processor core, and the shared cache directory is updated. If a miss occurs, the shared cache directory is checked to determine whether the read request hits on other processor cores. If a hit occurs, the data is returned to the processor core that issued the read request through a processor core-to-processor core transfer method, and the shared cache directory is updated. If a miss occurs, the read request is sent to memory (an example of system memory) to retrieve the requested data.

[0084] Figure 8B A flowchart illustrating a cache management method for write-back requests is shown.

[0085] like Figure 8B As shown, the processor core first issues a write-back request to the corresponding near memory bank and determines whether the write-back request hits the corresponding near memory bank. If it hits, the shared cache directory is updated. If it misses, the first sacrificial cache line is set in the corresponding near memory bank using a replacement algorithm, and then it is determined whether to generate a second sacrificial cache line. If there is a free cache line in the corresponding near memory bank to store the first sacrificial cache line, no second sacrificial cache line is generated, and the shared cache directory is updated. If there is no free cache line in the corresponding near memory bank to store the first sacrificial cache line, a second sacrificial cache line is generated, and then the second sacrificial cache line is migrated to the corresponding far memory bank. The write-back request is then routed to the corresponding far memory bank. Next, it is determined whether the second sacrificial cache line hits the corresponding far memory bank. If it hits, the write-back request is completed. If it misses, the replacement algorithm determines whether to generate a third sacrificial cache line in the corresponding far memory bank. If a free cache line exists in the corresponding remote memory, a third sacrificial cache line is not generated. The system then checks whether the second sacrificial cache line should be written back to memory. If yes, it is written back to memory; otherwise, it is set in the corresponding remote memory. If no free cache line exists in the corresponding remote memory, a third sacrificial cache line is generated, the second sacrificial cache line is set in the corresponding remote memory, and the third sacrificial cache line is written back to memory.

[0086] Figure 8CA schematic block diagram illustrating an example of cache line migration is shown.

[0087] like Figure 8C As shown, in the initial state, there are 4 processor cores (Core0, Core1, Core2, Core3). Core3 has a copy D. The shared cache includes 4 memory banks (Memory Bank 0, Memory Bank 1, Memory Bank 2, Memory Bank 3). Memory Bank 0 has a copy A, Memory Bank 1 has a copy B, Memory Bank 2 has a copy C, and Memory Bank 3 has a copy F. Memory Bank 0 is the near memory bank of Core0 and the far memory bank of Core1 / 2 / 3; Memory Bank 1 is the near memory bank of Core1 and the far memory bank of Core0 / 2 / 3; Memory Bank 2 is the near memory bank of Core2 and the far memory bank of Core0 / 1 / 3; Memory Bank 3 is the near memory bank of Core3 and the far memory bank of Core0 / 1 / 2.

[0088] First, Core0 reads data from its corresponding near memory bank (Memory Bank 0), Core1 reads data from its corresponding far memory bank (Memory Bank 2), Core2 reads data from other processor cores (Core3), and Core3 reads data from memory. After these read operations, copy A in Memory Bank 0 is migrated to Core0, copy C in Memory Bank 2 is migrated to Core1 while Memory Bank 2 retains copy C, copy D in Core3 is migrated to Core2 while Core3 retains copy D, and copy E in memory is migrated to Core3. Then, Core0, Core1, and Core3 each generate write-back requests. After the write-back operations, the sacrificial cache line (copy A) in Core0 is written back to Memory Bank 0. The sacrificial cache line (copy C) in Core1 is written back to Memory Bank 1, the sacrificial cache line (copy B) in Memory Bank 1 is migrated to the far memory bank (Memory Bank 2) corresponding to Core1, and the sacrificial cache line (copy C) in Memory Bank 2 is written back to memory. The sacrificial cache line F in the near memory bank (memory bank 3) corresponding to Core3 is migrated to the corresponding far memory bank (memory bank 0), and the sacrificial cache line F is written back to memory from memory bank 0.

[0089] Figure 9A The diagram shows a schematic block diagram of a caching device 900 according to at least one embodiment of the present disclosure, which can be used to perform... Figure 4 The cache management method shown.

[0090] like Figure 9AAs shown, the cache device 900 includes a shared cache 901 for multiple processor cores to share and a cache management unit 902. The cache management unit 902 includes a near-memory bank receiving component 903, a far-memory bank receiving component 904, a near-memory bank pipeline control component 905, a far-memory bank pipeline control component 906, a near-memory bank return result component 907, and a far-memory bank return result component 908. Here, the shared cache 901 includes multiple memory banks.

[0091] The cache management unit 902 is configured to allocate a local memory bank and a distant memory bank to each processor core, and to make memory access requests for each processor core prioritize accessing the corresponding local memory bank before accessing the corresponding distant memory bank.

[0092] For example, the access latency of a processor core to its corresponding near memory bank is less than the access latency to its corresponding far memory bank.

[0093] The near memory receiving component 903 is configured to receive memory access requests sent to the corresponding near memory.

[0094] The remote memory receiving component 904 is configured to receive memory access requests sent to the corresponding remote memory.

[0095] The near-memory pipeline control component 905 is configured to determine the processing method of the memory access request received in the corresponding near memory and whether the request is hit in the corresponding near memory, and to execute the replacement algorithm for the corresponding near memory.

[0096] The remote memory pipeline control component 906 is configured to determine the processing method of the memory access request received by the corresponding remote memory and whether the memory access request is hit in the corresponding remote memory, and to execute the replacement algorithm for the corresponding remote memory.

[0097] The near-memory return result component 907 is configured to return the result required by the processor core to the processor core.

[0098] The remote memory return result component 908 is configured to return the result required by the processor core to the processor core.

[0099] The cache device 900 and Figure 4 The cache management methods shown have the same technical effect, so they will not be described in detail here.

[0100] Figure 9B A schematic diagram of the structure of a cache device 910 provided in at least one embodiment of the present disclosure is shown.

[0101] like Figure 9BAs shown, the near-memory receiving component 911 receives memory access requests sent by the processor to the near-memory. The near-memory receiving component 911 then sends the memory access requests to the near-memory pipeline control component 912. The near-memory pipeline control component 912 is connected to the near-memory storage component 913, the near-memory return result component 914, and the far-memory receiving component 915. The near-memory result return component 914 is responsible for returning the result to the processor. The far-memory receiving component 915 receives memory access requests from the near-memory pipeline control component 912 and sends the memory access requests to the far-memory pipeline control component 916. The far-memory pipeline control component 916 is connected to the far-memory storage component 917, the far-memory return result component 918, and the memory 919. The far-memory return result component 918 can read data from the memory 919 and is responsible for returning the result to the processor.

[0102] It should be noted that the near-memory receiving component and the far-memory receiving component can be implemented using hardware methods such as queues or FIFO (First In First Out) queues, and this disclosure does not impose any restrictions on this. The near-memory storage component and the far-memory storage component are used to store cache line information, and they can be in the form of static random access memory (SRAM), dynamic random access memory (DRAM), etc., and this disclosure does not impose any restrictions on this. The memory can be on-chip storage or off-chip storage, and this disclosure does not impose any restrictions on this.

[0103] For example, the cache device can be implemented in hardware, software, firmware, or any feasible combination thereof, without limitation herein.

[0104] At least one embodiment of this disclosure also provides a cache management apparatus, including: a memory for non-temporarily storing computer-executable instructions; and a processor for running the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to perform the cache management method provided in at least one embodiment of this disclosure.

[0105] Figure 10 A schematic diagram of a cache management apparatus 1000 according to an embodiment of the present disclosure is shown. For example... Figure 10 As shown, the cache management device 1000 according to an embodiment of the present disclosure may include a processing device 1001 and a memory 1002, which can be interconnected via a bus 1003.

[0106] The processing device 1001 can perform various actions and processes according to the program or code stored in the memory 1002. Specifically, the processing device 1001 can be an integrated circuit chip with signal processing capabilities. For example, the processing device can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, processes, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor, such as an x86 architecture or an ARM architecture.

[0107] Memory 1002 stores computer-executable instructions, which, when executed by processing device 1001, implement the cache management method provided in at least one embodiment of this disclosure. Memory 1002 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous interconnected dynamic random access memory (SLDRAM), and direct main memory bus random access memory (DRRAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0108] At least one embodiment of this disclosure also provides an electronic device including a cache and a cache device provided in at least one embodiment of this disclosure, as well as a plurality of processor cores. In one embodiment, the electronic device is, for example, a central processing unit, which is, for example, a single-core or multi-core processor. In one embodiment, the electronic device is a computer system, which includes one or more processors.

[0109] Figure 11 A schematic diagram of an electronic device 1100 according to an embodiment of the present disclosure is shown. (As shown) Figure 11 As shown, the electronic device 1100 according to an embodiment of the present disclosure may include a cache device 900, a cache 1101, and a plurality of cores 1102.

[0110] At least one embodiment of this disclosure provides a computer-readable storage medium for non-transitory storage of computer-executable instructions that, when executed by a processor, implement the cache management method provided in at least one embodiment of this disclosure.

[0111] Similarly, the computer-readable storage medium in the embodiments of this disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0112] Embodiments of this disclosure also provide a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a cache management method according to embodiments of this disclosure.

[0113] The technical effects of the aforementioned caching device, cache management device, electronic device, and storage medium are similar to those of... Figure 4 The cache management methods shown have the same technical effect, so they will not be described in detail here.

[0114] The following points need to be explained:

[0115] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0116] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0117] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A cache management method for a shared cache shared by multiple processor cores, the cache management method comprising: Each processor core is allocated its own near memory bank and far memory bank; For each memory access request of the processor core, the corresponding near memory bank is accessed first, and then the corresponding far memory bank is accessed. Specifically, for each processor core, priority is given to accessing the corresponding near memory bank, followed by accessing the corresponding far memory bank, including: In response to the memory access request including a read request, the corresponding near memory bank is operated on according to the physical address of the read request; and If the read request is not found in the nearby memory, the read request is routed to the corresponding distant memory based on the physical address of the read request, and the operation is performed on the corresponding distant memory. Specifically, for each processor core, priority is given to accessing the corresponding near memory bank, followed by accessing the corresponding far memory bank, including: In response to the memory access request including a write-back request, the corresponding near memory bank is operated on according to the physical address of the write-back request; In the case of a cache line migration operation generated by the corresponding near memory, the operation is performed on the corresponding far memory according to the address information of the first sacrificed cache line of the corresponding near memory; Specifically, the operation on the corresponding near memory bank based on the physical address of the write-back request in the memory access request includes: If the write-back request hits the corresponding near memory, update the state stored in the corresponding near memory; If the write-back request fails to find the corresponding near memory, the first sacrificial cache line is set in the corresponding near memory using a replacement algorithm; The first sacrificed cache line is determined based on the following method: Access the corresponding nearby memory and perform a lookup operation using the tag information of the physical address. In response to the tag information of the physical address matching the tag of the corresponding cache line in the corresponding near memory, the corresponding cache line is used as the first sacrificed cache line.

2. The cache management method according to claim 1 further includes: If the read request fails to find a match in either the corresponding near memory or the corresponding far memory, the operation is performed on other processor cores by checking the directory of the shared cache.

3. The cache management method according to claim 1, wherein, Operations are performed on the corresponding nearby memory bank based on the physical address of the read request in the memory access request, including: If the read request is found in the corresponding near memory, the data in the near memory is returned to the processor core.

4. The cache management method according to claim 1, wherein, Operations are performed on the corresponding nearby memory bank based on the physical address of the read request in the memory access request, including: If the read request hits the corresponding near memory, invalidate the copy in the corresponding near memory and update the directory of the shared cache.

5. The cache management method according to claim 1, wherein, Operating on the corresponding remote memory includes: If the read request is found in the corresponding remote memory, the data in the corresponding remote memory is returned to the processor core.

6. The cache management method according to claim 5, wherein, Operating on the corresponding remote memory also includes: The copy in the remote storage is retained, the aging information of the cache line corresponding to the data stored in the corresponding remote storage is added, and the directory of the shared cache is updated.

7. The cache management method according to claim 2, wherein, Operating on other processor cores by viewing the shared cache directory includes: If the data to be searched exists in the other processor cores, the data in the other processor cores is returned to the processor core that issued the read request, and the directory in the shared cache is updated; If the data to be searched is not found in the other processor cores, the read request is sent to memory to retrieve the data to be searched.

8. The cache management method according to claim 1, wherein, The first sacrificed cache line is stored in the corresponding near memory using a replacement algorithm, including: If there is a free cache line in the corresponding near memory to store the first sacrificed cache line, store the first sacrificed cache line in the corresponding near memory and update the directory of the shared cache; If there is no free cache line in the corresponding near memory to store the first sacrificed cache line, the corresponding near memory generates a second sacrificed cache line, performs a migration operation on the second sacrificed cache line, marks a flag indicating that it should be written back to the far memory, and sends the second sacrificed cache line and the aging information corresponding to the second sacrificed cache line to the corresponding far memory to update the directory of the shared cache.

9. The cache management method according to claim 8, wherein, Operations are performed on the corresponding far memory based on the address information of the first sacrificial cache line of the corresponding near memory, including: If the write-back request hits the corresponding remote memory, update the state of the cache line in the corresponding remote memory; If the write-back request fails to find a cached line in the corresponding remote memory, a replacement algorithm is used to determine whether to select a third sacrificial cache line from the corresponding remote memory and write it back to memory.

10. The cache management method according to claim 9, wherein, The replacement algorithm determines whether to select a third sacrificial cache line from the corresponding remote memory and write it back to memory, including: If the replacement algorithm indicates that there is a free cache line available in the corresponding remote memory, the second sacrifice cache line is stored in the corresponding remote memory. If the replacement algorithm shows that there are no free cache lines available in the corresponding remote memory, the second or third sacrifice cache line is written back to the memory.

11. The cache management method according to claim 10, wherein, Writing the second or third sacrificed cache line back to memory includes: Compare the aging value of the second sacrificial cache line with the aging value of the third sacrificial cache line; Write the cache line with the larger aging value from the second and third sacrifice cache lines back to the memory; or If the aging value of the second sacrificial cache line is equal to the aging value of the third sacrificial cache line, the second sacrificial cache line is written back to the memory.

12. The cache management method according to claim 10, wherein, Writing the second or third sacrificed cache line back to memory includes: Configure the register to prioritize writing back the second sacrificed cache line or prioritize writing back the third sacrificed cache line.

13. The cache management method according to claim 10, wherein, Writing the second or third sacrificed cache line back to memory includes: If the second sacrificial cache line exists in the near memory corresponding to other processor cores, the second sacrificial cache line in the near memory corresponding to the other processor cores is written back to the memory.

14. A buffer device, comprising: A shared cache for use by multiple processor cores, the shared cache comprising multiple memory banks. The cache management unit is configured to allocate a local memory bank and a distant memory bank to each processor core, and to make memory access requests for each processor core first access the corresponding local memory bank and then access the corresponding distant memory bank. Specifically, for each processor core, the corresponding near memory bank is accessed first, followed by the corresponding far memory bank. The cache management unit is further configured as follows: In response to the memory access request including a read request, the corresponding near memory bank is operated on according to the physical address of the read request; If the read request is not found in the nearby memory, the read request is routed to the corresponding distant memory based on the physical address of the read request, and the operation is performed on the corresponding distant memory. Specifically, for each processor core, the corresponding near memory bank is accessed first, followed by the corresponding far memory bank. The cache management unit is further configured as follows: In response to the memory access request including a write-back request, the corresponding near memory bank is operated on according to the physical address of the write-back request; In the case of a cache line migration operation generated by the corresponding near memory, the operation is performed on the corresponding far memory according to the address information of the first sacrificed cache line of the corresponding near memory; Specifically, the cache management unit is further configured to operate on the corresponding near memory bank based on the physical address of the write-back request in the memory access request. If the write-back request hits the corresponding near memory, update the state stored in the corresponding near memory; If the write-back request fails to find the corresponding near memory, the first sacrificial cache line is set in the corresponding near memory using a replacement algorithm; The first sacrificed cache line is determined based on the following method: Access the corresponding nearby memory and perform a lookup operation using the tag information of the physical address. In response to the tag information of the physical address matching the tag of the corresponding cache line in the corresponding near memory, the corresponding cache line is used as the first sacrificed cache line.

15. The caching device according to claim 14, wherein, The cache management unit includes: A near memory receiving component is configured to receive memory access requests sent to the corresponding near memory; A remote memory receiving component is configured to receive memory access requests sent to the corresponding remote memory. The near-memory pipeline control component is configured to determine the processing method of the memory access request received by the corresponding near-memory and whether it is hit in the corresponding near-memory, and to execute the replacement algorithm for the corresponding near-memory; The remote memory pipeline control component is configured to determine the processing method of the memory access request received by the corresponding remote memory and whether it is hit in the corresponding remote memory, and to execute the replacement algorithm for the corresponding remote memory. The near-memory return result component returns the result required by the processor core to the processor core; The remote memory return result component returns the result required by the processor core to the processor core.

16. A cache management device, comprising: processor; as well as Memory, which stores computer-executable instructions. The computer-executable instructions, when executed by the processor, implement the cache management method according to any one of claims 1-13.

17. An electronic device comprising a cache and the cache device of claim 14, and a plurality of processor cores.

18. A computer-readable storage medium for non-transitory storage of computer-executable instructions, in, The computer-executable instructions, when executed by a processor, implement the cache management method according to any one of claims 1-13.

Images