Cache consistency optimization method and device, equipment and storage medium

Abstract

The embodiment of the application relates to the technical field of computers, and discloses a cache consistency optimization method and device, equipment and a storage medium, which comprise the following steps: in response to a cache consistency operation request initiated by a current processor core, a memory address corresponding to the cache consistency operation request is acquired; a target listening bus is determined from a plurality of buses according to the memory address corresponding to the cache consistency operation request, and the cache consistency operation request is completed by using the target listening bus; or a plurality of related processor cores that need to respond to the cache consistency operation request are determined according to the memory address corresponding to the cache consistency operation request, and the cache consistency operation request is completed by using the current processor and all related processor cores, so that cache consistency is efficiently maintained with a small cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cache consistency optimization method, apparatus, device and storage medium. Background Technology

[0002] Each processor core in a modern many-core processor has a cache to improve data access speed and ensure ease of program writing. When running parallel programs, multiple threads or processes running on multiple processor cores may read and write data in the same shared memory region. To ensure the correctness of parallel program execution, cache coherence is required between different processors within the same compute node and between different processor cores within the same processor, so that the same data remains consistent across the caches of multiple processor cores.

[0003] Snoop-based coherence protocols (Snoop protocols for short) are one of the main methods for maintaining cache coherence. The implementation of snoop protocols relies on a bus or bus-like network connection (such as a 2D mesh). Based on this network connection, all requests issued by the private cache of a single processor core are broadcast to the private caches of all other processor cores in the system. Access requests from all processor cores can also be ordered on this bus to achieve the memory access ordering requirements in both cache coherence and memory identity models.

[0004] Although the snooping protocol has low hardware implementation overhead and low power consumption, the competitive access and ordered response of all processor cores to the bus can create a performance bottleneck, limiting its application to scenarios with a small number of processor cores. Currently, a single CPU can contain hundreds of cores, and a supercomputer node typically has at least two CPUs, necessitating support for cache coherency with 200 or more processor cores. Therefore, there is a pressing need to optimize the snooping protocol to enable its application to scenarios with a large number of cores, thereby achieving the goal of efficiently maintaining cache coherency with minimal hardware overhead. Summary of the Invention

[0005] The purpose of this invention is to provide at least one cache consistency optimization method, apparatus, device, and storage medium, which can at least solve the performance bottleneck of existing snooping protocols and at least achieve the effect of efficiently maintaining cache consistency with relatively small hardware overhead.

[0006] To address the aforementioned technical problems, at least one embodiment of this application provides a cache consistency optimization method, comprising:

[0007] In response to a cache coherency operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherency operation request;

[0008] The target listening bus is determined from multiple buses based on the memory address corresponding to the cache coherence operation request, and the cache coherence operation request is completed using the target listening bus; or, several relevant processor cores that need to respond to the cache coherence operation request are determined based on the memory address corresponding to the cache coherence operation request, and the current processor and all relevant processor cores are used to complete the cache coherence operation request.

[0009] In some optional embodiments, the target listening bus is determined from multiple buses based on the memory address corresponding to the cache coherency operation request, including:

[0010] Determine the address tag corresponding to the memory address of the cache consistency operation request. The address tag includes the memory address bits other than the cache block size.

[0011] The target listening bus is determined from multiple buses based on the mapping relationship between address tags and bus numbers.

[0012] In some optional embodiments, the method further includes:

[0013] Determine the load balance of multiple buses within a preset time period. Load balance is used to measure how close the number of cache coherency operation requests processed by each bus is.

[0014] When the load balance of multiple buses falls below a preset threshold within a preset time period, the mapping relationship between address tags and bus numbers is adjusted.

[0015] In some optional embodiments, determining the load balance of multiple buses over a preset time period includes:

[0016] The load balance of the multiple buses within the preset time period is determined by the ratio of the average to the maximum number of cache consistency operation requests processed by the multiple buses within the preset time period.

[0017] In some optional embodiments, several relevant processor cores that need to respond to the cache consistency operation request are determined based on the memory address corresponding to the cache consistency operation request, including:

[0018] Based on the memory address corresponding to the cache coherence operation request and the global cache status bit table, several relevant processor cores that need to respond to the cache coherence operation request are determined. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in each entry corresponds to an identifier address segment, which is used to indicate whether the corresponding processor core is currently caching data in the corresponding identifier address segment. The global cache status bit table is consistent with the real-time cache status table of each processor core. Here, an identifier address segment is a contiguous subspace in the memory address space.

[0019] In some optional embodiments, the method further includes:

[0020] If all data cached in the current core group belongs to the same identifier address range and no other processor cores cache data in that identifier address range, then the local bus of the current core group is separated from the global bus; wherein, all processor cores of the current processor are divided into several non-overlapping core groups.

[0021] In some optional embodiments, several relevant processor cores that need to respond to the cache consistency operation request are determined based on the memory address corresponding to the cache consistency operation request, including:

[0022] Based on the memory address corresponding to the cache coherence operation request and the directory entries of each core group, the relevant core groups that need to respond to the cache coherence operation request are determined, and all relevant processor cores of all relevant core groups are monitored. The directory entries record the sharing of each cache block among all core groups. Among them, all processor cores of the current processor are divided into several non-overlapping core groups.

[0023] In some optional embodiments, the directory entries record the sharing of each cache block across all core groups, including:

[0024] The methods for recording the sharing of each cache block across all core groups include any one of real-time counting, history marking, or bitmap methods, or a combination thereof.

[0025] The real-time counting method has several counters in each directory entry, where each counter records the number of times the corresponding cache block is shared in real time by a set of kernels consisting of several kernel groups.

[0026] The history tagging method has several history tag bits in each directory entry, where each history tag bit records whether a set of kernels consisting of several kernel groups has ever accessed the corresponding cache block;

[0027] Bitmap method is used to record the precise sharing of corresponding cache blocks among the processor cores in a single owner core group when a directory entry belongs to a single owner core group.

[0028] At least one embodiment of this application also provides a cache consistency optimization apparatus, comprising:

[0029] The response module is used to respond to cache coherency operation requests initiated by the current processor core and obtain the memory address corresponding to the cache coherency operation request.

[0030] The processing module is used to determine the target listening bus from multiple buses based on the memory address corresponding to the cache coherence operation request, and use the target listening bus to complete the cache coherence operation request; or, based on the memory address corresponding to the cache coherence operation request, determine several related processor cores that need to respond to the cache coherence operation request, and use the current processor and all related processor cores to complete the cache coherence operation request.

[0031] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the cache coherency optimization method described above.

[0032] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the cache coherency optimization method described above.

[0033] The cache coherence optimization method, apparatus, device, and storage medium provided in the embodiments of this application obtain the memory address corresponding to the cache coherence operation request in response to the cache coherence operation request initiated by the current processor core; determine the target listening bus from multiple buses according to the memory address corresponding to the cache coherence operation request, and use the target listening bus to complete the cache coherence operation request; or, determine a number of related processor cores that need to respond to the cache coherence operation request according to the memory address corresponding to the cache coherence operation request, and use the current processor and all related processor cores to complete the cache coherence operation request, thereby achieving efficient cache coherence maintenance with minimal overhead. Attached Figure Description

[0034] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0035] Figure 1 This is a flowchart of a cache consistency optimization method provided in one embodiment of this application. Figure 1 ;

[0036] Figure 2 This is a flowchart of a cache consistency optimization method provided in another embodiment of this application. Figure 2 ;

[0037] Figure 3 This is another embodiment of the cache consistency optimization method provided in this application. Figure 3 ;

[0038] Figure 4 This is a flowchart of a cache consistency optimization method provided in another embodiment of this application. Figure 4 ;

[0039] Figure 5 This is a schematic diagram of a cache consistency optimization apparatus provided in one embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0042] To facilitate understanding of the embodiments of this application, relevant content regarding cache consistency will be introduced first.

[0043] Currently, there are two main cache coherence protocols: listener-based coherence protocols (hereinafter referred to as listener protocols) and directory-based coherence protocols (hereinafter referred to as directory protocols). Listener protocols rely on bus or bus-like network connections. Using this network connection, all requests from a single processor core's private cache are broadcast to the private caches of all other processor cores in the system. Access requests from all processors can also be ordered on this bus to meet the memory access ordering requirements of the cache coherence model and memory identity model. Listener protocols can also effectively handle multiple conflicting requests for the same data block through the bus structure, and multiple processors' private caches can communicate directly through this bus structure, reducing communication latency. Directory protocols, on the other hand, use a directory structure to manage cache lines. In a directory protocol, memory access requests from a processor core's private cache are first sent to the directory structure that owns the corresponding cache line. This directory structure records the current sharing status of the cache line, and the controller determines the processor core's private cache or memory to respond to the request based on this current sharing status.

[0044] Both snooping and directory protocols have their advantages and disadvantages. For current commercial CPU applications, snooping protocols have lower hardware implementation overhead and lower power consumption, but the competitive access and ordered response of all processor cores to the bus can easily become a bottleneck for parallel performance. Directory protocols enable parallel maintenance of consistency for cache blocks at different addresses, thus achieving efficient cache coherency. However, on the one hand, the need to record the shared access states of a large number of cache blocks incurs hardware overhead, and on the other hand, the process of finding the corresponding record for a cache block in the directory introduces significant latency. Moreover, the number of cores in a single CPU has reached hundreds, and a computing node in a supercomputer typically has at least two CPUs, requiring support for cache coherency of 200 or more processor cores. For snooping protocols, the competitive use of the bus (or network) will have a more significant impact on the performance of parallel programs. Directory protocols will encounter bottleneck problems, especially as directories become increasingly large and access latency increases. To address these issues, simplified directory protocols or a multi-level hybrid of directory and snooping protocols can be adopted. Specifically, a single entry in the directory manages shared information for multiple (e.g., 4 or 8) contiguous memory cache blocks, significantly reducing the directory size (i.e., the number of entries); processor cores are grouped, with each group sharing a directory, and each directory corresponding to a contiguous range of physical memory addresses, with different directories corresponding to non-overlapping memory address ranges; processor cores are grouped, with each group sharing a bus for monitoring, and inter-group communication using directories, where each entry in the directory records which core groups are using the corresponding cache block (but does not record the specific cores within the core group).

[0045] To address at least one of the aforementioned technical problems, this invention proposes a cache consistency optimization method. The implementation details of the cache consistency optimization method in this embodiment are described below. The following content is only for ease of understanding and is not necessary for implementing this solution.

[0046] Example 1:

[0047] The cache consistency optimization method provided in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes:

[0048] S101. In response to the cache coherency operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherency operation request.

[0049] In scenarios with multiple bus architectures, it's crucial to ensure that multiple cache coherency requests originating from different processor cores and corresponding to the same cache block are assigned to the same bus. In other words, it's necessary to ensure that cache coherency requests processed on different buses correspond to different cache blocks. Whether two cache coherency requests belong to the same cache block is determined by their corresponding memory addresses. Obtaining the memory address of the cache coherency request is essential to identifying the bus handling it. In a NUMA architecture based on multi-channel memory technology, each identifier address segment corresponds to a memory channel, and each memory channel corresponds to a core group. Obtaining the memory address of the cache coherency request is also crucial to identifying the core group handling it.

[0050] S102. Determine the target listening bus from multiple buses based on the memory address corresponding to the cache coherence operation request, and use the target listening bus to complete the cache coherence operation request; or, determine several relevant processor cores that need to respond to the cache coherence operation request based on the memory address corresponding to the cache coherence operation request, and use the current processor and all relevant processor cores to complete the cache coherence operation request.

[0051] The cache coherence optimization method provided in this embodiment obtains the memory address corresponding to the cache coherence operation request in response to the cache coherence operation request initiated by the current processor core; determines the target listening bus from multiple buses based on the memory address corresponding to the cache coherence operation request, and uses the target listening bus to complete the cache coherence operation request; or, determines several related processor cores that need to respond to the cache coherence operation request based on the memory address corresponding to the cache coherence operation request, and uses the current processor and all related processor cores to complete the cache coherence operation request. This method achieves efficient cache coherence maintenance with minimal overhead, enabling the listening protocol to be applied to situations with a large number of cores and solving the performance bottleneck caused by contention for bus usage.

[0052] Example 2:

[0053] In this application, "bus" is used to refer to the network connection, either a bus or a bus-like structure, required by the listening protocol. In traditional processors employing listening protocols, all processor cores managed by the protocol typically use only one bus to maintain cache coherency. However, with the continuous development of computer technology, multiple buses exist between all processor cores in a computer. Enabling multiple buses to cooperate and perform multiple cache coherency operations in parallel is an important means of improving cache coherency maintenance efficiency.

[0054] The cache consistency optimization method provided in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 2 As shown, it includes:

[0055] S201. In response to the cache coherency operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherency operation request.

[0056] When multiple processor cores simultaneously initiate multiple cache coherence operation requests, it is necessary to determine which bus each cache coherence operation request uses. The bus has the function of ordering multiple cache coherence operation requests to ensure that multiple cache coherence operation requests corresponding to the same cache block are executed in the correct order. In the case of multiple buses, it is necessary to ensure that multiple cache coherence operation requests corresponding to the same cache block initiated by different processor cores are assigned to the same bus. In other words, it is necessary to ensure that cache coherence operation requests processed on different buses correspond to different cache blocks. Whether two cache coherence operation requests belong to the same cache block is determined by the memory addresses corresponding to these two requests. Therefore, in this embodiment, when responding to a cache coherence operation request initiated by the current processor core, the memory address corresponding to the cache coherence operation request is obtained to determine the cache block corresponding to that cache coherence operation request.

[0057] S202. Determine the address marker of the memory address corresponding to the cache consistency operation request. The address marker includes the memory address bits other than the cache block size.

[0058] Whether two cache coherence operation requests belong to the same cache block is determined by the memory addresses corresponding to these two requests, specifically by the memory address bits other than the cache block size. For example, corresponding to a 64-bit memory address and a 64-byte cache block size, if the high 58 bits of the memory addresses corresponding to two requests are the same, then the two requests correspond to the same cache block. In this application, the memory address bits other than the cache block size are recorded as address markers. Therefore, in this embodiment, after obtaining the memory address corresponding to the cache coherence operation request, the memory address bits other than the cache block size in that memory address are determined as the address markers of the memory address corresponding to the cache coherence operation request.

[0059] S203. Determine the target listening bus from multiple buses based on the mapping relationship between address tags and bus numbers, and use the target listening bus to complete the cache coherency operation request.

[0060] The key to bus collaboration lies in the mapping relationship between address tags and bus numbers. This mapping relationship can be pre-established, mapping each bus to different cache blocks. In one optional implementation, a subset of bits in the address tag of a cache coherence operation request can be used to determine the bus number corresponding to that request. For example, the lowest few bits in the address tag can be used to determine the bus number. Taking four sets of buses as an example, these buses are numbered 0 to 3, and then the lowest two bits in the address tag are used to determine the bus number. When the lowest two bits in the address tag of a cache coherence operation request are 10, then this request is handled by bus number 2, meaning the target listening bus is bus number 2. The maximum memory capacity of a compute node typically does not exceed 4TB, and the cache block size is usually 64 bytes. Therefore, 36 bits are sufficient for the address tag; that is, a 36-bit address width on one bus is sufficient, separating multiple sets of buses from the network for exchanging cache block data between processing cores.

[0061] In this embodiment, after determining the address tag of the memory address corresponding to the cache coherence operation request, the target listening bus can be determined from multiple buses according to the mapping relationship between the address tag and the bus number, and the target listening bus is used to complete the cache coherence operation request. This makes it possible to have only one network for handling the exchange of cache block data between cores even when there are multiple listening buses, which can significantly reduce the hardware overhead of implementing multiple listening buses.

[0062] It is understandable that in the case of multiple buses, load balancing needs to be maintained among them to ensure that all buses can fully utilize their acceleration capabilities under conditions of intensive cache coherence operation requests. One way to quantify load balancing is to determine whether the number of cache coherence operation requests handled by different buses is similar over a period of time. The target listening bus responding to cache coherence operation requests can be adjusted based on load balancing to further improve the efficiency of maintaining cache coherence. Based on the above embodiments, the cache coherence optimization method provided in this embodiment may further include:

[0063] S204. Determine the load balance of multiple buses within a preset time period. Load balance is used to measure the similarity of the number of cache consistency operation requests processed by each bus.

[0064] In this embodiment, load balancing is used to measure the similarity of the number of cache coherency operation requests processed by each bus. A specific method for calculating load balancing is to calculate the ratio of the maximum number of requests among all buses to the average number of requests among all buses. The closer the ratio is to 1, the more balanced it is.

[0065] In some optional implementations, determining the load balance of multiple buses within a preset time period can specifically include: determining the load balance of the multiple buses within the preset time period as the ratio of the average to the maximum number of cache coherence operation requests processed by the multiple buses. The lower the ratio, the more unbalanced the load of each bus.

[0066] S205. When the load balance of multiple buses is lower than the preset threshold within a preset time period, adjust the mapping relationship between the address tag and the bus number.

[0067] The degree to which the memory access methods and mapping relationships of different application threads match largely determines load balancing. For example, using a mapping relationship with the lowest bits of the address tag is suitable for sequential memory access by different threads; when each thread performs skip accesses at intervals of 4 cache blocks, it is possible that most cache coherence operation requests will be allocated to the same bus while other buses remain idle. The memory access behavior of applications is often not fixed, and the memory access behavior of different applications often differs. Therefore, in this embodiment, load balancing across multiple buses can be achieved by adjusting the mapping relationships.

[0068] In this embodiment, mapping relationships can be divided into two categories: static mapping relationships and dynamic mapping relationships. Static mapping relationships refer to a processor having only one mapping relationship; that is, different applications and different time periods of the same application all use the same mapping relationship. Dynamic mapping relationships, on the other hand, refer to mapping relationships that can change during program execution. To achieve load balancing across multiple buses, this embodiment employs dynamic mapping relationships. One specific implementation method is as follows: the management device for multiple buses monitors the load balance among the buses within a threshold time period (or a preset duration). When the load balance is found to be lower than the preset threshold, the mapping relationship is adjusted. For example, in the case of four buses, if a mapping relationship using the lowest two bits of the address tag is found to cause load imbalance, the mapping relationship can be adjusted to use the second and third lowest bits of the address tag. Dynamically adjusting mapping relationships is easy to implement for multiple buses, partly because the number of buses is not too large, and partly because the buses do not store data but only perform transaction processing. When it is necessary to dynamically adjust the mapping relationship, it is only necessary to pause the allocation of new cache coherence operation requests to the bus. Once all existing cache coherence operation requests on all buses have been completed, the mapping relationship can be adjusted. After the mapping relationship is adjusted, new cache coherence operation requests can be allocated according to the new mapping relationship.

[0069] The cache consistency optimization method provided in this embodiment, based on the above embodiment, further improves the efficiency of maintaining cache consistency by dynamically adjusting the mapping relationship between address tags and bus numbers, thereby achieving load balancing among multiple buses.

[0070] Example 3:

[0071] Many many-core processors currently employ Non-Uniform Memory Access (NUMA) architecture based on multi-channel memory. Specifically, when a processor can connect to eight memory channels, all processor cores are divided into eight core groups, with each core group corresponding to one memory channel. Each core group accesses its corresponding channel's memory quickly, while accessing other channels is much slower. Generally, the memory address segments of different channels are completely separated, which manifests as differences in the higher-order bits of the memory addresses between different channels. Taking eight memory channels (numbered 0-7) with a capacity of 32GB per channel as an example, bits 0-34 of the memory address (bit 0 being the least significant bit) are used to distinguish different memory locations within the same channel, while bits 35-37 represent the memory channel number. Modern operating systems, compilers, and even parallel programmers perform numerous performance optimizations based on the NUMA architecture to ensure that a core group accesses its corresponding channel's memory as much as possible and avoids using other channels' memory, thereby improving the performance of parallel programs. Therefore, in most cases, cache coherence operations initiated by processor cores within a core group only involve processor cores within that core group. Nevertheless, traditional snooping protocols still broadcast any cache coherence operation to all processor cores and require responses from all processor cores. This not only incurs a large amount of redundant response overhead, but also causes each processor core to perform a large number of redundant address lookups and corresponding power consumption.

[0072] To reduce redundant responses and address lookups in the eavesdropping protocol, this embodiment first divides the memory address space into several non-overlapping address segments, called identifier address segments. When a NUMA architecture based on multi-channel memory technology exists, each identifier address segment corresponds exactly to one memory channel, and each memory channel corresponds to one core group. Each processor core records whether its private cache currently caches data for each identifier address segment, i.e., it maintains a real-time cache status table for all identifier address segments. When a cache replacement occurs, the real-time cache status table changes. Specifically, each entry in the real-time cache status table is a cache block count for one identifier address segment. That is, when a cache block corresponding to an identifier address segment is swapped in, the entry in the real-time cache status table for that identifier address segment is incremented by 1; when a cache block corresponding to an identifier address segment is swapped out, the entry in the real-time cache status table for that identifier address segment is decremented by 1. When an entry in the real-time cache status table corresponding to an identifier address segment is 0, it indicates that the current processor core does not cache data in the corresponding identifier address segment. The monitoring protocol management device records the global cache status bit table for all processor cores. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in the entry corresponds to an identifier address range, used to indicate whether the corresponding processor core is currently caching data in the corresponding identifier address range. The global cache status bit table needs to be consistent with the real-time cache status tables of each processor core, but synchronization of the global cache status bit table is not required every time any real-time cache status table changes. The global cache status bit table only needs to be modified when an entry in a cache status table changes between 0 and non-zero. This implementation avoids frequent changes to the global cache status bit table and the associated overhead.

[0073] The cache consistency optimization method provided in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 3 As shown, it can specifically include:

[0074] S301. In response to the cache coherency operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherency operation request.

[0075] S302. Based on the memory address corresponding to the cache coherence operation request and the global cache status bit table, determine several relevant processor cores that need to respond to the cache coherence operation request, and use the current processor and all relevant processor cores to complete the cache coherence operation request. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in each entry corresponds to an identifier address segment, used to indicate whether the corresponding processor core is currently caching data in the corresponding identifier address segment. The global cache status bit table is consistent with the real-time cache status table of each processor core. An identifier address segment is a contiguous subspace within the memory address space.

[0076] In response to a cache coherency request initiated by the current processor core, the system determines the processor cores to be monitored based on the current memory address corresponding to the cache coherency request and the global cache status bit table, and only initiates monitoring requests to these processor cores. Specifically, it determines the current identifier address range corresponding to the current memory address, identifies all relevant processor cores that cache data in the current identifier address range based on the global cache status bit table, and then initiates monitoring requests to these relevant processor cores. For all unrelated processor cores that do not cache data in the current identifier address range, no monitoring requests are initiated, thus avoiding waiting for responses from these unrelated processor cores and significantly reducing redundant responses and redundant address lookups.

[0077] To further improve the efficiency of cache coherency operations, the cache coherency optimization method provided in this embodiment may further include: if all data cached by the current core group belongs to the same identifier address range and none of the other processor cores cache data in that identifier address range, then the local bus of the current core group is separated from the global bus. Specifically, when the listening protocol management device discovers that all data cached by the current core group belongs to the same identifier address range, and none of the other processor cores cache data in that identifier address range, the local bus of the current core group can be separated from the global bus, allowing the cache coherency operation of this core group to be processed in parallel with the cache coherency operations of other processor cores, thereby accelerating the efficiency of cache coherency operations.

[0078] Example 4:

[0079] Pure bus-based snooping protocols are unsuitable for scenarios with a large number of processor cores, and pure directory protocols are also unsuitable because precisely marking the sharing status of each cache block across all cores would incur unacceptable hardware overhead (one bit per core, hundreds of bits for hundreds of cores). Many-core processors can use a hybrid directory and snooping approach, such as snoop filtering. This allows a single directory entry to manage the sharing status of multiple contiguous cache blocks, while avoiding the need for precise recording of cache block sharing information: for example, recording the core IDs of one or a few owners of the cache block, and the current number of times it has been effectively shared. Without precise recording of cache block sharing, a cache coherence operation might require snooping across all processor cores. For instance, if a directory entry can only record the core IDs of at most two cache blocks, and a cache block is being read and shared by three or more cores, and another core needs to modify this cache block, snooping across all processor cores is required. This significantly reduces the efficiency of maintaining cache coherence. To address this issue, all processor cores are first divided into several non-overlapping core groups, each with its own subdirectory. When a NUMA architecture based on multi-channel memory technology exists, all cores within the same core group and all entries in its subdirectories can correspond to the same memory channel. Each directory entry records the sharing of corresponding cache blocks across all core groups. Specifically, any one of real-time counting, history marking, or bitmap methods, or a combination thereof, can be used to record the sharing between core groups.

[0080] a) Real-time Counting Method: Each directory entry contains several counters, each recording the number of times a core set (consisting of several core groups) shares the corresponding cache block in real time. Specifically, each directory entry has at least two counters, each corresponding to a core set and recording the real-time sharing of all cache blocks corresponding to the directory entry by all processor cores within that core set. The count in a counter within a directory entry can be understood as the number of times all cores in the corresponding core set have currently validly cached the corresponding cache block of the directory entry. For example, if a directory entry has one counter corresponding to 16 processor cores and the directory entry corresponds to 2 cache blocks, and currently 6 cores have validly cached the first cache block and 3 cores have validly cached the second cache block, then the current count of the counter is 9. When a cache block of a core becomes valid or invalidated, the corresponding counter in the unique directory entry corresponding to that cache block will change (increment or decrease by 1). There can be multiple correspondences between counters and core sets. When the number of counters is the same as the number of core groups, each counter can correspond to one core group. When the number of counters is half the number of core groups, each counter can correspond to two core groups. When the number of counters is 2 and the number of core groups is greater than 2, one counter for a directory entry can correspond to the owner core group of that directory entry (when a directory entry only appears in a subdirectory of a unique core group, that core group is called the owner core group of the directory entry), while the other counter can correspond to all core groups or all other core groups except the owner core group; through these two counters, the sharing of the corresponding cache blocks by the owner core group and the sharing of all other core groups as a whole can be determined.

[0081] b) History Tagging Method: Each directory entry has several history tags, each recording whether a core set consisting of several core groups has ever accessed the corresponding cache block. Specifically, each directory entry has at least two history tags, each corresponding to a core set consisting of several core groups, recording whether all processor cores in each core set have ever accessed all cache blocks corresponding to the directory entry. When a core set has not accessed any cache block corresponding to a directory entry, the corresponding history tag for that directory entry is marked as unaccessed; otherwise, it is marked as accessed. When a cache block of a core becomes invalid, the history tags in the directory entry do not change. Therefore, the history tagging method cannot accurately record the sharing status of cache blocks; that is, there may be cases where there was sharing in the past but not currently, but are considered to be currently shared. There can be multiple correspondences between history tags and core groups. When the number of history tags is the same as the number of core groups, each history tag can correspond to one core group. When the number of history tags is half the number of core groups, each history tag can correspond to two core groups. When the number of history flags is 2 and the number of core groups is greater than 2, one history flag of a directory entry can correspond to the owner core group of that directory entry, while the other history flag can correspond to all other core groups other than the owner core group. Through these two history flags, the sharing of the corresponding cache block by the owner core group and the sharing of all other core groups as a whole can be determined.

[0082] c) Bitmap method: Used to record the precise sharing of the corresponding cache block among the processor cores in the owner core group when a directory entry belongs to a unique owner core group.

[0083] When a cache entry belongs to a single owner core group, it can combine bitmap-based methods with real-time counting or history marking methods to simultaneously record the precise sharing of the corresponding cache block among the processor cores in the owner core group and the sharing among core groups. For example, for a processor with 8 core groups and 16 cores in each core group, each cache entry can have 16 bits (i.e., a 16-bit bitmap) to record the precise sharing of the corresponding cache block among the processor cores in its owner core group, and combine real-time counting or history marking methods to record the sharing among core groups.

[0084] The cache consistency optimization method provided in this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 4 As shown, it includes:

[0085] S401. In response to the cache coherency operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherency operation request.

[0086] S402. Based on the memory address corresponding to the cache coherence operation request and the directory entries of each core group, determine the relevant core groups that need to respond to the cache coherence operation request, and initiate listening to all relevant processor cores of all relevant core groups. The directory entries record the sharing of each cache block among all core groups. Among them, all processor cores of the current processor are divided into several non-overlapping core groups.

[0087] Cache coherence protocols respond to cache coherence operations initiated by the current processor core for the current cache block. First, they determine whether the core numbers of all processors sharing the current cache block can be accurately determined from the corresponding directory entries. If this cannot be accurately determined, they identify all core groups related to the current cache block based on the inter-core group sharing record method. Then, they initiate listening to all relevant cores in all relevant core groups, filtering out irrelevant core groups, i.e., they do not listen to irrelevant core groups. When it is found that only all relevant cores in the current core group to which the current processor core belongs need to be listened to, the primary bus of the current core group can be separated from the secondary bus between core groups, allowing listening between different core groups to be performed in parallel. This is particularly suitable for situations where there is almost no data sharing between core groups, and can significantly improve the efficiency of maintaining cache coherence. Currently, inter-core interconnects on processors often adopt a method similar to a 2D mesh network, which can easily implement the function of initiating listening only to a subset of cores, or in other words, easily achieve "separation of the primary bus from the secondary bus between core groups."

[0088] Example 5:

[0089] Another embodiment of this application relates to a cache consistency optimization device. The implementation details of this cache consistency optimization device are described below. The following details are for ease of understanding and are not essential for implementing this solution. A schematic diagram of the cache consistency optimization device in this embodiment can be seen as follows: Figure 5 As shown, it includes a response module 501 and a processing module 502.

[0090] The response module 501 is used to respond to the cache coherency operation request initiated by the current processor core and obtain the memory address corresponding to the cache coherency operation request;

[0091] The processing module 502 is used to determine the target listening bus from multiple buses according to the memory address corresponding to the cache coherence operation request, and use the target listening bus to complete the cache coherence operation request; or, it determines several related processor cores that need to respond to the cache coherence operation request according to the memory address corresponding to the cache coherence operation request, and uses the current processor and all related processor cores to complete the cache coherence operation request.

[0092] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0093] In some optional implementations, the processing module 502 is used to determine the target listening bus from multiple buses based on the memory address corresponding to the cache coherence operation request, specifically including:

[0094] Determine the address tag corresponding to the memory address of the cache consistency operation request. The address tag includes the memory address bits other than the cache block size.

[0095] The target listening bus is determined from multiple buses based on the mapping relationship between address tags and bus numbers.

[0096] In some alternative implementations, the processing module 502 is further configured to:

[0097] Determine the load balance of multiple buses within a preset time period. Load balance is used to measure how close the number of cache coherency operation requests processed by each bus is.

[0098] When the load balance of multiple buses falls below a preset threshold within a preset time period, the mapping relationship between address tags and bus numbers is adjusted.

[0099] In some optional implementations, the processing module 502 is used to determine the load balance of multiple buses within a preset time period, which may specifically include:

[0100] The load balance of the multiple buses within the preset time period is determined by the ratio of the average to the maximum number of cache consistency operation requests processed by the multiple buses within the preset time period.

[0101] In some optional implementations, the processing module 502 is used to determine, based on the memory address corresponding to the cache coherence operation request, several relevant processor cores that need to respond to the cache coherence operation request, which may specifically include:

[0102] Based on the memory address corresponding to the cache coherence operation request and the global cache status bit table, several relevant processor cores that need to respond to the cache coherence operation request are determined. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in the entry corresponds to an identifier address range, which is used to indicate whether the corresponding processor core is currently caching data in the corresponding identifier address range. The global cache status bit table is consistent with the real-time cache status table of each processor core.

[0103] In some alternative implementations, the processing module 502 is further configured to:

[0104] If all data cached in the current core group belongs to the same identifier address range and none of the other processor cores cache data in that identifier address range, then the local bus of the current core group will be detached from the global bus.

[0105] In some optional implementations, the processing module 502 is used to determine, based on the memory address corresponding to the cache coherence operation request, several relevant processor cores that need to respond to the cache coherence operation request, which may specifically include:

[0106] Based on the memory address corresponding to the cache coherence operation request and the directory entries of each core group, the relevant core groups that need to respond to the cache coherence operation request are determined, and all relevant processor cores of all relevant core groups are monitored. The directory entries record the sharing status of each cache block among all core groups.

[0107] Example 6:

[0108] Another embodiment of this application relates to an electronic device, such as... Figure 6 As shown, it includes: at least one processor 601; and a memory 602 communicatively connected to the at least one processor 601; wherein the memory 602 stores instructions executable by the at least one processor 601, the instructions being executed by the at least one processor 601 to enable the at least one processor 601 to execute the cache coherence optimization method in the above embodiments.

[0109] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0110] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0111] Example 7:

[0112] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0113] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A cache consistency optimization method, characterized in that, include: In response to a cache coherence operation request initiated by the current processor core, obtain the memory address corresponding to the cache coherence operation request; The target listening bus is determined from multiple buses based on the memory address corresponding to the cache coherence operation request, and the cache coherence operation request is completed using the target listening bus; or, several related processor cores that need to respond to the cache coherence operation request are determined based on the memory address corresponding to the cache coherence operation request, and the current processor core and several related processor cores are used to complete the cache coherence operation request. The step of determining the target listening bus from multiple buses based on the memory address corresponding to the cache consistency operation request includes: Determine the address tag of the memory address corresponding to the cache consistency operation request, wherein the address tag includes memory address bits other than the cache block size in the memory address; The target listening bus is determined from multiple buses based on the mapping relationship between address tags and bus numbers; The method further includes: Determine the load balance of multiple buses within a preset time period. The load balance is used to measure the similarity of the number of cache consistency operation requests processed by each bus. When the load balance of multiple buses is lower than a preset threshold within a preset time period, the mapping relationship between the address tag and the bus number is adjusted. The step of determining the number of relevant processor cores that need to respond to the cache consistency operation request based on the memory address corresponding to the cache consistency operation request includes: Based on the memory address corresponding to the cache coherence operation request and the global cache status bit table, several related processor cores that need to respond to the cache coherence operation request are determined. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in each entry corresponds to an identifier address segment, which is used to characterize whether the corresponding processor core is currently caching data in the corresponding identifier address segment. The global cache status bit table is consistent with the real-time cache status table of each processor core. One of the identified address segments is a contiguous subspace within the memory address space.

2. The method according to claim 1, characterized in that, Determining the load balance of multiple buses within a preset time period includes: The load balance of the multiple buses within the preset time period is determined by the ratio of the average to the maximum number of cache consistency operation requests processed by the multiple buses within the preset time period.

3. The method according to claim 1, characterized in that, The method further includes: If all data cached in the current core group belongs to the same identifier address range and no other processor cores cache data in that identifier address range, then the local bus of the current core group will be detached from the global bus. In this context, all processor cores of the current processor are divided into several non-overlapping core groups.

4. The method according to claim 1, characterized in that, The step of determining the number of relevant processor cores that need to respond to the cache consistency operation request based on the memory address corresponding to the cache consistency operation request includes: Based on the memory address corresponding to the cache coherence operation request and the directory entries of each core group, the relevant core groups that need to respond to the cache coherence operation request are determined, and listening is initiated on all relevant processor cores of all relevant core groups. The directory entries record the sharing status of each cache block among all core groups. In this context, all processor cores of the current processor are divided into several non-overlapping core groups.

5. The method according to claim 4, characterized in that, The directory entries record the sharing of each cache block across all core groups, including: The methods for recording the sharing of each cache block across all core groups include any one of real-time counting, history marking, or bitmap methods, or a combination thereof. The real-time counting method has several counters in each directory entry, where each counter records the number of times the corresponding cache block is shared in real time by a set of kernels consisting of several kernel groups. The history marking method has several history mark bits in each directory entry, where each history mark bit records whether a set of kernels consisting of several kernel groups has ever accessed the corresponding cache block. The bitmap method is used to record the precise sharing of the corresponding cache block among the processor cores in the owner core group when a directory entry belongs to a unique owner core group.

6. A cache consistency optimization device, characterized in that, include: The response module is used to respond to the cache coherency operation request initiated by the current processor core and obtain the memory address corresponding to the cache coherency operation request. The processing module is configured to determine the target listening bus from multiple buses based on the memory address corresponding to the cache consistency operation request, and use the target listening bus to complete the cache consistency operation request; or, based on the memory address corresponding to the cache consistency operation request, determine several related processor cores that need to respond to the cache consistency operation request, and use the current processor core and several related processor cores to complete the cache consistency operation request. The processing module is used to determine the target listening bus from multiple buses based on the memory address corresponding to the cache consistency operation request. Specifically, it may include: Determine the address tag corresponding to the memory address of the cache consistency operation request. The address tag includes the memory address bits other than the cache block size. The target listening bus is determined from multiple buses based on the mapping relationship between address tags and bus numbers; The processing module is also used for: Determine the load balance of multiple buses within a preset time period. Load balance is used to measure how close the number of cache coherency operation requests processed by each bus is. When the load balance of multiple buses falls below a preset threshold within a preset time period, the mapping relationship between address tags and bus numbers is adjusted. The processing module is used to determine, based on the memory address corresponding to the cache consistency operation request, several relevant processor cores that need to respond to the cache consistency operation request. Specifically, these cores may include: Based on the memory address corresponding to the cache coherence operation request and the global cache status bit table, determine several relevant processor cores that need to respond to the cache coherence operation request. Each entry in the global cache status bit table corresponds to a processor core, and each status bit in each entry corresponds to an identifier address range, which is used to indicate whether the corresponding processor core is currently caching data in the corresponding identifier address range. The global cache status bit table is consistent with the real-time cache status table of each processor core. One of the identified address segments is a contiguous subspace within the memory address space.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the cache coherency optimization method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the cache coherence optimization method as described in any one of claims 1 to 5.

Images