Many-core cache coherency system, method, electronic device, storage medium and product

Abstract

The disclosure provides a kind of many-core cache consistency system, method, electronic equipment, storage medium and product, system is based on network-on-chip architecture, by introducing multiple nodes, corresponding router and the distributed directory deployed in router.Each node contains one or more processor cores and distributed last level cache, memory address is divided into multiple address blocks and is associated with distributed last level cache by dynamic address mapping rule. Distributed directory is used to record the state of cache line with distributed last level cache one by one, and the router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directory and updating the cache line state, which realizes efficient management of cache consistency.The technical scheme belongs to the field of communication technology, effectively solves the problem of high load and communication delay of master node in traditional cache consistency protocol, and improves the data consistency and access efficiency in multi-processor system and large-scale parallel computing environment.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of communication technology, and in particular to a many-core cache coherence system, method, electronic device, storage medium and product. BACKGROUND

[0002] In modern computer architecture, especially in multi-processor systems and large-scale parallel computing environments, cache coherence is the key to ensuring data consistency and avoiding access conflicts. Traditional cache coherence protocols, such as the MESI protocol, manage cache coherence through a HN (Home Node). The HN is responsible for receiving requests from RNs (Request Nodes), querying directory information to determine the state of the cache line, and processing the request accordingly. In the MESI protocol, a cache line can be in one of four states: Modified, Exclusive, Shared, and Invalid.

[0003] However, traditional cache coherence protocols have obvious drawbacks. As the central point of cache coherence protocols, all cache coherence-related communications must pass through the HN, which leads to high load on the HN when the number of processors is large. In addition, since all cross-processor cache coherence communications need to pass through the HN, the centralized design of the HN increases communication latency. SUMMARY

[0004] The present disclosure provides a many-core cache coherence system, method, electronic device, storage medium and product to at least solve the above technical problems in the prior art.

[0005] According to a first aspect of the present disclosure, a many-core cache coherence system is provided, characterized in that the system comprises a plurality of nodes, routers corresponding to the nodes, and a distributed directory deployed in the routers; the routers are connected based on a network-on-chip communication;

[0006] Each node includes at least one processor core and a distributed last-level cache; memory addresses are divided into a plurality of address blocks, each address block is associated with the distributed last-level cache through a dynamic address mapping rule;

[0007] The distributed directory corresponds one-to-one to the distributed last-level cache and is used to record the state of the cache line corresponding to the distributed last-level cache associated with the router;

[0008] The router is configured to receive and respond to requests from local nodes or remote nodes, query whether the corresponding distributed directory hits, and feed back and update the state of the cache line in the distributed directory according to the hit result of the query.

[0009] In an implementation, the router comprises an input module, an output module and a cache coherence module;

[0010] The input module is configured to receive a request from a local node or a remote node and transmit the request to the cache coherence module;

[0011] The cache coherence module is configured to receive and respond to the request, acquire and feedback a response or a monitoring request, and modify a state of a cache line in a corresponding distributed directory;

[0012] The output module is configured to output the response or the monitoring request.

[0013] In an implementation, the cache coherence module is specifically configured to:

[0014] receive the request from the input module;

[0015] if the distributed directory of the local node hits and the state meets the request requirement, acquire a response feedback node from a corresponding distributed last-level cache; if the distributed directory of the local node hits but the state does not meet the request requirement, send a monitoring request to other remote nodes, feedback the request node after collecting monitoring responses, and update the state of the cache line in the corresponding distributed directory;

[0016] if the distributed directory of the local node does not hit, send the request to a remote node holding the cache line; if the distributed directory of the remote node hits and the state meets the request requirement, acquire a response feedback the request node from the distributed last-level cache; if the distributed directory of the remote node hits but the state does not meet the request requirement, send a monitoring request to other remote nodes, feedback the request node after collecting monitoring responses, and update the state of the cache line in the corresponding distributed directory;

[0017] if all nodes do not hit, acquire a response feedback the request node from the memory, and update the state of the cache line in the corresponding distributed directory;

[0018] update the state of the cache line in the distributed directory of the corresponding node.

[0019] In an implementation, the input module comprises an input queue and is further configured to perform merging processing on multiple requests to the same destination node according to a merging rule, and generate a single request to send to the cache coherence module.

[0020] In an implementation, the router further comprises a routing calculation module, an allocation module and a crossbar module;

[0021] The routing calculation module is configured to calculate a routing path of the request node according to a data packet, wherein the data packet comprises a request, a listening request, a listening response and a response.

[0022] The distribution module is configured to determine data packets to be outputted in each time period, and obtain a determination result.

[0023] The crossbar module is configured to connect the output module and the input module, and establish a connection or disconnect a specific input channel and a specific output channel according to the determination result and the routing path.

[0024] In an implementation, the routing calculation module is further configured to:

[0025] When the request node sends data packets to multiple destination nodes, the routing path of the data packets is determined according to a routing algorithm and a hop number of the destination nodes from the request node, so that the data packets pass through the multiple destination nodes one by one at one time.

[0026] In an implementation, when the distributed last-level cache is full, the cache coherence module preferentially replaces a cache line that is not occupied by a local node.

[0027] In an implementation, the dynamic address mapping rule is to determine an associated distributed last-level cache according to high N bits of a memory address, wherein N is a number of binary bits of the distributed last-level cache.

[0028] When a number of times of sending a request to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

[0029] In an implementation, a bit number of each entry in the distributed directory is determined by a total number of processor cores and a number of processor cores per node.

[0030] In an implementation, a plurality of processor cores in each node maintain intra-node cache coherence through a bus snooping protocol, and maintain cross-node cache coherence through the network-on-chip.

[0031] In an implementation, each processor core corresponds to an independent level-1 cache and a level-2 cache; the level-1 cache is connected to the level-2 cache, and the level-1 cache and the level-2 cache of different processor cores are not interconnected.

[0032] According to a second aspect of the present disclosure, a many-core cache coherence method is provided, the method being applied to a router, and the method comprising:

[0033] receiving a request of a local node or a remote node;

[0034] If the distributed directory of the local node hits and the status meets the request requirement, the response feedback request node is obtained from the corresponding distributed last-level cache; if the distributed directory of the local node hits but the status does not meet the request requirement, the listening request is sent to other remote nodes, the request node is fed back after the listening responses are collected, and the status of the cache line in the corresponding distributed directory is updated;

[0035] If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirement, the response feedback request node is obtained from the distributed last-level cache; if the distributed directory of the remote node hits but the status does not meet the request requirement, the listening request is sent to other remote nodes, the request node is fed back after the listening responses are collected, and the status of the cache line in the corresponding distributed directory is updated;

[0036] If all nodes do not hit, the response feedback request node is obtained from the memory, and the status of the cache line in the corresponding distributed directory is updated.

[0037] In an implementable manner, multiple requests to the same destination node are merged according to a merging rule to generate a single request.

[0038] In an implementable manner, the method further comprises:

[0039] According to the data packet, the route path for obtaining and feeding back the request node is calculated; the data packet includes a request, a listening request, a listening response and a response;

[0040] The data packet outputted in each time period is determined to obtain a determination result;

[0041] The output module and the input module are connected, and a specific input channel and an output channel are connected or disconnected according to the determination result and the route path.

[0042] In an implementable manner, the method further comprises:

[0043] When the request node sends data packets to multiple destination nodes, the route path of the data packet is determined according to a routing algorithm and the hop number of the destination node from the request node, so that the data packet passes through the multiple destination nodes one by one at a time.

[0044] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0045] at least one processor; and

[0046] a memory connected to the at least one processor in communication; wherein

[0047] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0048] According to a fourth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, the computer instructions being used to cause the computer to perform the method of the present disclosure.

[0049] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer programs or instructions for causing a processor to implement the method when executed.

[0050] The many-core cache coherence system, method, electronic device, storage medium and product of the present disclosure are based on a network-on-chip architecture, and by introducing a plurality of nodes, corresponding routers and a distributed directory deployed in the routers, efficient management of cache coherence is achieved. Each node includes one or more processor cores and a distributed last-level cache. Memory addresses are divided into a plurality of address blocks and associated with the distributed last-level cache through dynamic address mapping rules. The distributed directory corresponds to the distributed last-level cache one-to-one and records the state of the cache line to achieve cross-node data consistency. The router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directory and updating the cache line state. This technical solution reduces single-point bottlenecks through distributed cooperation, significantly reduces access delay, while supporting the scalability of a thousand-core system, effectively solves the problems of high load and communication delay of the master node in the traditional cache coherence protocol, and improves the data consistency and access efficiency in a multi-processor system and a large-scale parallel computing environment.

[0051] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0052] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will be more apparent from the following detailed description read in conjunction with the accompanying drawings, in which:

[0053] In the drawings, identical or corresponding reference numerals indicate identical or corresponding parts.

[0054] Figure 1 A structural schematic diagram of the many-core cache coherence system of the embodiment of the present disclosure is shown;

[0055] Figure 2 A structural schematic diagram of the router inside the embodiment of the present disclosure is shown;

[0056] Figure 3 A schematic diagram showing the internal structure of a node according to an embodiment of the present disclosure is shown;

[0057] Figure 4 The process diagram of the multi-core cache consistency method of the embodiment of the present disclosure is shown Figure 1 ;

[0058] Figure 5 The process diagram of the multi-core cache consistency method of the embodiment of the present disclosure is shown Figure 2 ;

[0059] Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0060] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0061] The present disclosure provides a many-core cache consistency system, such as Figure 1 As shown, the system includes multiple nodes (Cluster), routers (Router, represented by R in the figure) corresponding to the nodes, and a distributed directory deployed in the router (not shown in the figure); the router is connected based on on-chip network communication.

[0062] The many-core cache coherence system is built based on the Network-on-Chip (NoC) architecture. The NoC is an efficient communication architecture integrated within a single chip. It connects multiple processor cores, caches, memory controllers, and other functional modules through routers and interconnect links to enable data exchange and collaborative work between multi-core systems. Figure 1 The following is an example of a sixteen-core system consisting of 16 distributed directories, with only one processor core in each node, using a 4×4 Mesh NoC network. The specific components of the many-core cache coherence system are as follows:

[0063] Each node contains at least one processor core and a distributed last-level cache (Level 3 Cache, L3); the memory address is divided into multiple address blocks, and each address block is associated with the distributed last-level cache through dynamic address mapping rules.

[0064] In the present example, a node is a basic processing unit in a many-core cache coherent system, each node containing one or more processor cores and a distributed last level cache. The processor core is used to receive information and generate request instructions after processing. The distributed last level cache is a shared last level cache in a multi-core system, i.e. physically dispersed in multiple nodes of the system, but logically constitutes a unified address space, used to store frequently accessed memory data blocks. In addition, the memory address is divided into multiple address blocks, and the number of address blocks is equal to the number of nodes, so that each address block can be associated with the distributed last level cache of each node through a dynamic address mapping rule.

[0065] The distributed directory corresponds to the distributed last level cache, and is used to record the state of the cache line corresponding to the distributed last level cache associated with the router.

[0066] In the present example, the distributed directory is a metadata storage structure strictly bound to the distributed last level cache, each directory entry corresponding to a cache line, which is the smallest unit of data cached, used to record the global consistency state of the cache line and hold node information. Among them, the cache line state describes the permission attribute of the cache line in the consistency protocol, such as "modified state" (only the current node holds the latest data) and "shared state" (multiple nodes hold read-only copies), which is maintained by the distributed directory to achieve cross-node data consistency.

[0067] The router is used to receive and respond to requests from local nodes or remote nodes, query whether the corresponding distributed directory hits, and update the state of the cache line in the distributed directory according to the hit result of the query.

[0068] In the present example, the router assumes the core function of request scheduling and state coordination in the many-core cache coherent system. When the request of the local node (i.e. the node directly connected to the current router, such as the processor core initiating data reading) or the remote node (i.e. the node indirectly connected to other routers, forwarded through the on-chip network) arrives at the router, the router first parses the target address of the request, determines the home node corresponding to the address according to the dynamic address mapping rule, and then accesses the distributed directory bound to the node. According to the hit result of the directory query, feedback is given to the requesting node, and the router also needs to update the directory state according to the operation type (read operation or write operation) of the request to ensure data consistency. This mechanism significantly reduces access delay by reducing redundant memory access and fine-grained cache line state management, while avoiding single-point bottlenecks through distributed collaboration, supporting the scalability of a thousand-core system.

[0069] The disclosure proposes a many-core cache coherence system based on a network-on-chip architecture, which realizes efficient management of cache coherence by introducing multiple nodes, corresponding routers and a distributed directory deployed in the routers. Each node contains one or more processor cores and a distributed last-level cache. Memory addresses are divided into multiple address blocks and associated with the distributed last-level cache through dynamic address mapping rules. The distributed directory corresponds to the distributed last-level cache one-to-one and records the state of the cache line to realize cross-node data consistency. The router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directory and updating the cache line state. This technical solution reduces single-point bottlenecks through distributed cooperation, significantly reduces access delay, while supporting the scalability of a thousand-core system, effectively solving the problems of high load and communication delay of the master node in traditional cache coherence protocols, and improving data consistency and access efficiency in multi-processor systems and large-scale parallel computing environments.

[0070] In one embodiment, as shown in Figure 2 The router comprises an input module, an output module and a cache coherence module.

[0071] The input module is configured to receive requests from local nodes or remote nodes and transmit the requests to the cache coherence module.

[0072] In this example, the input module is responsible for receiving requests from local nodes or remote nodes, such as data read, data write, cache line state query and other operation requests. When the input module receives a request, it will divide the queue according to the request type and push the decoded request data packet to the cache coherence module through the internal bus.

[0073] The cache coherence module is configured to receive and respond to the requests, obtain and feedback responses or monitor requests, and modify the state of the cache line in the corresponding distributed directory.

[0074] In this example, the cache coherence module is the core protocol processing unit of the router, which implements the state machine logic of the system cache coherence protocol and is responsible for processing all operations related to cache coherence. The cache coherence module receives the requests transmitted by the input module and then performs corresponding operations according to the type of the request. Specifically, it queries the distributed directory associated with the target address, judges whether the cache line state hits, directly returns a response for a hit request, triggers a monitoring request for a miss request, and feeds back the request node according to the monitoring response. While feeding back the request node, the cache coherence module also updates the state of the cache in the distributed directory according to the operation type of the request.

[0075] It should be noted that the cache coherency module in the present disclosure is applicable to MSI, MESI, MESIF, MOESI, CHI, CCIX, ACE and other cache coherency protocols, and is not limited to a specific protocol. For different protocols, the state of the cache line recorded in the directory is different.

[0076] The output module is configured to output the response or the monitoring request.

[0077] In the present example, when the cache coherency module completes the processing of the request and generates a response, the output module transmits the response to the local request node or to a remote request node through other routers. For a monitoring request, the output module is responsible for notifying all relevant remote nodes of the modified cache line state.

[0078] In one embodiment, the cache coherency module is specifically configured to:

[0079] receive the request from the input module.

[0080] If the distributed directory of the local node hits and the state meets the request requirement, the response feedback request node is obtained from the corresponding distributed last-level cache; if the distributed directory of the local node hits but the state does not meet the request requirement, a monitoring request is sent to other remote nodes, the request node is fed back after all monitoring responses are received, and the state of the cache line in the corresponding distributed directory is updated.

[0081] In the present example, the cache coherency module first queries the distributed directory associated with the local node. If there is a target cache line record in the distributed directory and the state meets the request permission, for example, the request is a write operation request and requires exclusive state, the response of the request is directly obtained from the distributed last-level cache corresponding to the local node and fed back to the request node. The response can include the data information of the request or the state information. If the state recorded in the distributed directory does not meet the request requirement, for example, the request is a write request but the current node is in a shared state, a monitoring request is sent to other remote nodes holding the cache line to force them to perform state change (such as invalidation or degradation), and the final result is fed back to the request node after all monitoring responses are returned.

[0082] If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the state meets the request requirement, the response is obtained from the distributed last-level cache to feed back the request node; if the distributed directory of the remote node hits but the state does not meet the request requirement, a monitoring request is sent to other remote nodes, the request node is fed back after all monitoring responses are received, and the state of the cache line in the corresponding distributed directory is updated.

[0083] In this example, when the distributed directory corresponding to the local node does not hit the target cache line of the request, the cache coherence module forwards the request to the remote node responsible for the cache line according to the dynamic address mapping rule. If the distributed directory corresponding to the remote node hits and the state meets the requirements, the request is responded through the distributed last-level cache corresponding to the remote node. If the distributed directory of the remote node hits but the state does not meet the requirements, the monitoring process is recursively triggered: a monitoring request is sent to other cache line holders associated with the remote node, the state of the cache line in the corresponding distributed directory is updated until the request permission is met, the response is obtained from the distributed last-level cache of the remote node, and the response is returned to the source request node.

[0084] If all nodes do not hit, the response to the request node is obtained from the memory, and the state of the cache line in the corresponding distributed directory is updated.

[0085] In this example, when the distributed directory of all nodes does not hit the cache line corresponding to the request, the cache coherence module directly accesses the main memory to obtain the response. This process bypasses all cache levels, returns the data read from the memory to the request node, loads the new data into the last-level cache according to the cache replacement strategy, and updates the state of the corresponding distributed directory.

[0086] In one embodiment, the input module includes an input queue, and is further configured to merge a plurality of requests to the same destination node according to a merging rule to generate a single request and send the single request to the cache coherence module.

[0087] In this example, in the many-core cache coherence system, the input module of the router significantly reduces the message load and transmission delay of the on-chip network by designing an efficient request merging rule. Specifically, the input module includes an input queue for buffering requests from local nodes or other nodes, and merging a plurality of requests to the same destination node and having the same address according to a predefined merging rule.

[0088] Taking the CHI protocol as an example, the read request types include ReadOnce (single read), ReadClean (read clean data), ReadShared (read shared data), and ReadUnique (read unique data). The merging rule is divided into the following two categories:

[0089] Merging of the same type of request: for the same type of read request (for example, a plurality of ReadClean requests) to the same node and the same address, the input module merges the same type of read request into a single request. For example, if node A and node B send ReadClean requests to address 0x1000 of node C at the same time, the input queue identifies that the target addresses and types of the two are consistent, and then forwards a ReadClean request to the cache coherence module only once, and returns responses to node A and node B respectively after obtaining the data.

[0090] Combination of different types of requests: for different types of read requests sent to the same node and the same address, only ReadOnce can be combined with other types (such as ReadShared, ReadUnique). For example, if node A sends a ReadOnce request and node B sends a ReadShared request to the same address, since ReadOnce only needs to obtain a copy of the data and does not require the cache line state to be upgraded, its semantics is compatible with ReadShared, and thus can be combined into one ReadShared request; conversely, if node A sends a ReadClean request and node B sends a ReadUnique request, since ReadClean requires the data to be in a clean state, and ReadUnique needs to upgrade the cache line to an exclusive state, the semantics of the two are in conflict, and thus cannot be combined.

[0091] The single request after combination is uniformly processed by the cache consistency module. Taking the combined ReadShared request as an example, the module queries the distributed directory, and if the target cache line state is shared clean (SC), the data is directly returned to all requesting nodes; if the state is exclusive (U), a monitoring request is sent to the holding node to make it downgrade to a shared state, and then the data is returned. This mechanism reduces redundant message transmission by combining the monitoring, response and directory update operations of multiple requests into one, significantly reducing network congestion and consistency protocol overhead.

[0092] In one embodiment, as shown in FIG. 1, the router further comprises a routing calculation module, an allocation module and a crossbar module. Figure 2

[0093] The routing calculation module is configured to calculate the routing path to and from the requesting node according to the data packet, wherein the data packet comprises a request, a monitoring request, a monitoring response and a response.

[0094] In this example, the routing calculation module is the core component of the router responsible for dynamically planning the data packet transmission path. According to the type of data packet (such as request, monitoring request, monitoring response, response) and the destination information it carries, combined with the topology of the network on a chip, the module calculates the optimal routing path in real time. Specifically, when the routing calculation module receives a data packet, it first parses the content of the data packet to identify the type of the data packet and the destination information, and then calculates an optimal or feasible routing path from the current node to the target node according to the preset routing algorithm and strategy.

[0095] The allocation module is configured to determine the data packet to be output in each time period, and obtain a determination result.

[0096] ​In the present example, the allocation module is used to arbitrate the output priority of data packets in the input queue in each clock cycle, ensuring efficient use of network resources. The workflow is as follows: define the scheduling weight according to the data packet type, and preferentially send data packets with high scheduling weight. For example, the priority of the listening response is higher than that of the normal request, and the listening response is preferentially sent to avoid protocol timeout; for another example, the priority of the write request is higher than that of the read request, and the write request is preferentially sent to reduce data competition. In this way, when multiple data packets compete for the same output port, the allocation module selects the output order according to the pre-defined strategy, and buffers the unallocated data packets for processing in the next time cycle.

[0097] The crossbar module is used to connect the output module and the input module, and to establish or disconnect a specific input channel and an output channel according to the decision result and the routing path.

[0098] In the present example, the crossbar module is the core hardware unit for realizing the physical interconnection between the input and output channels in the router. It is located between the input module and the output module, and is responsible for dynamically configuring the connection between the input module and the output module in each clock cycle according to the output decision of the allocation module and the path information provided by the routing calculation module. When the allocation module decides that a data packet should be output, the crossbar module responds quickly, finds the input channel corresponding to the data packet and the output channel that should be connected, and then establishes the connection to enable the data packet to be smoothly transmitted to the corresponding node. At the same time, when the data packet is transmitted, the crossbar module also needs to timely disconnect the connection to free up channel resources for other data packets. This dynamic connection management mechanism enables the crossbar module to efficiently handle data packet forwarding tasks.

[0099] In one embodiment, the routing calculation module is further configured to: when a request node sends data packets to multiple destination nodes, determine the routing path of the data packets according to a routing algorithm and the number of hops from the request node to the destination nodes, so that the data packets pass through the multiple destination nodes one by one at a time.

[0100] In the present example, when a request node sends data packets to multiple destination nodes, the routing calculation module adopts a multi-destination path optimization strategy, i.e., determines the routing path of the data packets according to a routing algorithm and the number of hops from the request node to the destination nodes, so that the data packets pass through the multiple destination nodes one by one at a time, thereby reducing the load.

[0101] For example, if a listening request of a source node needs to be sent to nodes 1, 2, and 3, the routing calculation module generates a routing path for the data packet based on the hop number minimization principle according to the routing algorithm and the hop number of the destination node from the request node: source node→node 1→node 2→node 3, instead of sending three independent messages respectively. For the response data packet returned to the request node, the routing calculation module can directly use the shortest path to ensure low latency.

[0102] In addition, in a congestion scenario, the routing calculation module can also monitor the link load in real time and dynamically switch the path. For example, if the load of a certain link of the default path exceeds a threshold, a bypass path is selected, thereby reducing the load burden.

[0103] In one embodiment, when the distributed last-level cache is full, the cache consistency module preferentially replaces cache lines that are not occupied by the local node.

[0104] In the present example, when the storage space of the distributed last-level cache is full, the cache consistency module preferentially replaces cache lines that are not occupied by the local node, so as to maximize the data access efficiency of the local node. For example, in a system including 16 nodes, if the distributed last-level cache of node 1 is full and needs to replace cache lines, the cache consistency module first screens all cache lines that are not occupied by node 1, and then selects the cache lines to be replaced.

[0105] The data frequently accessed by the local node is more likely to be requested again in the future, and therefore retaining these data can improve the cache hit rate and reduce access latency. In contrast, the cache lines not occupied by the local node represent less frequently accessed data, and therefore replacing them has less impact on system performance. Through this strategy, the cache consistency module can more effectively utilize the limited cache resources and optimize the overall system performance.

[0106] In one embodiment, the dynamic address mapping rule is to determine the associated distributed last-level cache according to the high N bits of the memory address, where N is the number of binary bits of the distributed last-level cache. When the number of times of sending requests to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

[0107] In the present example, the dynamic address mapping rule divides the address block through the high N bits of the memory address (N is the number of binary bits of the distributed last-level cache) and binds it to the specified distributed last-level cache. For example, in a many-core cache consistency system including 16 nodes, N=4 (because 216=65536, which is greater than the number of nodes, so N=4). 4= 16), the high 4 bits of the memory address are used to determine the target distributed last level cache. The initial mapping is statically assigned, the address range 0000xxxxx (high 4 bits are 0000) is mapped to the distributed last level cache associated with router 1; the address range 0001xxxxx (high 4 bits are 0001) is mapped to the distributed last level cache associated with router 2, and so on.

[0108] The initial mapping is implemented by a hardware address decoder. When each node accesses the memory, the target router is directly located by the high N bits of the address, and the distributed directory thereof is queried to determine the cache hit state for further optimization of performance. The many-core cache coherence system also introduces a dynamic adjustment mechanism: when the number of cross-node requests received by a certain distributed last level cache exceeds a preset threshold (such as 1000 times per second), it is indicated that the address block of the distributed last level cache is frequently accessed by remote nodes, triggering address block remapping. For example, if the address block 0000xxxxx of the distributed last level cache 1 is frequently accessed by node 5, the system will dynamically remap it to the distributed last level cache 5 near node 5, thereby reducing the number of hops of cross-chip networks. The dynamic mapping process of the cache address is implemented by updating the global address mapping table, and the same is broadcast to all nodes to ensure consistency.

[0109] In one example, the number of bits of each entry in the distributed directory is determined by the total number of processor cores and the number of processor cores per node.

[0110] In this example, the data of each core within each node can be determined according to the total number of cores, and the state of each cache line in all nodes needs to be recorded in each distributed directory, so the number of bits of each entry in the directory is related to the number of nodes. Specifically, the number of bits of each entry in the distributed directory is determined by the total number of processor cores (M) and the number of processor cores per node (N), and the calculation formula is: directory entry bit number = M / N. For example, in a 1024-core system, if there is 1 core in each node, 1024 bits are needed in the directory to record the state of each node, and if there are 4 cores in each node, 256 bits are needed in the directory to record the state of each node.

[0111] In one embodiment, the plurality of processor cores within each node maintain the intra-node cache coherence through a bus snoop protocol, and the inter-node cache coherence is maintained through the on-chip network.

[0112] In this example, within each node, the plurality of processor cores maintain the intra-node cache coherence through a bus snoop protocol, ensuring that when a certain core modifies a cache line, other cores can timely perceive and update their cache copies, thereby maintaining data consistency. For example, in a node containing 4 cores, a write operation of core A on address X will broadcast an invalidation signal through the bus to make the cache lines holding X in cores B, C, and D invalid.

[0113] Between nodes, the system maintains cross-node cache coherence through the on-chip network. As an efficient communication mechanism, the on-chip network can quickly transmit cache coherence information, ensuring that cache data between different nodes remains synchronized. For example, when core A in node 1 accesses address X mapped to a remote node, the local router queries the directory. If it finds that X is held by node 5 and is in state U, it sends a snoop request to node 5 via the NoC, triggering a writeback of node 5's cache line and downgrading it to the S state. The data is then returned to core A in node 1.

[0114] By combining cache consistency maintenance strategies inside and outside the node, the system can ensure data accuracy and consistency in complex multi-core and multi-node environments.

[0115] In one embodiment, each processor core corresponds to an independent first-level cache and second-level cache; the first-level cache is connected to the second-level cache, and the first-level caches and second-level caches of different processor cores are not interconnected.

[0116] In this example, each processor core has an independent cache hierarchy design. In this design, each processor core has its own Level 1 cache (L1) and Level 2 cache (L2). As the cache level closest to the core, the L1 cache offers extremely fast access speeds but limited capacity. The L2 cache, on the other hand, provides a larger capacity to store more data copies. It's important to note that there's no direct data path between cache levels on different processor cores, meaning each processor core manages its cache resources independently.

[0117] Specifically, such as Figure 3 Figure 1 shows the internal structure of a node with four processor cores. Each processor core has a completely independent L1 instruction cache (L1-I), L1 data cache (L1-D), and L2 cache, with no direct data path between cache levels of different processor cores. For example, the L1 / L2 caches of processor core 1 are connected only to its private bus, and the cache levels of processor core 2 are similar. Both achieve coherency through a shared bus within the node and a snooping protocol. The L2 cache, as a private cache for the processor core, only caches data that misses in the local L1 and is not shared with other processor cores.

[0118] This design can reduce data access conflicts between processor cores. While improving cache utilization and access efficiency, the independent cache hierarchy also provides each processor core with greater flexibility and scalability, enabling the system to better adapt to different workloads and performance requirements.

[0119] In one embodiment, the process of implementing cache coherency for the many-core cache coherency system of the present disclosure will be illustrated by the following example.

[0120] In this embodiment, the state of the cache line in the directory can be I, SC, SD and U, using 2-bit encoding can represent the state of the cache line: 00 (I), 01 (SC), 10 (SD), 11 (U). Where I is invalid state, SC is shared clean state, SD is shared dirty state, and U is exclusive state.

[0121] A 16-core system composed of 16 distributed directories as shown in the figure, only one processor core in each node is taken as an example, using a 4x4 Mesh NoC network. Figure 1

[0122] The first row of nodes is numbered from left to right as 1, 2, 3, 4, the second row of nodes is numbered from left to right as 5, 6, 7, 8, the third row of nodes is numbered from left to right as 9, 10, 11, 12, and the fourth row of nodes is numbered from left to right as 13, 14, 15, 16. The associated distributed last level cache is determined by the high 4 bits of the memory address, and the memory address is divided into 16 address blocks and mapped into the 16 distributed last level caches respectively. For example, when the request address issued by the node is 0000xxxx, the cache hit or miss is determined by querying the distributed directory of the router corresponding to node 1; when the request address issued by the node is 0001xxxx, the cache hit or miss is determined by querying the distributed directory of the router corresponding to node 2; and so on, when the request address issued by the node is 1111xxxx, the cache hit or miss is determined by querying the distributed directory of the router corresponding to node 16.

[0123] ​Embodiment 1: When node 1 initiates a write request to the memory address 0000xxxx, the write request is directly sent to the router of the local node, and the router of the local node queries the local distributed directory, and the query result is a hit. At the same time, it is found that the state of node 4 corresponding to the address is 01, the state of node 8 is 01, and the state of node 15 is 10. By designing the format of the data packet in the NoC, the domain segment of the destination is set to multiple destination nodes, and a listening message with node 4, node 8 and node 15 as the destination nodes is sent, so that the three nodes are invalidated. The propagation path of the data packet containing the listening message in the 4x4 Mesh network is: from node 1, first transmitted to node 4 along the X-axis direction, then transmitted to node 8 along the Y-axis direction, then transmitted to 16 along the Y-axis, and then transmitted to node 15 through the X-axis direction. The transmission path can be changed according to the requirement of load balancing, as long as the three destinations are passed through, and the listening response message is waited for from node 4, node 8 and node 15, the distributed directory state is modified, and the cache consistency module in the router of node 1 sends a response to node 1.

[0124] Embodiment 2: When node 1 initiates a write request to the memory address 0000xxxx, the write request is directly sent to the router of the local node, and the router of the local node queries the local distributed directory, and the query result is a hit. If the corresponding distributed last-level cache is not full at this time, the data can be read from the memory to the cache or a listening request can be sent to other remote nodes; if the distributed last-level cache is full, a cache line needs to be evicted for cache replacement. When cache replacement is performed, the data not occupied by the local core can be replaced first, so that most of the data in the local cache is occupied by the local core, which can speed up the data access speed of the local core, and when the data access hit rate of each processor core in the local core is higher than that in other cores, the data access speed of the entire system can be improved.

[0125] Embodiment 3: When node 0, node 1 and node 2 all initiate a read request to the memory address 0000xxxxx, the three requests are transmitted to the input queue of the input module in the router corresponding to node 1, and the input queue finds that the three requests are for the same address and meet the merging rule, so the messages are merged and transmitted to the cache consistency module. The cache consistency module queries the distributed directory, and the state of node 3 is recorded as 11, so only one listening request needs to be sent to node 3 to make it 01 state. After the listening response is collected, responses are sent to node 0, node 1 and node 2 respectively, and the cache line state of the distributed directory of the address is modified.

[0126] The present disclosure also provides a many-core cache coherence method, which is applied in a router, as shown in the figure, the method comprises: Figure 4

[0127] Step 401: receiving a request of a local node or a remote node.

[0128] Step 402: if the distributed directory of the local node is hit and the state meets the request requirement, obtaining a response feedback request node from the corresponding distributed end cache; if the distributed directory of the local node is hit but the state does not meet the request requirement, sending a monitoring request to other remote nodes, and after receiving all the monitoring responses, feeding back the request node and updating the state of the cache line in the corresponding distributed directory.

[0129] Step 403: if the distributed directory of the local node is not hit, sending the request to the remote node holding the cache line; if the distributed directory of the remote node is hit and the state meets the request requirement, obtaining a response feedback request node from the distributed end cache; if the distributed directory of the remote node is hit but the state does not meet the request requirement, sending a monitoring request to other remote nodes, and after receiving all the monitoring responses, feeding back the request node and updating the state of the cache line in the corresponding distributed directory.

[0130] Step 404: if all nodes are not hit, obtaining a response feedback request node from the memory and updating the state of the cache line in the corresponding distributed directory.

[0131] In one embodiment, multiple requests to the same destination node are processed according to the merging rule to generate a single request.

[0132] In one embodiment, as shown in the figure, the method further comprises: Figure 5

[0133] Step 501: according to a data packet, calculating and obtaining a routing path for feeding back the request node; the data packet comprises a request, a monitoring request, a monitoring response and a response;

[0134] Step 502: determining the data packet to be output in each time period to obtain a determination result;

[0135] Step 503: connecting the output module and the input module, and establishing or disconnecting a specific input channel and an output channel according to the determination result and the routing path.

[0136] In one embodiment, the method further comprises:

[0137] ​​When the request node sends data packets to multiple destination nodes, a routing path of the data packets is determined according to a routing algorithm and a hop number of the destination nodes from the request node, so that the data packets pass through the multiple destination nodes one by one at one time.

[0138] According to embodiments of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0139] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0140] As shown in Figure 6 The electronic device 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0141] Various components in the device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc., an output unit 607, such as various types of displays, speakers, etc., the storage unit 608, such as a magnetic disk, an optical disk, etc., and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0142] The computing unit 601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the many-core cache coherency method. For example, in some embodiments, the many-core cache coherency method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded onto the RAM 603 and executed by the computing unit 601, one or more steps of the many-core cache coherency method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the many-core cache coherency method by any other suitable means, such as by means of firmware.

[0143] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0144] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0145] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0147] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0148] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0149] The embodiment of the present disclosure further provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the many-core cache consistency method described above in the embodiments of the present disclosure.

[0150] It should be understood that the steps shown above can be reordered, added, or deleted using various forms of flow. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which is not limited herein.

[0151] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0152] The above is merely specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A many-core cache consistency system, characterized in that: The system includes a plurality of nodes, routers corresponding to the nodes, and a distributed directory deployed in the routers, so as to maintain cache consistency of the system through the distributed directory; The router is connected based on on-chip network communication; Each node contains at least one processor core and a distributed last-level cache; The memory address is divided into a plurality of address blocks, and each address block is associated with the distributed last-level cache through a dynamic address mapping rule; The distributed directory corresponds to the distributed last-level cache in a one-to-one manner and is used to record the status of the cache line corresponding to the distributed last-level cache associated with the router; The router is used to receive and respond to requests from local nodes or remote nodes, and query whether the corresponding distributed directory is hit; Feedback is provided based on the query hit result and the status of the cached row in the distributed directory is updated; The router includes: an input module, an output module and a cache consistency module; The input module is used to receive requests from local nodes or remote nodes and transmit them to the cache consistency module; The cache consistency module is used to receive and respond to the request, obtain and feed back a response or listen request, and modify the state of the cache line in the corresponding distributed directory; The output module is used to output the response or monitoring request; The input module includes an input queue and is further configured to merge multiple requests to the same destination node according to a merge rule, including: For read requests of the same type sent to the same node and the same address, the input module merges them into a single request; for read requests of different types sent to the same node and the same address, only ReadOnce and other types are allowed to be merged into a single request; The single request is sent to the cache coherence module.

2. The system according to claim 1, wherein: The cache consistency module is specifically used to: Receive requests from input modules; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit the request, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the response is fed back to the requesting node, and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, get the response feedback from the memory to the requesting node and update the status of the cache line in the corresponding distributed directory; Updates the state of the cached row of the distributed directory in the corresponding node.

3. The system according to claim 2, characterized in that The router further comprises: a routing calculation module, a distribution module and a crossbar switch module; The routing calculation module is used to calculate and obtain the routing path of the requesting node and feedback the routing path of the requesting node according to the data packet; the data packet includes a request, a listening request, a listening response and a response; The allocation module is used to determine the data packets to be output in each time period and obtain a determination result; The crossbar switch module is used to connect the output module and the input module, and to connect or disconnect a specific input channel and an output channel according to the decision result and the routing path.

4. The system according to claim 3, characterized in that The routing calculation module is also used for: When a requesting node sends a data packet to multiple destination nodes, a routing path of the data packet is determined according to a routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

5. The system according to claim 1, wherein: When the distributed last-level cache is full, the cache coherence module preferentially replaces cache lines that are not occupied by local nodes.

6. The system according to claim 1, wherein: The dynamic address mapping rule is to determine the associated distributed last-level cache according to the high N bits of the memory address, where N is the number of binary bits of the distributed last-level cache; When the number of times the request is sent to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

7. The system according to claim 1, wherein: The number of bits of each entry in the distributed directory is determined by the total number of processor cores and the number of processor cores per node.

8. The system according to claim 1, wherein: The multiple processor cores in each node maintain intra-node cache consistency through a bus snooping protocol, and inter-node cache consistency is maintained between nodes through the on-chip network.

9. The system according to claim 1, wherein: Each processor core corresponds to an independent first-level cache and second-level cache; the first-level cache is connected to the second-level cache, and the first-level caches and second-level caches of different processor cores are not interconnected.

10. A multi-core cache consistency method, characterized in that: The method is applied to a router, and the method includes: Receive requests from local nodes or remote nodes; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit the request, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the response is fed back to the requesting node, and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, get the response feedback from the memory to the requesting node and update the status of the cache line in the corresponding distributed directory; Maintaining many-core cache consistency through the distributed directory; Multiple requests to the same destination node are merged according to the merging rules to generate a single request, including: For read requests of the same type sent to the same node and the same address, they are merged into a single request; for read requests of different types sent to the same node and the same address, only ReadOnce and other types are allowed to be merged into a single request.

11. The method according to claim 10, characterized in that The method further comprises: Calculate and obtain the routing path of the requesting node and feedback the routing path based on the data packet; the data packet includes a request, a listening request, a listening response, and a response; Determine the data packets to be output in each time period and obtain a determination result; The output module and the input module are connected, and according to the decision result and the routing path, a specific input channel and an output channel are connected or disconnected.

12. The method according to claim 10, characterized in that The method further comprises: When a requesting node sends a data packet to multiple destination nodes, a routing path of the data packet is determined according to a routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 10 to 12.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 10-12.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 10 to 12 is implemented.

Images