Many-core cache consistency system and method, electronic equipment, storage medium and product

Abstract

The invention provides a many-core cache consistency system, a many-core cache consistency method, electronic equipment, a storage medium and a product, and the system is based on a network-on-chip architecture, and a plurality of nodes, corresponding routers and distributed directories deployed in the routers are introduced. Each node comprises one or more processor cores and a distributed last-stage cache, and a memory address is divided into a plurality of address blocks and is associated with the distributed last-stage cache through a dynamic address mapping rule. The distributed directories and the distributed last-stage caches are in one-to-one correspondence and used for recording the states of cache lines, the router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directories and updating the states of the cache lines, and efficient management of cache consistency is achieved. The technical scheme belongs to the technical field of communication, effectively solves the problems of high load and communication delay of a main node in a traditional cache consistency protocol, and improves data consistency and access efficiency in a multiprocessor system and a large-scale parallel computing environment.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a many-core cache consistency system, method, electronic device, storage medium and product. Background Art

[0002] In modern computer architectures, especially in multi-processor systems and massively parallel computing environments, cache coherence is key to ensuring data consistency and avoiding access conflicts. Traditional cache coherence protocols, such as the MESI protocol, manage cache coherence through HN (Home Node). HN is responsible for receiving requests from RN (Request Node), querying directory information to determine the status of the cache line, and processing the request accordingly. In the MESI protocol, a cache line can be in four states: Modified, Exclusive, Shared, and Invalid.

[0003] However, traditional cache consistency protocols have obvious disadvantages. As the central point of the cache consistency protocol, all cache consistency-related communications must pass through HN, which causes HN to face high load when the number of processors is large. In addition, since all cache consistency communications across processors need to pass through the single path of HN, the centralized design of HN increases communication latency. Summary of the invention

[0004] The present disclosure provides a many-core cache consistency system, method, electronic device, storage medium and product to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, a many-core cache consistency system is provided, characterized in that the system comprises a plurality of nodes, routers corresponding to the nodes, and a distributed directory deployed in the routers; the routers are connected based on on-chip network communication; Each node includes at least one processor core and a distributed last-level cache; the memory address is divided into a plurality of address blocks, and each address block is associated with the distributed last-level cache through a dynamic address mapping rule; The distributed directory corresponds to the distributed last-level cache in a one-to-one manner, and is used to record the status of the cache line corresponding to the distributed last-level cache associated with the router; The router is used to receive and respond to a request from a local node or a remote node, query whether the corresponding distributed directory is hit; and provide feedback and update the state of the cache line in the distributed directory according to the query hit result.

[0006] In one possible implementation, the router includes: an input module, an output module, and a cache consistency module; The input module is used to receive a request from a local node or a remote node and transmit it to a cache consistency module; The cache consistency module is used to receive and respond to the request, obtain and feedback the response or monitor the request, and modify the state of the cache line in the corresponding distributed directory; The output module is used to output the response or monitoring request.

[0007] In one possible implementation, the cache consistency module is specifically used to: Receive requests from input modules; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, get the response feedback from the memory to the requesting node, and update the status of the cache line in the corresponding distributed directory; Updates the state of the cache line of the distributed directory in the corresponding node.

[0008] In one possible implementation, the input module includes an input queue, and is further used to merge multiple requests from the same destination node according to a merge rule, and generate a single request to send to the cache consistency module.

[0009] In one possible implementation, the router further includes: a routing calculation module, a distribution module and a crossbar switch module; The routing calculation module is used to calculate and obtain and feedback the routing path of the requesting node according to the data packet; the data packet includes a request, a listening request, a listening response and a response; The allocation module is used to determine the data packets to be output in each time period and obtain a determination result; The cross switch module is used to connect the output module and the input module, and to establish a connection or disconnect a specific input channel and an output channel according to the decision result and the routing path.

[0010] In one possible implementation, the routing calculation module is further used for: When the requesting node sends a data packet to multiple destination nodes, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

[0011] In one possible implementation, when the distributed last-level cache is full, the cache coherence module preferentially replaces cache lines that are not occupied by local nodes.

[0012] In one possible implementation, the dynamic address mapping rule is to determine the associated distributed last-level cache according to the high N bits of the memory address, where N is the number of binary bits of the distributed last-level cache; When the number of times the request is sent to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

[0013] In one possible implementation, the number of bits of each entry in the distributed directory is determined by the total number of processor cores and the number of processor cores per node.

[0014] In one possible implementation, multiple processor cores in each node maintain intra-node cache consistency through a bus snooping protocol, and inter-node cache consistency is maintained between nodes through the on-chip network.

[0015] In one possible implementation, each processor core corresponds to an independent first-level cache and second-level cache; the first-level cache is connected to the second-level cache, and the first-level caches and second-level caches of different processor cores are not interoperable.

[0016] According to a second aspect of the present disclosure, a multi-core cache consistency method is provided, the method being applied in a router, the method comprising: Receive requests from local nodes or remote nodes; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, a response is obtained from the memory to feed back the requesting node, and the status of the cache line in the corresponding distributed directory is updated.

[0017] In one possible implementation, multiple requests to the same destination node are merged according to a merging rule to generate a single request.

[0018] In one embodiment, the method further comprises: Calculate, obtain and feedback the routing path of the requesting node according to the data packet; the data packet includes a request, a listening request, a listening response and a response; Determine the data packets to be output in each time period and obtain the determination result; The output module and the input module are connected, and according to the decision result and the routing path, a specific input channel and an output channel are connected or disconnected.

[0019] In one embodiment, the method further comprises: When the requesting node sends a data packet to multiple destination nodes, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

[0020] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0021] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program or instructions for causing a processor to implement the method when executing the program or instructions.

[0023] The present invention discloses a multi-core cache consistency system, method, electronic device, storage medium and product. The system is based on a network-on-chip architecture and implements efficient management of cache consistency by introducing multiple nodes, corresponding routers and distributed directories deployed in the routers. Each node contains one or more processor cores and a distributed last-level cache. The memory address is divided into multiple address blocks and associated with the distributed last-level cache through dynamic address mapping rules. The distributed directory corresponds one-to-one with the distributed last-level cache, records the status of the cache line, and implements cross-node data consistency. The router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directory and updating the cache line status. This technical solution reduces single-point bottlenecks through distributed collaboration, significantly reduces access latency, and supports the scalability of thousands of core systems. It effectively solves the problems of high load and communication latency of the master node in traditional cache consistency protocols, and improves data consistency and access efficiency in multi-processor systems and large-scale parallel computing environments.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0026] Figure 1 A schematic diagram of the structure of a multi-core cache consistency system according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram showing the internal structure of a router according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram showing the internal structure of a node according to an embodiment of the present disclosure is shown; Figure 4 The schematic diagram shows the process of the multi-core cache consistency method of the embodiment of the present disclosure. Figure 1 ; Figure 5 The schematic diagram shows the process of the multi-core cache consistency method of the embodiment of the present disclosure. Figure 2 ; Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0028] The present disclosure provides a many-core cache consistency system, such as Figure 1 As shown, the system includes multiple nodes (Cluster), routers (Router, represented by R in the figure) corresponding to the nodes, and a distributed directory (not shown in the figure) deployed in the router; the router is connected based on on-chip network communication.

[0029] The many-core cache consistency system is built based on the Network-on-Chip (NoC) architecture. The NoC is an efficient communication architecture integrated in a single chip. It connects multiple processor cores, caches, memory controllers and other functional modules through routers and interconnection links to achieve data exchange and collaborative work between multi-core systems. Figure 1 A sixteen-core system consisting of 16 distributed directories is shown as an example, with only one processor core in each node, and a 4×4 Mesh NoC network is used. The specific composition of each part of the many-core cache consistency system is as follows: Each node includes at least one processor core and a distributed last-level cache (Level 3 Cache, L3); the memory address is divided into multiple address blocks, and each address block is associated with the distributed last-level cache through a dynamic address mapping rule.

[0030] In this example, a node is a basic processing unit in a multi-core cache consistency system, and each node contains one or more processor cores and a distributed last-level cache. The processor core is used to receive information and generate request instructions after processing. The distributed last-level cache is a shared last-level cache in a multi-core system, that is, it is physically dispersed among multiple nodes of the system, but logically constitutes a unified address space for storing high-frequency access memory data blocks. In addition, the memory address is divided into multiple address blocks, and the number of address blocks is equal to the number of nodes. Therefore, each address block can be associated with the distributed last-level cache of each node through dynamic address mapping rules.

[0031] The distributed directory corresponds to the distributed last-level cache in a one-to-one manner, and is used to record the status of the cache line corresponding to the distributed last-level cache associated with the router.

[0032] In this example, the distributed directory is a metadata storage structure that is strictly bound to the distributed last-level cache. Each directory entry corresponds to a cache line, which is the smallest data unit of the cache and is used to record the global consistency state of the cache line and hold node information. The cache line state describes the permission attributes of the cache line in the consistency protocol, such as "modified state" (only the current node holds the latest data), "shared state" (multiple nodes hold read-only copies), etc., which are maintained by the distributed directory to achieve cross-node data consistency.

[0033] The router is used to receive and respond to a request from a local node or a remote node, query whether the corresponding distributed directory is hit; and provide feedback and update the state of the cache line in the distributed directory according to the query hit result.

[0034] In this example, the router assumes the core functions of request scheduling and state coordination in the many-core cache consistency system. When a request from a local node (i.e., a node directly connected to the current router, such as a processor core initiating data reading) or a remote node (i.e., a node indirectly connected to other routers, forwarded through the on-chip network) reaches the router, the router first resolves the target address of the request, determines the corresponding home node of the address according to the dynamic address mapping rules, and then accesses the distributed directory bound to the node. Feedback is given to the requesting node based on the directory query hit result, and the router also needs to update the directory status according to the requested operation type (read operation or write operation) to ensure data consistency. This mechanism significantly reduces access latency by reducing redundant memory accesses and refining the state management of cache lines, while avoiding single-point bottlenecks through distributed collaboration, supporting the scalability of thousand-core systems.

[0035] The present disclosure proposes a multi-core cache consistency system, which is based on an on-chip network architecture and realizes efficient management of cache consistency by introducing multiple nodes, corresponding routers, and distributed directories deployed in the routers. Each node contains one or more processor cores and a distributed last-level cache. The memory address is divided into multiple address blocks and associated with the distributed last-level cache through dynamic address mapping rules. The distributed directory corresponds one-to-one with the distributed last-level cache, records the status of the cache line, and realizes cross-node data consistency. The router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directory and updating the cache line status. This technical solution reduces single-point bottlenecks through distributed collaboration, significantly reduces access latency, and supports the scalability of thousands of core systems. It effectively solves the problems of high load and communication latency of the master node in traditional cache consistency protocols, and improves data consistency and access efficiency in multi-processor systems and large-scale parallel computing environments.

[0036] In one embodiment, Figure 2As shown, the router includes: an input module, an output module and a cache consistency module; The input module is used to receive requests from local nodes or remote nodes and transmit them to the cache consistency module.

[0037] In this example, the input module is responsible for receiving requests from local nodes or remote nodes, such as data read, data write, cache line status query, etc. After receiving the request, the input module divides the queue according to the request type and pushes the decoded request data packet to the cache consistency module through the internal bus.

[0038] The cache consistency module is used to receive and respond to the request, obtain and feed back a response or monitor the request, and modify the state of the cache line in the corresponding distributed directory.

[0039] In this example, the cache consistency module is the core protocol processing unit of the router, which implements the state machine logic of the system cache consistency protocol and is responsible for processing all operations related to cache consistency. The cache consistency module receives requests from the input module and then performs corresponding operations according to the type of request. Specifically, it queries the distributed directory associated with the target address to determine whether the cache line status is a hit. It directly returns a response to a hit request, triggers a listening request for a miss request, and feeds back the requesting node based on the listening response. While feeding back the requesting node, the cache consistency module also updates the status of the cache in the distributed directory according to the type of operation requested.

[0040] It should be noted that the cache consistency module in the present disclosure is applicable to cache consistency protocols such as MSI, MESI, MESIF, MOESI, CHI, CCIX, ACE, etc., and is not limited to a certain established protocol. For different protocols, the status of the cache lines recorded in the directory is different.

[0041] The output module is used to output the response or monitoring request.

[0042] In this example, when the cache coherence module completes the processing of the request and generates a response, the output module transmits the response to the local requesting node or to the remote requesting node through other routers. For listening requests, the output module is responsible for notifying all relevant remote nodes of the modified cache line status.

[0043] In one embodiment, the cache consistency module is specifically used to: Receives requests from input modules.

[0044] If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache to feedback the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated.

[0045] In this example, the cache consistency module first queries the distributed directory associated with the local node. If the target cache line record exists in the distributed directory and the status meets the request permissions, for example, the request is a write operation request and an exclusive state is required, then the response to the request is obtained directly from the distributed last-level cache corresponding to the local node and fed back to the requesting node. The response can include both the requested data information and the status information. If the record status in the distributed directory does not meet the request requirements, for example, the request is a write request but the current node is in a shared state, then a listening request is sent to other remote nodes holding the cache line to force them to perform a state change (such as invalidation or degradation). After all listening responses are returned, the final result is fed back to the requesting node.

[0046] If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated.

[0047] In this example, when the distributed directory corresponding to the local node does not hit the target cache line of the request, the cache consistency module forwards the request to the remote node responsible for the cache line according to the dynamic address mapping rules. If the distributed directory corresponding to the remote node hits and the status meets the requirements, the request is responded to through its corresponding distributed last-level cache. If the distributed directory of the remote node hits but the status does not meet the requirements, the monitoring process is recursively triggered: a monitoring request is sent to other cache line holders associated with the remote node, the status of the cache line in the corresponding distributed directory is updated until the request permission is met, and the response is obtained from the distributed last-level cache of the remote node and returned to the source request node.

[0048] If all nodes miss, a response is obtained from the memory to feed back the requesting node, and the status of the cache line in the corresponding distributed directory is updated.

[0049] In this example, when the distributed directories of all nodes do not hit the cache line corresponding to the request, the cache consistency module will directly access the main memory to obtain the response. This process bypasses all cache levels, returns the data read from the memory to the requesting node, and loads the new data into the last-level cache according to the cache replacement policy, and updates the corresponding distributed directory status.

[0050] In one embodiment, the input module includes an input queue, and is further used to merge multiple requests from the same destination node according to a merge rule, and generate a single request to send to the cache consistency module.

[0051] In this example, in the many-core cache consistency system, the input module of the router significantly reduces the message load and transmission delay of the on-chip network by designing efficient request merging rules. Specifically, the input module includes an input queue for caching requests from local nodes or other nodes, and merging multiple requests sent to the same destination node and with the same address according to predefined merging rules.

[0052] Taking the CHI protocol as an example, read request types include ReadOnce (single read), ReadClean (read clean data), ReadShared (read shared data) and ReadUnique (read exclusive data). The merge rules are divided into the following two categories: Merging of requests of the same type: For read requests of the same type (such as multiple ReadClean requests) sent to the same node and the same address, the input module merges them into a single request. For example, if node A and node B simultaneously send ReadClean requests to address 0x1000 of node C, the input queue recognizes that the target addresses and types of the two are consistent, and only forwards the ReadClean request once to the cache consistency module, and returns responses to nodes A and B respectively after obtaining the data.

[0053] Merging of different types of requests: For different types of read requests sent to the same node and the same address, only ReadOnce is allowed to be merged with other types (such as ReadShared, ReadUnique). For example, if node A sends a ReadOnce request and node B sends a ReadShared request to the same address, since ReadOnce only needs to obtain a copy of the data and does not require the cache line status to be upgraded, its semantics are compatible with ReadShared, so they can be merged into one ReadShared request; conversely, if node A sends a ReadClean request and node B sends a ReadUnique request, since ReadClean requires the data to be in a clean state, while ReadUnique requires the cache line to be upgraded to an exclusive state, the two have conflicting semantics and cannot be merged.

[0054] The merged single request is uniformly processed by the cache consistency module. Taking the merged ReadShared request as an example, the module queries the distributed directory. If the target cache line status is shared clean (SC), the data is directly returned to all requesting nodes; if the status is exclusive (U), a listening request is sent to the holding node to downgrade it to a shared state, and then the data is returned. This mechanism significantly reduces network congestion and consistency protocol overhead by reducing redundant message transmission and combining the listening, response, and directory update operations of multiple requests into one.

[0055] In one embodiment, Figure 2 As shown, the router further includes: a routing calculation module, a distribution module and a cross switch module; The routing calculation module is used to calculate, obtain and feed back the routing path of the requesting node according to the data packet; the data packet includes a request, a monitoring request, a monitoring response and a response.

[0056] In this example, the routing calculation module is the core component of the router responsible for dynamically planning the data packet transmission path. This module calculates the optimal routing path in real time based on the type of data packet (such as request, listening request, listening response, response) and the destination information it carries, combined with the topology of the on-chip network. Specifically: When the routing calculation module receives a data packet, it will first parse the content of the data packet, identify the type of data packet and the destination information, and then calculate an optimal or feasible routing path from the current node to the target node based on the preset routing algorithm and strategy.

[0057] The allocation module is used to determine the data packets to be output in each time period and obtain a determination result.

[0058] In this example, the allocation module is used to arbitrate the output priority of the data packets in the input queue within each clock cycle to ensure the efficient use of network resources. Its workflow is as follows: define the scheduling weight according to the data packet type, and give priority to sending data packets with high scheduling weight. For example, the priority of the listening response is higher than that of the ordinary request, so the listening response is sent first to avoid protocol timeout; for another example, the priority of the write request is higher than that of the read request, so the write request is sent first to reduce data competition. In this way, when multiple data packets compete for the same output port, the allocation module selects the output order according to the predefined strategy, and caches the unallocated data packets until the next time period for processing.

[0059] The cross switch module is used to connect the output module and the input module, and to establish a connection or disconnect a specific input channel and an output channel according to the decision result and the routing path.

[0060] In this example, the crossbar module is the core hardware unit that realizes the physical interconnection between the input and output channels inside the router. It is located between the input module and the output module, and is responsible for dynamically configuring the connection between the input module and the output module in each clock cycle according to the output decision of the allocation module and the path information provided by the routing calculation module. When the allocation module decides that a data packet should be output, the crossbar module will respond quickly, find the input channel corresponding to the data packet and the output channel that should be connected, and then establish a connection so that the data packet can be smoothly transmitted to the corresponding node. At the same time, when the data packet is transmitted, the crossbar module also needs to disconnect the connection in time to free up channel resources for other data packets. This dynamic connection management mechanism enables the crossbar module to efficiently handle data packet forwarding tasks.

[0061] In one embodiment, the routing calculation module is also used to: when a requesting node sends a data packet to multiple destination nodes, determine the routing path of the data packet based on a routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

[0062] In this example, when the requesting node sends a data packet to multiple destination nodes, the routing calculation module adopts a multi-destination path optimization strategy, that is, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time, thereby reducing the load.

[0063] For example, if a source node's listening request needs to be sent to nodes 1, 2, and 3, the routing calculation module generates a routing path for the data packet based on the routing algorithm and the number of hops from the destination node to the requesting node based on the principle of minimizing the number of hops: source node → node 1 → node 2 → node 3, rather than sending three independent messages separately. For the response data packet returned to the requesting node, the routing calculation module can directly use the shortest path to ensure low latency.

[0064] Moreover, in congestion scenarios, the routing calculation module can also monitor the link load in real time and dynamically switch paths. For example, if the load of a link on the default path exceeds the threshold, a detour path is selected to reduce the load burden.

[0065] In one embodiment, when the distributed last-level cache is full, the cache coherence module preferentially replaces cache lines that are not occupied by local nodes.

[0066] In this example, when the storage space of the distributed last-level cache is full, the cache consistency module will give priority to replacing cache lines that are not occupied by local nodes to maximize the data access efficiency of local nodes. For example, in a system containing 16 nodes, if the distributed last-level cache of node 1 is full and the cache line needs to be replaced, the cache consistency module first screens out all cache lines that are not occupied by node 1, and then selects the cache line to be replaced.

[0067] Data frequently accessed by the local node is more likely to be requested again in the future, so retaining this data can improve the cache hit rate and reduce access latency. In contrast, cache lines not occupied by the local node represent less frequently accessed data, so replacing them has less impact on system performance. With this strategy, the cache consistency module can more effectively utilize limited cache resources and optimize overall system performance.

[0068] In one embodiment, the dynamic address mapping rule is to determine the associated distributed last-level cache according to the high N bits of the memory address, where N is the number of binary bits of the distributed last-level cache. When the number of requests sent to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

[0069] In this example, the dynamic address mapping rule divides the address block by the high N bits of the memory address (N is the number of binary bits of the distributed last-level cache) and binds it to the specified distributed last-level cache. For example, in a multi-core cache coherence system including 16 nodes, N=4 (because 2 4 =16), the upper 4 bits of the memory address are used to determine the target distributed last-level cache. The initial mapping is statically allocated, and the address range 0000xxxxx (the upper 4 bits are 0000) is mapped to the distributed last-level cache associated with router 1; the address range 0001xxxxx (the upper 4 bits are 0001) is mapped to the distributed last-level cache associated with router 2, and so on.

[0070] The initial mapping is implemented through a hardware address decoder. When each node accesses the memory, the target router is directly located by the high N bits of the address, and its distributed directory is queried to determine the cache hit status. To further optimize performance, the many-core cache consistency system also introduces a dynamic adjustment mechanism: when the number of cross-node requests received by a distributed last-level cache exceeds the preset threshold (such as 1000 times / second), it means that the address block of the distributed last-level cache is frequently accessed by remote nodes, triggering address block remapping. For example, if the address block 0000xxxxx of distributed last-level cache 1 is frequently accessed by node 5, the system will dynamically remap it to distributed last-level cache 5 near node 5, thereby reducing the number of cross-chip Internet hops. The dynamic mapping process of the cache address is achieved by updating the global address mapping table and broadcasting it to all nodes to ensure consistency.

[0071] In one example, the number of bits of each entry in the distributed directory is determined by the total number of processor cores and the number of processor cores per node.

[0072] In this example, the data of the core in each node can be determined based on the total number of cores. Each distributed directory needs to record the status of the cache line in all nodes, so the number of bits of each entry in the directory is related to the number of nodes. Specifically, the number of bits of each entry in the distributed directory is determined by the total number of processor cores (M) and the number of processor cores per node (N), and the calculation formula is: number of bits of directory entry = M / N. For example, in a 1024-core system, if there is 1 core in each node, 1024 bits are required in the directory to record the status of each node. If there are 4 cores in each node, 256 bits are required in the directory to record the status of each node.

[0073] In one embodiment, multiple processor cores in each node maintain intra-node cache consistency through a bus snooping protocol, and inter-node cache consistency is maintained through the on-chip network between nodes.

[0074] In this example, multiple processor cores in each node use a bus snooping protocol to maintain cache consistency within the node, ensuring that when a core modifies a cache line, other cores can sense and update their cache copies in a timely manner to maintain data consistency. For example, in a node containing four cores, a write operation to address X by core A will broadcast an invalidation signal through the bus, invalidating the cache lines holding X in cores B, C, and D.

[0075] Between nodes, the system uses the on-chip network to maintain cache consistency across nodes. As an efficient communication mechanism, the on-chip network can quickly transmit cache consistency information to ensure that cache data between different nodes remains synchronized. For example, when address X accessed by core A in node 1 is mapped to a remote node, the local router queries the directory. If it finds that X is held by node 5 and the state is U, it sends a snoop request to node 5 through the NoC, triggering the cache line of node 5 to be written back and downgraded to the S state, and then returns the data to core A in node 1.

[0076] By combining cache consistency maintenance strategies inside and outside the node, the system can ensure data accuracy and consistency in a complex multi-core and multi-node environment.

[0077] In one embodiment, each processor core corresponds to an independent first-level cache and second-level cache; the first-level cache is connected to the second-level cache, and the first-level caches and second-level caches of different processor cores are not interoperable.

[0078] In this example, each processor core has an independent cache hierarchy design, in which each processor core has an independent Level 1 Cache (L1) and Level 2 Cache (L2). As the cache level closest to the core, the Level 1 cache has extremely fast access speed but limited capacity. The Level 2 cache provides a larger capacity to store more data copies. It should be noted that there is no direct data path between the cache levels of different processor cores, which means that each processor core manages its cache resources independently.

[0079] Specifically, Figure 3 The internal structure of a node with four processor cores is shown. Each processor core is configured with completely independent L1 instruction cache (L1-I), L1 data cache (L1-D), and L2 cache, and there is no direct data path between the cache levels of different processor cores. For example, the L1 / L2 cache of processor core 1 is only connected to its private bus, and the cache level of processor core 2 is the same. The two achieve consistency through a shared bus and snooping protocol within the node. The L2 cache is a private cache of the processor core, which only caches data that is not hit by the local L1, and is not shared with other processor cores.

[0080] Such a design can reduce data access conflicts between processor cores. While improving cache utilization and access efficiency, the independent cache hierarchy also provides each processor core with greater flexibility and scalability, enabling the system to better adapt to different workloads and performance requirements.

[0081] In one embodiment, the process of achieving cache consistency in the many-core cache consistency system of the present disclosure will be illustrated by the following example.

[0082] This embodiment takes the CHI protocol as an example. The state of the cache line in the directory can be I, SC, SD and U. The state of the cache line can be represented by a 2-bit code: 00 (I), 01 (SC), 10 (SD), 11 (U). Among them, I is an invalid state, SC is a shared clean state, SD is a shared dirty state, and U is an exclusive state.

[0083] by Figure 1 The sixteen-core system composed of 16 distributed directories shown in the figure has only one processor core in each node as an example, and adopts a 4×4 Mesh NoC network.

[0084] The nodes in the first row are numbered 1, 2, 3, and 4 from left to right, the nodes in the second row are numbered 5, 6, 7, and 8 from left to right, the nodes in the third row are numbered 9, 10, 11, and 12 from left to right, and the nodes in the fourth row are numbered 13, 14, 15, and 16 from left to right. The associated distributed last-level cache is determined by the high 4 bits of the memory address, and the memory address is divided into 16 address blocks and mapped to the corresponding 16 distributed last-level caches. For example, when the request address sent by the node is 0000xxxx, the distributed directory of the router corresponding to node 1 is queried to determine whether the cache hits; when the request address sent by the node is 0001xxxx, the distributed directory of the router corresponding to node 2 is queried to determine whether the cache hits, and so on. When the request address sent by the node is 1111xxxx, the distributed directory of the router corresponding to node 16 is queried to determine whether the cache hits.

[0085] Embodiment 1: When node 1 initiates a write request to the memory address 0000xxxx, the write request is directly sent to the router of the local node, and the router of the local node queries the local distributed directory, and the query result is a hit. At the same time, it is found that the state of node 4 corresponding to the address is 01, the state of node 8 is 01, and the state of node 15 is 10. By designing the format of the data packet in the NoC, setting the domain segment of the destination to multiple destination nodes, sending a listening message with the destination nodes being node 4, node 8 and node 15, the three nodes are invalidated. In the 4×4Mesh network, the propagation path of the data packet containing the listening message is: starting from node 1, first transmitted to node 4 along the X-axis direction, then transmitted to node 8 along the Y-axis direction, continued to transmit to 16 along the Y-axis, and then transmitted to node 15 through the X-axis direction. The transmission path can be changed according to the requirements of load balancing, as long as it is guaranteed to pass through three destinations, waiting for nodes 4, 8 and 15 to return the listening response message, modifying the distributed directory state, and the cache consistency module in the router in node 1 sends a response to node 1.

[0086] Embodiment 2: When node 1 initiates a write request to the memory address 0000xxxx, the write request is directly sent to the router of the local node, and the router of the local node queries the local distributed directory, and the query result is a non-hit. If the corresponding distributed last-level cache is not full at this time, data can be read from the memory into the cache or a listening request can be sent to other remote nodes; if the distributed last-level cache is full, a cache line needs to be evicted for cache replacement. When performing cache replacement, data that is not occupied by the local core can be replaced first, so that most of the data in the local cache is occupied by the local core, and the data access speed of the local core can be accelerated. When the hit rate of the data that each processor core wants to access in the local core is higher than the hit rate in other cores, the memory access speed of the entire system can be improved.

[0087] Example 3: When node 0, node 1 and node 2 all initiate a read request for memory address 0000xxxxx, these three requests are transmitted to the input queue of the input module in the router corresponding to node 1. The input queue finds that the three requests are for the same address and meet the merge rule, so the message is merged and transmitted to the cache consistency module. The cache consistency module queries the distributed directory, which records the state of node 3 as 11, so it only needs to send a listening request to node 3 once to change it to 01 state. After receiving all the listening responses, it sends responses to node 0, node 1 and node 2 respectively, and modifies the cache line state of the distributed directory in the address.

[0088] The present disclosure also provides a multi-core cache consistency method, which is applied to a router, such as Figure 4 As shown, the method includes: Step 401: Receive a request from a local node or a remote node.

[0089] Step 402: If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache to feed back the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated.

[0090] Step 403: If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated.

[0091] Step 404: If all nodes miss, obtain a response from the memory to feed back the requesting node, and update the status of the cache line in the corresponding distributed directory.

[0092] In one embodiment, multiple requests to the same destination node are merged according to a merge rule to generate a single request.

[0093] In one embodiment, Figure 5 As shown, the method also includes: Step 501: Calculate and obtain the routing path of the requesting node and feedback the routing path according to the data packet; the data packet includes a request, a listening request, a listening response and a response; Step 502: Determine the data packets to be output in each time period and obtain a determination result; Step 503: Connect the output module and the input module, and establish or disconnect a specific input channel and an output channel according to the decision result and the routing path.

[0094] In one embodiment, the method further comprises: When the requesting node sends a data packet to multiple destination nodes, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0096] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0097] like Figure 6As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0098] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the many-core cache consistency method. For example, in some embodiments, the many-core cache consistency method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the many-core cache consistency method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the many-core cache consistency method in any other appropriate manner (e.g., by means of firmware).

[0100] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0102] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0105] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0106] The embodiment of the present disclosure also provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the multi-core cache consistency method described in the embodiment of the present application.

[0107] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0108] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0109] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A many-core cache consistency system, characterized in that: The system includes a plurality of nodes, routers corresponding to the nodes, and a distributed directory deployed in the routers; the routers are connected based on on-chip network communication; Each node contains at least one processor core and a distributed last-level cache; The memory address is divided into a plurality of address blocks, each address block is associated with the distributed last-level cache through a dynamic address mapping rule; The distributed directory corresponds to the distributed last-level cache in a one-to-one manner, and is used to record the status of the cache line corresponding to the distributed last-level cache associated with the router; The router is used to receive and respond to a request from a local node or a remote node, and query whether the corresponding distributed directory is hit; Feedback is provided based on the query hit result and the status of the cached row in the distributed directory is updated.

2. The system according to claim 1, characterized in that The router comprises: an input module, an output module and a cache consistency module; The input module is used to receive a request from a local node or a remote node and transmit it to a cache consistency module; The cache consistency module is used to receive and respond to the request, obtain and feedback the response or monitor the request, and modify the state of the cache line in the corresponding distributed directory; The output module is used to output the response or monitoring request.

3. The system according to claim 2, characterized in that The cache consistency module is specifically used for: Receive requests from input modules; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, get the response feedback from the memory to the requesting node, and update the status of the cache line in the corresponding distributed directory; Updates the state of the cache line of the distributed directory in the corresponding node.

4. The system according to claim 2, characterized in that The input module includes an input queue, and is also used to merge multiple requests from the same destination node according to a merging rule, generate a single request and send it to the cache consistency module.

5. The system according to claim 3, characterized in that The router also includes: a routing calculation module, a distribution module and a cross switch module; The routing calculation module is used to calculate and obtain and feedback the routing path of the requesting node according to the data packet; the data packet includes a request, a listening request, a listening response and a response; The allocation module is used to determine the data packets to be output in each time period and obtain a determination result; The cross switch module is used to connect the output module and the input module, and to establish a connection or disconnect a specific input channel and an output channel according to the decision result and the routing path.

6. The system according to claim 5, characterized in that The routing calculation module is also used for: When the requesting node sends a data packet to multiple destination nodes, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

7. The system according to claim 2, characterized in that When the distributed last-level cache is full, the cache coherence module preferentially replaces cache lines that are not occupied by local nodes.

8. The system according to claim 1, characterized in that The dynamic address mapping rule is to determine the associated distributed last-level cache according to the high N bits of the memory address, where N is the number of binary bits of the distributed last-level cache; When the number of times the request is sent to the same node is greater than a preset threshold, the mapping relationship between the address block and the distributed last-level cache is adjusted.

9. The system according to claim 1, characterized in that The number of bits of each entry in the distributed directory is determined by the total number of processor cores and the number of processor cores per node.

10. The system according to claim 1, characterized in that The multiple processor cores in each node maintain the cache consistency within the node through the bus snooping protocol, and the nodes maintain the cross-node cache consistency through the on-chip network.

11. The system according to claim 1, characterized in that Each processor core corresponds to an independent first-level cache and second-level cache; the first-level cache is connected to the second-level cache, and the first-level caches and second-level caches of different processor cores are not interoperable.

12. A multi-core cache consistency method, characterized in that: The method is applied in a router, and the method comprises: Receive requests from local nodes or remote nodes; If the distributed directory of the local node is hit and the status meets the request requirements, a response is obtained from the corresponding distributed last-level cache and fed back to the requesting node; if the distributed directory of the local node is hit but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If the distributed directory of the local node does not hit, the request is sent to the remote node holding the cache line; if the distributed directory of the remote node hits and the status meets the request requirements, a response is obtained from the distributed last-level cache and fed back to the requesting node; if the distributed directory of the remote node hits but the status does not meet the request requirements, a listening request is sent to other remote nodes, and after receiving all the listening responses, the requesting node is fed back and the status of the cache line in the corresponding distributed directory is updated; If all nodes miss, a response is obtained from the memory to feed back the requesting node, and the status of the cache line in the corresponding distributed directory is updated.

13. The method according to claim 12, characterized in that Multiple requests to the same destination node are merged according to the merge rule to generate a single request.

14. The method according to claim 12, characterized in that The method further comprises: Calculate, obtain and feedback the routing path of the requesting node according to the data packet; the data packet includes a request, a listening request, a listening response and a response; Determine the data packets to be output in each time period and obtain the determination result; The output module and the input module are connected, and according to the decision result and the routing path, a specific input channel and an output channel are connected or disconnected.

15. The method according to claim 14, characterized in that The method further comprises: When the requesting node sends a data packet to multiple destination nodes, the routing path of the data packet is determined according to the routing algorithm and the number of hops from the destination node to the requesting node, so that the data packet passes through multiple destination nodes in sequence at one time.

16. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 12 to 15.

17. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 12-15.

18. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 12 to 15 is implemented.

Images