Cache maintenance system of heterogeneous computing system and electronic equipment

Abstract

The invention discloses a cache maintenance system of a heterogeneous computing system and electronic equipment, and relates to the technical field of data processing, a first directory controller manages the consistency state of a host end, and a second directory controller manages the consistency state of an equipment end. The consistency maintenance operation is firstly efficiently processed by the corresponding directory controller in the storage domain, cross-device frequent coordination communication is reduced, and when cross-domain data access occurs, the two directory controllers cooperate through an established communication coupling mechanism and execute global consistency maintenance as required. The method not only can adapt to a high-burst and high-parallel access mode of a device end, but also can avoid direct conflicts with a cache management mechanism of a host end, and realizes global consistency through distributed collaboration. The technical problem of performance bottleneck of cache consistency in the heterogeneous system is solved, and the technical effect that the overall data access efficiency of the heterogeneous system is improved while the correctness is maintained is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a cache maintenance system of a heterogeneous computing system and an electronic device. BACKGROUND

[0002] In related technologies, cache consistency protocols (such as bus-based monitoring protocols or centralized directory protocols) are mainly designed for application scenarios of homogeneous multi-core CPUs (Central Processing Unit), and have obvious limitations in the scenario of a heterogeneous computing system. Devices such as GPUs (Graphics Processing Unit) use independent cache and memory spaces, and the high-frequency and burst data access mode generated by a large number of parallel threads of the GPU is prone to conflict with the cache block granularity management mechanism of the CPU, triggering frequent cross-device consistency maintenance operations and causing performance bottlenecks.

[0003] Therefore, there is an urgent need for a cache consistency architecture suitable for a heterogeneous computing system to achieve cache management of different computing units. SUMMARY

[0004] The present application provides a cache maintenance system of a heterogeneous computing system and an electronic device to at least solve the technical problem of performance bottlenecks of cache consistency in related technologies, achieving the technical effect of improving the overall data access efficiency of the heterogeneous system while maintaining correctness.

[0005] The present application provides a cache maintenance system of a heterogeneous computing system, the heterogeneous computing system comprising a host end and a device end, the host end comprising a plurality of processor nodes, and the device end comprising at least one accelerator; the cache maintenance system comprising: a distributed directory controller comprising a first directory controller arranged at the host end and a second directory controller arranged at the device end; the first directory controller being communicatively coupled with the second directory controller; the first directory controller being configured to maintain a first directory table for managing the consistency state of the memory and cache of the host end; the second directory controller being configured to maintain a second directory table for managing the consistency state of the memory and cache of the device end; the cache maintenance system being configured to: in response to a data access request of a processor node of the host end to target data, performing global consistency maintenance operations between the host end and the device end according to the consistency state of the directory controller corresponding to the storage domain where the target data is located.

[0006] The present application also provides an electronic device comprising the cache maintenance system of the heterogeneous computing system as described above.

[0007] Through the application, since the first directory controller manages the consistency state of the host side, and the second directory controller manages the consistency state of the device side, the consistency maintenance operation is first efficiently processed by the corresponding directory controller in the storage domain, reducing frequent coordination communication across devices. When cross-domain data access occurs, the two directory controllers cooperate through the established communication coupling mechanism to perform global consistency maintenance on demand. Not only can it adapt to the high burst and high parallel access mode of the device side, but also can avoid direct conflict with the cache management mechanism of the host side, and achieve global consistency through distributed cooperation. The technical problem of the performance bottleneck of cache consistency in a heterogeneous system is solved, and the technical effect of improving the overall data access efficiency of the heterogeneous system while maintaining correctness is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A schematic diagram of a cache maintenance system of a heterogeneous computing system provided by an embodiment of the present application.

[0010] Figure 2 A partial data processing flowchart corresponding to a write request provided by an embodiment of the present application.

[0011] Figure 3 Another partial data processing flowchart corresponding to a write request provided by an embodiment of the present application.

[0012] Figure 4 A partial data processing flowchart corresponding to a read request provided by an embodiment of the present application.

[0013] Figure 5 Another partial data processing flowchart corresponding to a read request provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0015] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0016] In order to enable those skilled in the art to better understand the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0017] First, the following embodiments involve the following terms are explained, cache block corresponding to the four states are modified state, shared state, exclusive state and invalid state, wherein, the modified state represents that the data in the corresponding cache block has been changed and not written into the host memory or device memory, the exclusive state represents that the data in the cache block is consistent with the data in the host memory or device memory, and only one cache has a data copy; the shared state represents that the data in the cache block is consistent with the data in the host memory or device memory, and at least two caches have a data copy; the invalid state represents that the cache block is useless data.

[0018] In a first aspect, the present application provides a cache maintenance system of a heterogeneous computing system, the heterogeneous computing system including a host end and a device end, the host end including a plurality of processor nodes, and the device end including at least one accelerator; the heterogeneous computing system is a high-performance computing architecture integrating different types of computing units, which cooperates the host end (usually including a plurality of CPU processor nodes) good at logical control and general computing with the device end (integrating at least one accelerator such as GPU, FPGA (Field-Programmable Gate Array, i.e. field programmable gate array) and the like) good at parallel processing and special computing. This architecture can fully exert the strengths of different processing units to cope with the growing demand for computing power in artificial intelligence, scientific computing and the like. However, there are differences in hardware structure, memory model and access mode between CPU and accelerator. For example, GPU usually has independent large-scale parallel threads and high-bandwidth memory, and its data access has the characteristics of burst and high frequency, which is easy to conflict with the cache block-based management mechanism of CPU. These heterogeneities bring great challenges to maintaining the consistency of memory data in the whole system, and the traditional consistency protocol designed for homogeneous CPU is difficult to be directly applicable, which is easy to become a performance bottleneck.

[0019] As Figure 1The cache maintenance system provided in the application comprises: a distributed directory controller, comprising a first directory controller arranged at a host end and a second directory controller arranged at a device end; the first directory controller is communicatively coupled with the second directory controller; the first directory controller is configured to maintain a first directory table for managing the consistency state of the memory and cache at the host end; the second directory controller is configured to maintain a second directory table for managing the consistency state of the memory and cache at the device end; the cache maintenance system is configured to: in response to a data access request of a target data by a processor node at the host end, perform consistency state cooperative control with the directory controller corresponding to the storage domain where the target data is located, so as to perform global consistency maintenance operation between the host end and the device end. Figure 1 It is also shown in the application that the processor nodes communicate through an interconnection network.

[0020] The cache maintenance system of the embodiment is provided with a distributed directory controller structure. The structure comprises a first directory controller arranged at a host end and a second directory controller arranged at a device end. The two directory controllers are not independently operated, but are coupled together through a specific communication mechanism. The consistency management responsibility traditionally concentrated in a single location or a single device is explicitly divided according to physical and logical storage domains.

[0021] In the embodiment, the first directory controller is specially responsible for maintaining and managing the consistency state of the memory and cache at the host end, and the state information is recorded in the first directory table maintained by the first directory controller. Correspondingly, the second directory controller is specially responsible for maintaining and managing the consistency state of the memory and cache at the device end (i.e. the accelerator side), and the related information is recorded in the second directory table. This domain management mode makes the data access and consistency maintenance within the host end or the device end be efficiently completed under the management of the corresponding local directory controller, avoiding the communication overhead and potential bottleneck caused by transferring all requests to a central node.

[0022] When a processor node at the host end initiates a data access request for a target data, the cache maintenance system of the embodiment will first determine the storage domain where the target data currently locates. For example, the target data can be but is not limited to stored in the memory at the host end, or can reside in the memory at the device end. After determining the storage domain, the directory controller responsible for the storage domain is communicated with to start a consistency state cooperative control process. If the target data is in the host domain, the first directory controller dominates; if the target data is in the device domain, the second directory controller dominates and communicates with the first directory controller.

[0023] Through communication coupling between the first and second directory controllers, collaborative control of the target storage domain is extended to the entire heterogeneous computing system. The two directory controllers exchange necessary state information and, according to pre-defined consistency protocol rules, jointly determine and execute global consistency maintenance operations across the host and device ends. This process ensures that regardless of where the data is modified physically, all processor nodes and accelerators see a consistent view of the data, thus achieving global memory consistency in a distributed architecture.

[0024] like Figure 1 In one exemplary embodiment, at least one processor node includes: multiple processor cores; multiple caches, each corresponding to one of the multiple processor cores; and multiple cache controllers. Figure 1 (Not shown in the diagram) Corresponding to multiple caches, each is configured to manage the consistency state of its corresponding cache; the node consistency controller is configured to coordinate cache consistency operations between multiple cache controllers within a processor node and serves as the communication interface between different processor nodes and between a processor node and the first directory controller; the cache maintenance system is specifically configured as follows: in response to a data access request from the requesting processor to verify target data, it coordinates the consistency state with the directory controller corresponding to the storage domain where the target data resides, and the data access request includes the target physical address corresponding to the target data. Furthermore, Figure 1 The document also illustrates communication between the cache (specifically the cache controller) and the node consistency controller via a listening broadcast bus.

[0025] This embodiment details the internal structure of a processor node. A processor node contains multiple processor cores, each equipped with its own dedicated cache. These caches correspond one-to-one with each processor core, and each cache includes at least one cache block. This architecture allows multiple processor cores to work simultaneously, each with its own local copy of the data to accelerate access. To manage these caches, each cache is also associated with an independent cache controller. This cache controller is responsible for maintaining the consistency state of the cache blocks within its corresponding cache, such as determining whether data is in an exclusive or shared state, and modifying the state of the corresponding cache blocks.

[0026] Each processor node has an internal node consistency controller. This controller serves a dual role as both an internal coordination hub and an external communication interface. Internally, the node consistency controller coordinates the operations between all cache controllers within the node. When a processor core needs to access target data that may exist in the cache of another processor core within the same node, the node consistency controller intervenes, coordinating the cache consistency protocol internally, such as initiating internal listening or state transitions, without alerting the higher-level first directory controller.

[0027] Externally, this node consistency controller acts as a unified agent for the entire processor node's interaction with the external architecture. It manages direct communication between this node and other processor nodes, and also serves as the interface for communication between this node and the first directory controller. This design hides the complex multi-core cache consistency details within the node from the outside world, allowing external components to interact only with this single node consistency controller, thus simplifying the complexity of the global interconnect.

[0028] When a processor core within this node (referred to as the request processor core) initiates a data access request for the target data, the cache maintenance process in this embodiment is activated. The node consistency controller takes over this request and analyzes the current storage domain of the target data. The target data may be stored, but is not limited to, in the cache of this node or other nodes, or it may reside in the outermost host-side memory or device-side memory.

[0029] Based on the judgment results, the node consistency controller, representing the entire processor node, collaborates with the directory controller responsible for that storage domain. If the target data is believed to reside in the host-side storage domain, the node consistency controller communicates with the first directory controller; if it resides in the device-side storage domain, it communicates with the second directory controller. Through this hierarchical, domain-specific collaboration mechanism, a local access request from a single processor core can be routed to the corresponding global consistency management component, thereby triggering consistency maintenance operations that may involve both the host and device sides across the entire heterogeneous system.

[0030] In one exemplary embodiment, when the target data is located in the local cache corresponding to the request processor core, the local cache controller corresponding to the local cache is configured to: in response to the request processor core's data access request for the target data, perform a corresponding local consistency maintenance operation according to the type of the data access request; wherein, the type of data access request includes read requests and write requests.

[0031] This embodiment describes a data access scenario where the target data requested by the processor core happens to be located in its own local cache. In this case, the processing of the data access request is confined to the most micro level of the processor node, without the intervention of the node consistency controller or the upper-level directory controller.

[0032] At this point, the local cache controller responsible for managing the local cache will directly respond to the data access request. In this embodiment, the local cache controller performs operations based on the specific type of the access request. For example, if the requesting processor core initiates only a read request, the local cache controller may only need to check and confirm that the consistency status of the data copy in the target cache block of the local cache is valid, and then it can provide the data to the processor core. This operation is completed entirely locally and is relatively fast.

[0033] If the requesting processor core initiates a write request, the local cache controller can, but is not limited to, performing corresponding local consistency maintenance operations based on the current state of the cache block. For example, if the data is currently in an exclusive state, the local cache controller can directly allow the processor core to modify it. If the data is in a shared state, the local cache controller may first need to initiate an operation to invalidate other cache replicas within the node through the node consistency controller, or upgrade the state of the cache block to exclusive, to ensure that the modification does not conflict with other replicas. This local operation minimizes the risk of escalating a simple data update into a global event, which is fundamental to ensuring the efficient operation of the entire system.

[0034] The following is combined with Figure 2 The first branch on the left explains another case where the data access request is a read request.

[0035] In one exemplary embodiment, when the data access request type is a read request, the local cache controller is specifically configured to read the target data from the target cache block corresponding to the target physical address and transfer the target data to the request processor core.

[0036] This embodiment describes the detailed processing flow when a request processor core initiates a read request and the target data is located in its local cache. The local cache controller reads the target data directly from the target cache block. The data reading process occurs over a very short internal distance, resulting in low latency. Subsequently, the local cache controller is responsible for transmitting the read target data back to the request processor core for subsequent operations.

[0037] Regarding cache block state management, the local cache controller typically maintains the original state of the cache block after completing data reading and transfer. For example, if the cache block was previously in a modified state, indicating that the processor core has the latest and unique copy of the data, it will remain in the modified state after the read operation; if it was previously in an exclusive state, it will also remain in the exclusive state after the read operation; if it was in a shared state, it will also remain in the shared state after the read operation. This state remains unchanged because a simple read operation does not change the value of the target data, nor does it introduce inconsistencies with other copies.

[0038] Therefore, for locally hit read requests, the processing can, but is not limited to, be completed entirely within the processor core's local cache subsystem. The entire process requires no coordination with other cache controllers within the node, nor does it need to alert the node's consistency controller or the parent directory controller. This approach minimizes access latency and avoids unnecessary overhead from global consistency protocol interactions.

[0039] The following is combined with Figure 4The first branch on the left explains another case where the data access request is a write request.

[0040] In an exemplary embodiment, when the data access request type is a write request, the local cache controller is specifically configured to: write the target data into the target cache block corresponding to the target physical address, update the consistency state of the target cache block to the modified state, and send a state update notification to the corresponding node consistency controller; the state update notification carries at least the physical address of the target data and the identifier of the request processor core; the node consistency controller is configured to: forward the state update notification to the first directory controller; the first directory controller is configured to: in response to the state update notification, perform consistency state coordination control according to the original state and the modified state to be updated of the target cache block in the first directory table.

[0041] This embodiment describes the processing flow when a request processor core initiates a write request and the target data resides in its local cache. Unlike local read operations, write operations modify data content, thus requiring the consistency of all potential data copies in the system. This process stems from modifications made to local data by the local cache controller.

[0042] In this embodiment, upon receiving a write request, the local cache controller first writes the new target data to the target cache block. Secondly, regardless of the previous consistency state of the target cache block, the local cache controller updates its state to modified. This modified state indicates that the data in the target cache block has been changed and is the latest version, and its content has not yet been written back to the host memory or device memory. After completing the local update, the local cache controller needs to send a status update notification to the next higher level, namely the node consistency controller within this processor node. This notification carries at least the target physical address of the modified data and the identifier of the requesting processor core that initiated the modification, so that the higher-level component can accurately locate and track this change.

[0043] After receiving a state update notification from its subordinate cache controller, the node consistency controller forwards this information to the superior organization responsible for global consistency management, namely the first directory controller located on the host. The node consistency controller acts as a bridge, transforming a local state change within a processor node into an event requiring global coordination. Upon receiving the state update notification, the first directory controller initiates consistency state coordination operations. In this embodiment, the first directory controller queries its maintained first directory table to obtain the original state of the target cache block corresponding to the target data before the current write operation. Then, combined with the target state to be updated (i.e., the modified state), it determines and executes necessary coordination operations according to preset consistency protocol rules. For example, it can, but is not limited to, check the first directory table. If it finds that the data has a shared copy in the cache of other processor nodes or devices, the first directory controller needs to send invalidation instructions to those components holding copies through paths such as the node consistency controller, to ensure that the cache block with the modified state becomes the only valid copy of the data.

[0044] In one exemplary embodiment, the first directory controller is specifically configured to: query the first directory table in response to a state update notification; update the state of the target cache block in the first directory table to the modified state when the original state of the target cache block in the first directory table is modified or exclusive; and perform a global invalidation operation when the original state of the target cache block in the first directory table is shared. The global invalidation operation includes: updating the state of the target cache block in the first directory table to the modified state, and sending an invalidation instruction to the second directory controller and / or the node consistency controller corresponding to the processor node holding other copies, so that the state of the corresponding cache copy in the device and other processor nodes is updated to the invalid state.

[0045] This embodiment specifically describes the judgment and execution performed by the first directory controller after receiving a status update notification from a processor node. The first directory controller queries its maintained first directory table, which records the consistency status of all cache blocks on the host side. The purpose of the query is to determine the global replica distribution of the target data before the current write request occurred.

[0046] Based on the retrieved original state, when the first directory table shows that the target cache block's original state is already modified or exclusive, this indicates that the latest copy of the data is exclusively owned globally by the cache block that initiated the write request. Therefore, this write operation will not conflict with other copies. The first directory controller then performs a lightweight operation, simply updating the cache block's state in the directory table to modified, thus completing this consistency maintenance. This path requires no additional communication with other components, resulting in low processing latency.

[0047] When the first directory table shows the original state of the target cache block as shared, it means that at least two caches (which may be located on the same node or different nodes) simultaneously hold valid copies of the data. To ensure that the modified data becomes the unique authoritative copy, the first directory controller must perform a global invalidation operation. In this embodiment, this operation first updates the directory table state to the modified state to record the new ownership. The first directory controller needs to actively issue invalidation instructions, which are sent to the node consistency controllers of other processor nodes that may hold copies, as well as the second directory controller responsible for managing device-side caches. The purpose of these instructions is to command all other copies to change their state to invalid, thereby physically erasing the old data.

[0048] This approach avoids unnecessary invalidation broadcasts for data that is already exclusively owned or modified, minimizing consistency maintenance overhead. A global invalidation operation involving multi-component collaboration is triggered only when a true shared state is detected. This on-demand triggering method reduces invalidation storms that may be caused by large-scale caching on devices such as GPUs in heterogeneous environments, optimizes the utilization of overall communication bandwidth, and ensures that the overhead of consistency maintenance matches the actual degree of data sharing, rather than always executing the most conservative and costly operation.

[0049] In one exemplary embodiment, when the target data is located in another cache within the same processor node as the request processor core that initiated the data access request; the node consistency controller is configured to: coordinate data access and transmission between the request cache controller corresponding to the request processor core and the target cache controller storing the target data, and interact with the first directory controller; the first directory controller is configured to: update the consistency status of the corresponding cache block in the first directory table according to the message sent by the node consistency controller regarding data sharing and state changes within the processor node.

[0050] This embodiment describes a scenario where the target data required by a requesting processor core is located within the same processor node but belongs to the cache of another processor core. In this case, the request cannot be completed within the local closed loop of a single processor core and requires coordination within the node. In this scenario, this embodiment uses the node consistency controller within the processor node as the coordinator. The node consistency controller establishes a connection between the request cache controller corresponding to the requesting processor core and the target cache controller holding the target data, and coordinates the data access and transmission process.

[0051] The node consistency controller can, but is not limited to, transmit control messages and data between the request cache controller and the target cache controller via the node's internal bus or on-chip network. For example, for a read request, the node consistency controller can coordinate the target cache controller to provide a data copy to the request cache controller; for a write request, it may need to coordinate the target cache controller to degrade or invalidate the state of its held copy. The entire process is encapsulated within the processor node, and the node consistency controller ensures that the multiple cache controllers within it can interact correctly according to the consistency protocol.

[0052] During or after coordinating internal data interactions, the node consistency controller also needs to interact with the higher-level global manager, namely the first directory controller. The node consistency controller sends key messages such as data sharing and state changes that have occurred within the node to the first directory controller. Upon receiving these messages, the first directory controller updates the consistency state of the corresponding cache block in its maintained first directory table according to the message content, so as to reflect the latest ownership and sharing status of the data globally.

[0053] This embodiment encapsulates the complex multi-core cache interactions within a processor node through a node consistency controller, presenting them to the outside as a single logically unified interface. This simplifies the management complexity of the first directory controller, eliminating the need to be aware of the specific inter-core operations within the node; it only needs to handle the aggregated state updates from the node consistency controller. Simultaneously, resource sharing and state coordination within the node can be completed quickly, with latency far lower than cross-node communication, achieving efficient data locality utilization.

[0054] The following is combined with Figure 2 The second branch on the left explains another case where the data access request is a read request.

[0055] In an exemplary embodiment, when the data access request is a read request; the node consistency controller is configured to: receive a read request from a request processor core, the read request information including the physical address of the target data; query the cache distribution within the processor node to determine the target cache block storing the target data and its corresponding target cache controller; send a data read instruction to the target cache controller; receive the target data provided by the target cache controller and forward the target data to the request cache controller corresponding to the request processor core; forward a status update notification to the first directory controller; the target cache controller is configured to: if the current consistency state of the target cache block is exclusive or shared, then after providing the target data to the node consistency controller, update or maintain the consistency state of the target cache block to shared; if the current consistency state of the target cache block is modified, then after providing the target data to the node consistency controller, update the consistency state of the target cache block to shared; the request cache controller corresponding to the request processor core is configured to: after receiving the target data, set the consistency state of the corresponding request cache block to shared; the first directory controller is configured to: in response to the status update notification, update the status of the target cache block and the request cache block in the first directory table to shared, and perform consistency state collaborative control.

[0056] This embodiment describes the processing flow when a read request initiated by a requesting processor core targets data located in other caches within the same node. In this case, the node consistency controller within the processor node coordinates this internal data transfer and state update. The node consistency controller first receives the read request from the requesting processor core, which contains the physical address of the target data. The node consistency controller queries the cache distribution information within its processor node to determine which specific cache block holds the target data, i.e., it locates the target cache block and its corresponding target cache controller.

[0057] The node consistency controller sends a data read instruction to the target cache controller. Upon receiving the instruction, the target cache controller's behavior depends on the current consistency state of the target cache block it manages. In this embodiment, if the target cache block's current state is exclusive or shared, the target cache controller, after providing the target data to the requester through the node consistency controller, updates its own cache block's state to shared, reflecting that an additional copy of the data exists within the node. If the target cache block's current state is modified, indicating that the data is up-to-date and unique, the target cache controller, while providing the data, also needs to provide a data write-back instruction. This may, but is not limited to, instructing the node consistency controller to schedule a subsequent write-back of the data to main memory for persistence. Simultaneously, the target cache controller itself updates its state to shared.

[0058] After receiving the target data from the node consistency controller, the request cache controller corresponding to the request processor core loads the data into the request cache block it manages and sets the consistency state of the cache block to a shared state, indicating that it now holds a valid copy of the data. Once the data flow within the entire node is complete, the node consistency controller needs to forward this internal sharing event to the first directory controller in the form of a state update notification to synchronize the global state.

[0059] Upon receiving a state update notification, the first directory controller updates its first directory table, recording the states of both the target cache block and the requested cache block involved in the read operation as shared states, and may update the data replica distribution record. The first directory controller may also perform further consistency state coordination operations; for example, if the read request might involve potentially old replicas on the device, the first directory controller can interact with the second directory controller to ensure the global view is correct.

[0060] This embodiment standardizes and encapsulates the data sharing and state transition processes within a node. All internal operations are uniformly scheduled by the node consistency controller, which only reports the final state change results externally. This reduces the processing burden on the first directory controller, eliminating its need to intervene in complex internal node transactions. Simultaneously, the data transmission path within the node is short, with lower latency than cross-node or cross-device access, fully leveraging the principle of data locality. This allows processor cores to quickly access data cached by other cores within the same node, improving the efficiency of multi-core collaborative computing. Furthermore, this embodiment clearly distinguishes the processing under different original states, especially the write-back instruction accompanying the modification of state data, ensuring that efficient sharing does not compromise data consistency and durability requirements.

[0061] In one exemplary embodiment, the target cache controller is further configured to: provide a data write-back instruction when providing target data to the node consistency controller; the node consistency controller is further configured to: forward the data write-back instruction from the target cache controller to the first directory controller; the first directory controller is further configured to: in response to the data write-back instruction, write the target data back to its host memory or device memory according to the physical address of the target data.

[0062] This embodiment describes the specific process for a data write-back operation when an internal node read operation involves modifying state data. When the target cache controller provides a data write-back indication because its cache block state is modified, the node consistency controller is responsible not only for coordinating data transmission but also for passing this crucial indication upwards. In this embodiment, the node consistency controller forwards the received data write-back indication, along with relevant context information, to the first directory controller.

[0063] Upon receiving a data write-back instruction, the first directory controller initiates the data write-back operation to host memory. In this embodiment, the first directory controller determines the final storage domain to be written back based on the physical address of the target data. This determination can be made, but is not limited to, by checking the address mapping table or address range register. If the physical address of the target data belongs to the host memory address space, the first directory controller schedules the data to be written back to host memory. If the physical address of the target data belongs to the device memory address space, the first directory controller needs to coordinate with the second directory controller, which is responsible for ultimately writing the data back to device memory. This process ensures that the modified, unique, and up-to-date copy of the data is persisted from its temporary storage cache to the correct global main memory location.

[0064] This embodiment clarifies the triggering and execution path of the data write-back operation, connecting internal node events with global storage management. The first directory controller determines the write-back target based on the physical address, achieving unified management of heterogeneous memory space. This avoids the need for the node consistency controller or cache controller to be aware of complex global address mappings, simplifying the design.

[0065] The following is combined with Figure 4 The second branch on the left explains another case where the data access request is a write request.

[0066] In an exemplary embodiment, when the data access request is a write request; the node consistency controller is configured to: receive a write request from the request processor core, query the cache distribution within its own processor node, determine the target cache block storing the target data and its corresponding target cache controller, and send a write operation notification to the target cache controller; after receiving shared state existence information, forward the write request and shared state identifier to the first directory controller; after receiving global invalidation confirmation information, instruct the target cache controller to update the state of the target cache block to invalid state, and transmit the target data to the request cache controller corresponding to the request processor core; the target cache controller is configured to: respond to the write operation notification, if the consistency state of the target cache block is exclusive or modified, update the state of the target cache block to invalid state, and transmit the target data to the request cache controller corresponding to the request processor core; if the consistency state of the target cache block is shared state, feed back shared state existence information to the node consistency controller, and upon receiving... After receiving the global invalidation confirmation information, the state of the target cache block is updated to invalid. The first directory controller is configured to: respond to a write request and perform a global invalidation operation, which includes: updating the state of the requested cache block in the first directory table to modified, and sending an invalidation instruction to the second directory controller and / or the node consistency controller corresponding to the processor node holding other copies, so that the state of the corresponding cache copy in the device and other processor nodes is updated to invalid; receiving and sending global invalidation confirmation information to the target cache controller and the node consistency controller; updating the consistency state of the corresponding cache block in the first directory table according to the message sent by the node consistency controller regarding data sharing and state changes within the processor node; the request cache controller corresponding to the request processor core is configured to: allocate a cache block for the target data after receiving the target data, set the state of the allocated cache block to modified, and feed back data reception completion and state update information to the node consistency controller.

[0067] This embodiment describes the processing flow when a write request initiated by the request processor core targets data located in other caches within the same node. In this case, the node consistency controller first receives the write request, queries the internal cache distribution to locate the target cache block holding the target data and its target cache controller, and sends a write operation notification to the target cache controller, informing it that the data is about to be modified.

[0068] Upon receiving a notification, if the target cache block is in an exclusive or modified state, it indicates that there may be no other valid copies of the data outside the node. The target cache controller can directly update its own state to invalid and transmit the data content to the requester through the node's consistency controller; this entire process can be completed within the node. If the target cache block is in a shared state, it means that other shared copies exist. In this case, the target cache controller will not immediately perform any operations but will instead report the existence of a shared state to the node's consistency controller.

[0069] Upon receiving information indicating the existence of shared state, the node consistency controller forwards the write request, along with this shared state identifier, to the first directory controller, thus elevating an intra-node write operation to an event requiring global coordination. Upon receiving this message, the first directory controller performs a global invalidation operation, which may include, but is not limited to, sending invalidation instructions to the second directory controller and other processor nodes that may hold copies. Only after the first directory controller has collected all necessary invalidation acknowledgments and sent back global invalidation confirmation information will the node consistency controller instruct the target cache controller to update its state to invalid and transmit the data.

[0070] Upon receiving the target data, the cache controller corresponding to the request processor core needs to allocate a new cache block for the data, set the status of the cache block to modified, and report the completion information to the node consistency controller. The node consistency controller synchronizes the complete information of this data ownership change within the node to the first directory controller. Based on this, the first directory controller updates the directory table, updating the data owner to the requester and the status to modified, thereby achieving eventual consistency of the global state.

[0071] This embodiment manages the scope of write requests through the interaction between the node consistency controller and the target cache controller. For data that is already exclusively owned or modified, write operations can be completely confined to within the node, avoiding expensive and unnecessary global coordination and optimizing write performance in common scenarios. A global invalidation process, led by the first directory controller and involving multiple nodes and devices, is triggered only when a shared state is detected. This on-demand triggering coordination mechanism binds the overhead of global consistency maintenance to the actual degree of data sharing, avoiding communication storms and performance degradation that may occur in heterogeneous multi-core environments due to blindly broadcasting invalidation instructions.

[0072] In one exemplary embodiment, when the target data is located in the cache of a processor node that is different from the request processor node where the request processor core that initiated the data access request resides; the first directory controller is configured to: in response to a data access request forwarded by the request node consistency controller of the request node, determine at least one target node where the target data resides based on a first directory table, and coordinate the request node and the node consistency controllers corresponding to the at least one target node to perform cross-node data consistency maintenance operations; the request node is the processor node where the processor core that initiated the data access request resides, and the target node is a processor node that is different from the request node where the target data resides.

[0073] This embodiment describes a scenario where the target data required by the requesting processor core is located in the cache of another processor node (i.e., the target node). In this case, the data access request is beyond the management scope of the local node and must be coordinated by the first directory controller responsible for the global state. The node consistency controller within the requesting node forwards this access request to the first directory controller.

[0074] Upon receiving a request, the first directory controller queries its maintained first directory table. Based on the information recorded in the table, the first directory controller can determine one or more target nodes currently holding a valid copy of the target data. Subsequently, the first directory controller coordinates between the requesting node and the node consistency controllers corresponding to these target nodes. In this embodiment, this coordination may, but is not limited to, the first directory controller sending specific instructions to the relevant nodes. For example, for a read request, the first directory controller instructs the target node to provide a copy of the data; for a write request, the first directory controller instructs the target node to invalidate its cached copy.

[0075] Through centralized coordination by the first directory controller, the complex direct point-to-point communication that might have been necessary between the requesting node and the target node is transformed into a standardized interaction with the central directory controller. The first directory controller ensures that the requesting node can obtain data from the correct target node or complete state changes, and in the process updates the first directory table to reflect the latest data ownership and sharing state, thereby executing and completing a cross-node data consistency maintenance operation.

[0076] The following is combined with Figure 2 The first branch on the right explains another case where the data access request is a read request.

[0077] In an exemplary embodiment, when the data access request is a read request; the request node consistency controller is configured to: receive a read request sent from the request processor core of the request node, and forward the read request to the first directory controller; provide the received target data to the processor core that initiated the read request; the first directory controller is configured to: respond to the read request, query the first directory table, confirm the target cache block where the target data is located and its corresponding target node, and send a data transmission instruction to the target node consistency controller corresponding to the target node; determine the corresponding data migration path and state update strategy according to the consistency status in the target cache block where the target data is located recorded in the first directory table; and update the first directory controller after the read request is completed. The directory table records newly added shared replica nodes of the target data by the requesting node; in response to the data write-back instruction, the target data is written back to its host memory or device memory according to the physical address of the target data; the target node consistency controller of the target node is configured to: read the target data from the target cache block in response to the data transfer instruction; if the target data is in an exclusive or shared state in the target cache, the target cache state is updated or maintained as a shared state through the target cache controller after the data is provided; if it is in a modified state, the target cache state is updated to a shared state through the target cache controller at the same time as the data is provided, and a data write-back instruction is sent to the first directory controller to send the target data to the requesting node.

[0078] This embodiment describes the process when the data that the requesting processor core needs to read is located in the cache of another processor node. In this case, the consistency controller of the requesting node receives and forwards the read request to the first directory controller. The first directory controller queries the first directory table to locate the target node holding the data and issues a data transfer instruction to the consistency controller of that target node. The entire data acquisition path is planned uniformly by the first directory controller, avoiding the complex direct discovery and negotiation process between the requesting node and the target node.

[0079] Upon receiving an instruction, the target node's consistency controller is responsible for reading data from the target cache block within its node. In this embodiment, the specific actions of the data provider are related to the original state of its cache block. If the target cache block is in an exclusive or shared state, the target node, after providing data, updates or maintains its own state to a shared state through its cache controller to reflect the addition of a remote shared copy. If the target cache block is in a modified state, the target node, while providing data, not only downgrades its own state to a shared state but also sends a data write-back instruction to the first directory controller. This may, but is not limited to, triggering subsequent operations to write the data back to main memory.

[0080] After obtaining the target data, the requesting node consistency controller delivers the data to the processor core that initiated the request and sets its cache state to shared. Upon monitoring the completion of the entire transfer, the first directory controller updates its first directory table, recording the requesting node as a new shared replica node for that data. If a write-back instruction was previously received, the first directory controller also determines and executes the operation of writing the data back to host memory or device memory based on the physical address of the target data to ensure data persistence.

[0081] This directory-based centralized scheduling allows the first directory controller, with its global view, to select the optimal data source for cross-node read requests and coordinate efficient data migration paths. The design of target nodes responding differently based on their own state ensures that while providing data, their local state can be correctly transformed and kept synchronized with the global directory state.

[0082] The following is combined with Figure 4 The first branch on the right explains another case where the data access request is a write request.

[0083] In an exemplary embodiment, when the data access request is a write request; the requesting node consistency controller is configured to: receive and forward the write request to the first directory controller; receive the target data migrated from the target node, allocate a cache block for it through the corresponding cache controller and set it to the modified state to complete the write operation; the first directory controller is configured to: in response to the write request, query the first directory table to determine the consistency state of the target node holding the target data and the corresponding target cache block; if the target data is in an exclusive state or a modified state in the target cache block, send a data migration and state invalidation instruction to the target node consistency controller corresponding to the target node; if it is in a shared state, send an invalidation instruction to the node holding the target data and / or the second directory controller; and update the state of the target cache block in the first directory table to the modified state; the target node consistency controller is configured to: in response to the data migration and state invalidation instruction or invalidation instruction from the first directory controller, update the state of the target cache block to the invalid state through the corresponding target cache controller; if a data migration instruction is received, send the target data to the requesting node.

[0084] This embodiment describes the processing flow when a request processor core needs to initiate a write request for data cached on other nodes. In this cross-node write scenario, the request node's consistency controller acts as a local agent, receiving the write request and forwarding it to the first directory controller, elevating a local event to the global coordination level.

[0085] Upon receiving a write request, the first directory controller queries the first directory table to understand the global distribution of the target data, including which target nodes it resides on and its specific consistency status in the caches of these nodes. Based on the query results, the first directory controller executes differentiated instruction strategies. In this embodiment, if the target data is in an exclusive or modified state in the target node's cache, it indicates that there is likely only one valid copy of the data globally. In this case, the first directory controller sends a data migration and state invalidation instruction to the target node, aiming to transfer ownership of the data to the requesting node and clear the original copy. If the target data is in a shared state, it indicates that multiple copies exist. The first directory controller needs to broadcast an invalidation instruction to all nodes holding the data, as well as the second directory controller that may manage device-side copies, to clear all old copies and remove obstacles for the requesting node's exclusive write operation. In either case, the first directory controller synchronously updates its directory table, marking the data status as modified and recording the owner as the requesting node.

[0086] Upon receiving instructions from the first directory controller, the target node's consistency controller is responsible for executing specific operations within its own node. For data migration and state invalidation instructions, the target node's consistency controller instructs its corresponding cache controller to update the state of the target cache block to invalid and send the data content back to the requesting node. For simple invalidation instructions, only the state invalidation operation is performed, ensuring that once the first directory controller makes a decision, the relevant nodes can be reliably and systematically driven to execute.

[0087] After receiving the data migrated from the target node, the node consistency controller of the requesting node allocates a new cache block locally for the data, and sets the state of the cache block to the modified state through the corresponding cache controller, thus completing the write operation and enabling the request processor core to obtain exclusive ownership of the data.

[0088] In this embodiment, the first directory controller, based on its global information, can make optimal consistency decisions for cross-node write requests and select the most efficient instruction set according to the differences in the original state of the data (exclusive / modified and shared), thus avoiding invalid broadcasts or redundant operations.

[0089] like Figure 1In one exemplary embodiment, the device further includes: a device-side cache; a device-side cache controller configured to manage read and write operations and cache block status of the device-side cache; when the target data is located in the device-side cache; a first directory controller configured to: forward data access requests to a second directory controller to interact with the second directory controller and collaboratively perform data consistency maintenance operations for the device-side cache; the second directory controller configured to: in response to the data access requests forwarded by the first directory controller, query a second directory table and interact with the device-side cache controller to manage and maintain the consistency status of data in the device-side cache.

[0090] This embodiment describes a consistency maintenance process collaboratively managed by a first directory controller and a second directory controller when the target data resides in the device-side cache. The device internally has a device-side cache and a dedicated device-side cache controller, the latter responsible for executing specific cache read / write operations and managing its cache block state. When a request from the host involves device-side cached data, the first directory controller does not directly intervene in the device-side internal operations, but instead interacts with its peer, the second directory controller.

[0091] In this embodiment, the first directory controller forwards data access instructions or status maintenance requests to the second directory controller when needed. After receiving the instruction, the second directory controller queries its maintained second directory table to understand the distribution and status of data in the device-side cache; at the same time, it interacts with the device-side cache controller to drive the latter to perform specific data access, migration, or status update operations, such as setting the cache block status to invalid or shared.

[0092] Through this layered collaboration mechanism, this embodiment encapsulates the complex internal cache management details on the device side within the scope of the second directory controller. The first directory controller only needs to focus on cross-domain protocol collaboration and does not need to know the specific implementation of the device-side cache. This design enables the coherence protocol designed for the CPU to be extended to accelerator devices with different memory architectures, such as GPUs and FPGAs, through a standardized directory controller interface.

[0093] The following is combined with Figure 3 The first branch on the left explains another case where the data access request is a read request.

[0094] In an exemplary embodiment, when the data access request is a read request; the first directory controller is configured to: in response to the read request forwarded by the consistency controller of the request node corresponding to the request node processor core that initiated the read request, query the first directory table, and after confirming that there is no valid copy of the target data on the host side, send a data acquisition instruction to the second directory controller; receive and respond to the status information and target data from the second directory controller, update the first directory table, and forward the target data to the consistency controller of the request node that initiated the request; the second directory controller is configured to: in response to the data acquisition instruction, query the second directory table to determine the consistency status of the target data in the device cache; according to the consistency status, instruct the device cache controller to perform the corresponding data preparation and state transition operations; receive the target data provided by the device cache controller and forward it to the first directory controller; the device cache controller is configured to: in response to the instruction of the second directory controller, access the device cache to obtain the target data; and update its status in the device cache before or after providing the data according to the current consistency status of the target data.

[0095] This embodiment describes the read request processing flow when the data required by the requesting processor core exists only in the device-side cache. In this case, after receiving the read request forwarded from the consistency controller of the requesting node, the first directory controller queries its maintained first directory table. When it confirms that none of the processor nodes on the host side have a valid cached copy of the target data, the first directory controller routes the request to the device side and sends a data retrieval instruction to the second directory controller, thereby initiating a collaborative data retrieval process across heterogeneous domains.

[0096] Upon receiving the instruction, the second directory controller queries its own maintained second directory table to determine the specific consistency status of the target data in the device-side cache. Based on this status, the second directory controller issues an instruction to the device-side cache controller. In this embodiment, when the data is in an exclusive or shared state, the device-side cache controller can directly provide the data and update the cache status accordingly; if the data is in a modified state, it needs to perform an operation to write the data back to its own memory (which can be device memory or host memory) before providing the data to ensure data persistence, and then downgrade its own cache status to a shared state.

[0097] Under the instruction of the second directory controller, the device-side cache controller performs specific access to the device-side cache, retrieves the target data, and completes the state transition according to the rules. Subsequently, the second directory controller forwards the retrieved data along with necessary state information back to the first directory controller. Upon receiving the data, the first directory controller updates its first directory table, recording that the data now also has a shared copy on the host side, and finally forwards the data to the node consistency controller that initiated the request, which then completes the cache loading and state setting of the data on the host side.

[0098] This embodiment achieves transparent access from the host to the device's cached data through the interaction between the first directory controller and the second directory controller. The first directory controller does not need to understand the complex internal architecture of the device; it only needs to act as a central hub for request routing and state synchronization. The second directory controller encapsulates the heterogeneity of the device, unifying different cache states into a standard data preparation process.

[0099] In one exemplary embodiment, the device-side cache controller is specifically configured to: in response to an instruction from the second directory controller, access the device-side cache to obtain target data; if the current consistency state is modified, write the target data back to its memory and then update the state of the device-side cache to a shared state; if it is exclusive, update the state of the device-side cache to a shared state and then provide the target data; if it is shared, maintain the shared state and provide the target data.

[0100] In this embodiment, when the target data's current consistency state in the device-side cache is modified, the device-side cache controller writes the target data back to its associated memory. This associated memory can be, but is not limited to, the device's own memory, or it can be the host's memory, specifically determined by the data's physical address or memory ownership information. After completing the data write-back and ensuring the latest version is persisted, the device-side cache controller updates its own cache block's state to a shared state to reflect that the data will soon be shared by the host. Then, the target data is provided to the second directory controller.

[0101] If the target data is currently in an exclusive state, this indicates that the device holds the only valid copy of the data, but the copy is clean (consistent with memory). The device-side cache controller does not need to perform a write-back operation at this time. It updates the state of its own cache block from exclusive to shared to indicate that a new host-side sharer will be added, and then the target data can be provided to the second directory controller.

[0102] If the target data is already in a shared state, this indicates that multiple copies of the data may already exist in the device-side cache (e.g., shared by multiple computing units on the device). In this case, the device-side cache controller does not need to change the state of its cache blocks; it can simply maintain the shared state and provide the target data to the second directory controller.

[0103] The following is combined with Figure 5 This section explains another scenario where the data access request is a write request.

[0104] In one exemplary embodiment, when the data access request is a write request; the first directory controller is configured to: in response to the write request forwarded by the consistency controller of the request node corresponding to the request node processor core that initiated the write request, query the first directory table, and after confirming that there are no other valid copies of the target data on the host side, send an invalidation and data migration instruction to the second directory controller; receive and respond to the target data from the second directory controller, forward it to the consistency controller of the request node that initiated the request, and update the status of the target data in the first directory table to the modified status; the second directory controller is configured to: in response to the invalidation and data migration instruction, instruct the device-side cache controller to store the target data The target cache block status is updated to invalid and the target data is obtained. After receiving the target data sent from the device-side cache controller, the target data is forwarded to the first directory controller. The device-side cache controller is configured to: read the target data from the device-side cache in response to the instruction of the second directory controller, update the device-side cache block status corresponding to the target data to invalid, and then provide the data to the second directory controller. The request node consistency controller of the request node that initiated the request is configured to: receive the target data sent from the first directory controller, and store the target data into the request cache block and set it to modified status through the corresponding request cache controller to complete the write operation.

[0105] This embodiment describes the processing flow when the request processor core needs to initiate a write request for data that only exists in the device's cache. In this scenario, after receiving the write request forwarded from the request node's consistency controller, the first directory controller first queries its first directory table. When it confirms that there are no other valid cached copies of the target data on the host, the first directory controller determines that the device's cache holds the only or most recent copy of the data, and then sends an invalidation and data migration instruction to the second directory controller. This instruction explicitly requires the device to relinquish ownership of the data and invalidate its local copy.

[0106] Upon receiving an instruction from the first directory controller, the second directory controller executes specific operations within its managed device-side domain. This instruction instructs the device-side cache controller to update the status of the device-side cache block storing the target data to invalid, thus relinquishing ownership; and to read the content of the target data from the device-side cache. In response, the device-side cache controller first reads the data, then marks the status of the cache block it manages as invalid, and finally submits the data to the second directory controller. The second directory controller, after collecting the data, is responsible for forwarding it back to the first directory controller.

[0107] After receiving the target data from the second directory controller, the first directory controller forwards the received target data to the consistency controller of the node that initially initiated the write request. At the same time, in the first directory table it maintains, it updates the status of the target data to modified status and records the data owner as the processor node that initiated the request, thereby completing the formal transfer of ownership in the global directory.

[0108] After receiving the target data from the first directory controller, the consistency controller of the node initiating the request performs the final step of the write operation. Through the request cache controller it manages, it allocates a cache block locally for this migrated data, stores the data, and sets the consistency state of the cache block to the modified state. At this point, the request processor core obtains exclusive modification rights to the target data, and the entire cross-device write operation is complete. This embodiment achieves a smooth and reliable transfer of data ownership from the device side to the host side through the interaction between the first directory controller and the second directory controller.

[0109] like Figure 1 In one exemplary embodiment, the host side further includes: host memory, and a host memory controller configured to manage read and write access to the host memory; the device side further includes: device memory, and a device memory controller configured to manage read and write access to the device memory; when the target data is located in the host memory or device memory; the first directory controller is configured to: respond to a request node consistency controller that receives and forwards a data access request, query the first directory table, and after confirming that neither the cache on the host side nor the cache on the device side has a valid copy of the target data, determine the storage domain to which the target data belongs based on the physical address of the target data, and directly or indirectly obtain the target data stored in the corresponding memory controller by sending a data request to the second directory controller, so as to maintain data consistency after the target data is loaded into the request cache block.

[0110] This embodiment describes a scenario where the target data exists only in the host memory or device memory and not in any cache. In this case, after querying its first directory table, the first directory controller confirms that there are no valid copies of the target data in the host-side caches and device-side caches, and then determines that this access needs to retrieve the data from the main memory.

[0111] Based on the physical address of the target data, the first directory controller can determine its storage domain. If the target data is located in the host memory address space, the first directory controller can directly interact with the host memory controller to read the data from the host memory. If the target data is located in the device memory address space, the first directory controller needs to send a data request to the second directory controller, which then interacts with the device memory controller to indirectly obtain the target data from the device memory.

[0112] After retrieving data from the corresponding memory, the first directory controller forwards the data to the consistency controller of the requesting node and updates its directory table to record that the data has been loaded into the cache of the requesting node. This embodiment, through the centralized management of the first directory controller, ensures that the process of loading data from memory into the cache is correctly recorded and subsequently tracked for consistency status, thus maintaining global memory consistency.

[0113] The following is combined with Figure 3 The branches other than the first branch on the left explain another case where the data access request is a read request.

[0114] In an exemplary embodiment, when the data access request is a read request, the first directory controller is configured to: respond to the data access request received and forwarded by the request node consistency controller, query the first directory table, and after confirming that neither the host-side cache nor the device-side cache has a valid cache copy of the target data, determine the storage domain to which the target data belongs based on the physical address of the target data; if the target data is located in the host memory, send a first read instruction to the host memory controller to obtain the target data; if the target data is located in the device memory, send a second read instruction to the second directory controller to coordinate the acquisition of the target data from the device memory; receive the target data and forward it to the request node consistency controller that initiated the read request, and record the position of the target data in the request cache block and its status as exclusive in the first directory table; The second directory controller is configured to: in response to the second read instruction, send a third read instruction to the device memory controller to obtain target data from the device memory; after receiving the target data sent by the device memory controller, forward the target data to the first directory controller; the host memory controller is configured to: in response to the first read instruction, read the target data from the host memory and send it to the first directory controller; the device memory controller is configured to: in response to the third read instruction, read the target data from the device memory and send it to the second directory controller; the request node consistency controller of the request node is configured to: receive the target data from the first directory controller, instruct the corresponding request cache controller to store the target data in the request cache block, and set the status of the request cache block to an exclusive state for the request processor core to read.

[0115] In an exemplary embodiment, when the data access request is a write request; the first directory controller is configured to: respond to the data access request received and forwarded by the request node consistency controller, query the first directory table, and after confirming that there is no valid cached copy of the target data on the host, determine the storage domain to which the target data belongs based on the physical address of the target data; if the target data is located in the host memory, send a first read instruction to the host memory controller to obtain the target data; if the target data is located in the device memory, send a second read instruction to the second directory controller to coordinate the acquisition of the target data from the device memory; receive and forward the target data to the request node consistency controller that initiated the request, and record the position of the target data in the request cache block and its status as modified in the first directory table; The second directory controller is configured to: in response to the second read instruction, send a third read instruction to the device memory controller to obtain target data from the device memory; after receiving the target data, forward the target data to the first directory controller; the host memory controller is configured to: in response to the first read instruction, read the target data from the host memory and send it to the first directory controller; the device memory controller is configured to: in response to the third read instruction, read the target data from the device memory and send it to the second directory controller; the request node consistency controller of the request node is configured to: receive the target data from the first directory controller, instruct the corresponding request cache controller to store the target data into the request cache block, and after performing the write operation, set the status of the request cache block to the modified status to complete the write request.

[0116] This embodiment describes the complete process of handling read and write requests when the target data exists only in host memory or device memory (i.e., not in any cache). For both read and write requests, the first directory controller acts as the central hub for global querying and data routing. Upon receiving an access request forwarded by the requesting node's consistency controller, the first directory controller first queries its first directory table and collaborates with the second directory controller to confirm that there are no valid copies of the target data in any caches on the host and device sides, thus determining that the original data must be retrieved from main memory.

[0117] The first directory controller determines the storage domain to which the target data belongs based on its physical address. If the data is located in host memory, the first directory controller sends a read command directly to the host memory controller. If the data is located in device memory, the first directory controller sends a read command to the second directory controller, which acts as an intermediary to send a command to the device memory controller to retrieve the data. Regardless of the path, the data will ultimately be received by the first directory controller.

[0118] For a read request, the first directory controller, while forwarding the data to the requesting node's consistency controller, records in its first directory table that the data has been loaded into the requesting node's cache block and marks its status as exclusive. Upon receiving the data, the requesting node's consistency controller instructs its cache controller to store the data in a cache block and set it to exclusive status for the processor core to read. This status indicates that the node currently holds the only globally available cached copy of the data.

[0119] For write requests, the process is similar to that of read requests during the data retrieval phase. The difference lies in the state recorded in the directory table by the first directory controller: modified, rather than exclusive. After receiving the data, the consistency controller of the request node first allocates a cache block for the data. After the request processor core completes the actual write operation, it sets the corresponding cache block state to modified. This signifies that the node not only holds the data but has also modified it, becoming the latest and sole owner of the data.

[0120] This embodiment encapsulates the complex cross-domain address determination and data acquisition path through the management of the first directory controller, presenting a consistent interface to the requesting nodes. Whether reading or writing, and regardless of whether the data comes from host memory or device memory, the requesting node receives data from the first directory controller through its node consistency controller and sets the correct local cache state according to instructions. This design ensures that when data enters the cache system from memory, its initial ownership and consistency state are correctly and globally recorded.

[0121] Secondly, this application provides an electronic device, including a cache maintenance system for a heterogeneous computing system as described above. For a description of the electronic device, please refer to the above embodiments; further details will not be repeated here.

[0122] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0123] The foregoing has provided a detailed description of a cache maintenance system and electronic device for a heterogeneous computing system. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the methods and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A cache maintenance system for a heterogeneous computing system, characterized in that, The heterogeneous computing system includes a host side and a device side. The host side includes multiple processor nodes, and the device side includes at least one accelerator. The cache maintenance system includes: A distributed directory controller includes a first directory controller located on the host and a second directory controller located on the device; the first directory controller and the second directory controller are communicatively coupled. The first directory controller is configured to maintain a first directory table for managing the consistency state of memory and cache on the host side; the second directory controller is configured to maintain a second directory table for managing the consistency state of memory and cache on the device side. The cache maintenance system is configured to: respond to the data access request of the processor node on the host side for the target data, and perform consistency state coordination control with the directory controller corresponding to the storage domain where the target data is located, so as to perform global consistency maintenance operations between the host side and the device side.

2. The cache maintenance system for heterogeneous computing systems according to claim 1, characterized in that, At least one of the processor nodes includes: Multiple processor cores; Multiple caches, each corresponding one-to-one with one of the aforementioned processor cores; Multiple cache controllers, each corresponding to one of the caches, are configured to manage the consistency state of the corresponding cache; A node consistency controller is configured to coordinate cache consistency operations among the plurality of cache controllers within the processor node and to serve as a communication interface between different processor nodes and between the processor node and the first directory controller. The cache maintenance system is specifically configured as follows: in response to a data access request from a request processor to verify target data, it performs consistency state coordination control with the directory controller corresponding to the storage domain where the target data is located; the data access request includes the target physical address corresponding to the target data.

3. The cache maintenance system for heterogeneous computing systems according to claim 2, characterized in that, When the target data is located in the local cache corresponding to the request processor core; The local cache controller corresponding to the local cache is configured to: in response to the data access request of the request processor to verify the target data, perform the corresponding local consistency maintenance operation according to the type of the data access request; wherein, the type of the data access request includes read requests and write requests.

4. The cache maintenance system for heterogeneous computing systems according to claim 3, characterized in that, When the type of the data access request is a read request; The local cache controller is specifically configured to read the target data from the target cache block corresponding to the target physical address and transmit the target data to the request processor core.

5. The cache maintenance system for heterogeneous computing systems according to claim 3, characterized in that, When the type of the data access request is a write request; The local cache controller is specifically configured to: write the target data into the target cache block corresponding to the target physical address, update the consistency status of the target cache block to the modified status, and send a status update notification to the corresponding node consistency controller; The status update notification carries at least the target physical address of the target data and the identifier of the requesting processor core; the data in the cache block corresponding to the modified status representation has been changed and has not been written to host memory or device memory; the target cache corresponding to the target physical address includes the target cache block; The node consistency controller is configured to forward the status update notification to the first directory controller; The first directory controller is configured to: in response to the state update notification, perform consistent state coordination control based on the original state and the modified state to be updated of the target cache block in the first directory table.

6. The cache maintenance system for a heterogeneous computing system according to claim 5, characterized in that, The first directory controller is specifically configured as follows: In response to the status update notification, query the first directory table; When the original state of the target cache block in the first directory table is modified or exclusive, the state of the target cache block in the first directory table is updated to modified. When the original state of the target cache block in the first directory table is a shared state, a global invalidation operation is performed. The global invalidation operation includes: updating the state of the target cache block in the first directory table to a modified state, and sending an invalidation instruction to the second directory controller and / or the node consistency controller corresponding to the processor node holding other copies, so that the state of the corresponding cache copy in the device and other processor nodes is updated to an invalid state. The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only one cache has a copy of the data; the shared state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and at least two caches have copies of the data; the invalid state indicates that the cache block contains useless data.

7. The cache maintenance system for heterogeneous computing systems according to claim 2, characterized in that, When the target data is located in another cache within the same processor node as the requesting processor core that initiated the data access request; The node consistency controller is configured to: coordinate data access and transmission between the request cache controller corresponding to the request processor core and the target cache controller storing the target data, and interact with the first directory controller; The first directory controller is configured to update the consistency status of the corresponding cache block in the first directory table based on the message sent by the node consistency controller regarding data sharing and state changes within the processor node.

8. The cache maintenance system for a heterogeneous computing system according to claim 7, characterized in that, When the data access request is a read request; The node consistency controller is configured to: receive read requests from the request processor core; query the cache distribution within the processor node to determine the target cache block storing the target data and its corresponding target cache controller; Send a data read instruction to the target cache controller; receive the target data provided by the target cache controller, and forward the target data to the request cache controller corresponding to the request processor core; Forward the status update notification to the first directory controller; The target cache controller is configured to: if the current consistency state of the target cache block is exclusive or shared, then after providing the target data to the node consistency controller, update or maintain the consistency state of the target cache block to shared; if the current consistency state of the target cache block is modified, then after providing the target data to the node consistency controller, update the consistency state of the target cache block to shared. The request cache controller corresponding to the request processor core is configured to set the consistency state of the corresponding request cache block to a shared state after receiving the target data. The first directory controller is configured to: in response to the status update notification, update the status of the target cache block and the request cache block in the first directory table to the shared status, and perform consistent status collaborative control; The modified state indicates that the data in the corresponding cache block has been changed but not written to the host memory or device memory; The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only this one cache has a copy of the data; The shared state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and that at least two caches have copies of the data.

9. The cache maintenance system for a heterogeneous computing system according to claim 8, characterized in that, The target cache controller is further configured to provide a data write-back instruction when providing the target data to the node consistency controller; The node consistency controller is also configured to forward a data write-back instruction from the target cache controller to the first directory controller; The first directory controller is further configured to: in response to the data write-back instruction, write the target data back to its host memory or device memory according to the physical address of the target data.

10. The cache maintenance system for a heterogeneous computing system according to claim 7, characterized in that, When the data access request is a write request; The node consistency controller is configured to: receive a write request from the requesting processor core, query the cache distribution within its own processor node, determine the target cache block storing the target data and its corresponding target cache controller, and send a write operation notification to the target cache controller; Upon receiving information indicating that the shared state exists, the write request and the shared state identifier are forwarded to the first directory controller; Upon receiving the global invalidation confirmation information, the target cache controller is instructed to update the status of the target cache block to invalid status and transmit the target data to the request cache controller corresponding to the request processor core. The target cache controller is configured to: in response to the write operation notification, if the consistency state of the target cache block is an exclusive state or a modified state, update the state of the target cache block to an invalid state and transmit the target data to the request cache controller corresponding to the request processor core; If the consistency state of the target cache block is a shared state, then the node consistency controller is fed back with information that the shared state exists, and after receiving the global invalidation confirmation information, the state of the target cache block is updated to invalid state; The first directory controller is configured to: in response to the write request, perform a global invalidation operation, the global invalidation operation including: updating the status of the requested cache block in the first directory table to a modified status, and sending an invalidation instruction to the second directory controller and / or the node consistency controller corresponding to the processor node holding other copies, so that the status of the corresponding cache copy in the device and other processor nodes is updated to an invalid status; receiving and sending global invalidation confirmation information to the target cache controller and the node consistency controller; and updating the consistency status of the corresponding cache block in the first directory table according to the message sent by the node consistency controller regarding data sharing and status changes within the processor node. The request cache controller corresponding to the request processor core is configured to: after receiving the target data, allocate a cache block for the target data, set the status of the allocated cache block to the modified status, and feed back data reception completion and status update information to the node consistency controller.

11. The cache maintenance system for a heterogeneous computing system according to claim 2, characterized in that, When the target data is located in the cache of a different processor node than the request node where the request processor core that initiated the data access request is located; The first directory controller is configured to: in response to a data access request forwarded by the consistency controller of the request node corresponding to the request node, determine at least one target node where the target data is located based on the first directory table, and coordinate the request node and the node consistency controller corresponding to the at least one target node to perform cross-node data consistency maintenance operations. The requesting node is the processor node where the processor core that initiated the data access request is located, and the target node is a different processor node from the requesting node where the target data is located.

12. The cache maintenance system for a heterogeneous computing system according to claim 11, characterized in that, When the data access request is a read request; The request node consistency controller is configured to: receive a read request sent from the request processor core of the request node, and forward the read request to the first directory controller; and provide the received target data to the processor core that initiated the read request. The first directory controller is configured to: respond to the read request, query the first directory table, confirm the target cache block where the target data is located and its corresponding target node, send a data transmission instruction to the target node consistency controller corresponding to the target node; and determine the corresponding data migration path and status update strategy according to the consistency status in the target cache block where the target data is located recorded in the first directory table. After the read request is completed, the first directory table is updated to record that the requesting node has been added as a shared replica node of the target data; In response to a data write-back instruction, the target data is written back to its host memory or device memory according to the physical address of the target data; The target node consistency controller of the target node is configured to: read the target data from the target cache block in response to the data transmission instruction; if the target data is in an exclusive state or a shared state in the target cache, update or maintain the state of the target cache as a shared state through the target cache controller after providing the data; if it is in a modified state, update the state of the target cache as a shared state through the target cache controller while providing the data, and send the data write-back instruction to the first directory controller to send the target data to the requesting node; The modified state indicates that the data in the corresponding cache block has been changed but not written to the host memory or device memory; The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only this one cache has a copy of the data; The shared state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and that at least two caches have copies of the data.

13. The cache maintenance system for a heterogeneous computing system according to claim 11, characterized in that, When the data access request is a write request; The request node consistency controller is configured to: receive and forward the write request to the first directory controller; receive the target data migrated from the target node, allocate a cache block for it through the corresponding cache controller and set it to the modified state to complete the write operation; The first directory controller is configured to: in response to the write request, query the first directory table to determine the consistency status of the target node holding the target data and the corresponding target cache block; If the target data is in an exclusive or modified state in the target cache block, a data migration and state invalidation instruction is sent to the target node consistency controller corresponding to the target node; if it is in a shared state, a state invalidation instruction is sent to the node holding the target data and / or the second directory controller; and the state of the target cache block in the first directory table is updated to the modified state. The target node consistency controller is configured to: in response to a data migration and state invalidation instruction or invalidation instruction from the first directory controller, update the state of the target cache block to an invalid state through the corresponding target cache controller; if a data migration instruction is received, send the target data to the requesting node; The modified state indicates that the data in the corresponding cache block has been changed but not written to the host memory or device memory; The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only this one cache has a copy of the data; The shared state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and that at least two caches have copies of the data; The invalid state indicates that the cache block contains useless data.

14. The cache maintenance system for a heterogeneous computing system according to claim 2, characterized in that, The device also includes: Device-side caching; The device-side cache controller is configured to manage the read and write operations and cache block status of the device-side cache. When the target data is located in the device cache, the first directory controller is configured to forward the data access request to the second directory controller to interact with the second directory controller and jointly perform data consistency maintenance operations for the device cache. The second directory controller is configured to: in response to a data access request forwarded by the first directory controller, query the second directory table and interact with the device-side cache controller to manage and maintain the consistency state of the data in the device-side cache.

15. The cache maintenance system for a heterogeneous computing system according to claim 14, characterized in that, When the data access request is a read request; The first directory controller is configured to: in response to a read request forwarded by the consistency controller of the request node corresponding to the request node processor core that initiates the read request, query the first directory table, and after confirming that there is no valid copy of the target data on the host, send a data acquisition instruction to the second directory controller; Receive and respond to status information and target data from the second directory controller, update the first directory table and forward the target data to the requesting node consistency controller that initiated the request; The second directory controller is configured to: respond to the data acquisition instruction, query the second directory table to determine the consistency status of the target data in the device-side cache; instruct the device-side cache controller to perform corresponding data preparation and state transition operations based on the consistency status; receive the target data provided by the device-side cache controller and forward it to the first directory controller; The device-side cache controller is configured to: in response to an instruction from the second directory controller, access the device-side cache to obtain the target data; If the current consistency state is modified, the target data is written back to its memory and the state of the device cache is updated to shared; if the current consistency state is exclusive, the state of the device cache is updated to shared before the target data is provided. If the current consistency state is a shared state, then maintain the shared state and provide the target data; The modified state indicates that the data in the corresponding cache block has been changed but not written to the host memory or device memory; The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only this one cache has a copy of the data; The shared state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and that at least two caches have copies of the data.

16. The cache maintenance system for a heterogeneous computing system according to claim 14, characterized in that, When the data access request is a write request; The first directory controller is configured to: in response to a write request forwarded by the consistency controller of the request node corresponding to the request node processor core that initiates the write request, query the first directory table, and after confirming that there are no other valid copies of the target data on the host side, send an invalidation and data migration instruction to the second directory controller; Upon receiving and responding to the target data from the second directory controller, the target data is forwarded to the requesting node consistency controller that initiated the request, and the status of the target data in the first directory table is updated to modified status. The second directory controller is configured to: respond to the invalidation and data migration instruction, instruct the device-side cache controller to update the status of the target cache block storing the target data to an invalid state, and obtain the target data; after receiving the target data sent from the device-side cache controller, forward the target data to the first directory controller; The device-side cache controller is configured to: in response to the instruction of the second directory controller, read the target data from the device-side cache, update the status of the device-side cache block corresponding to the target data to an invalid state, and then provide the data to the second directory controller; The request node consistency controller of the requesting node that initiates the request is configured to: receive target data sent from the first directory controller, and store the target data into the request cache block and set it to the modified state through the corresponding request cache controller to complete the write operation; The modified state indicates that the data in the corresponding cache block has been changed and has not been written to the host memory or device memory; the invalid state indicates that the data in the cache block is useless.

17. The cache maintenance system for a heterogeneous computing system according to any one of claims 2-16, characterized in that, The host terminal also includes: Host memory, and a host memory controller configured to manage read and write access to said host memory; The device also includes: Device memory, and a device memory controller configured to manage read and write access to said device memory; When the target data is located in the host memory or the device memory; The first directory controller is configured to: respond to the data access request received and forwarded by the request node consistency controller, query the first directory table, and after confirming that neither the cache on the host side nor the cache on the device side has a valid copy of the target data, determine the storage domain to which the target data belongs based on the physical address of the target data, and directly or indirectly obtain the target data stored in the corresponding memory from the corresponding memory controller by sending a data request to the second directory controller, so as to maintain the data consistency after the target data is loaded into the request cache block.

18. The cache maintenance system for a heterogeneous computing system according to claim 17, characterized in that, When the data access request is a read request; The first directory controller is configured to: in response to the data access request received and forwarded by the request node consistency controller, query the first directory table, and after confirming that neither the cache on the host side nor the cache on the device side has a valid cache copy of the target data, determine the storage domain to which the target data belongs based on the physical address of the target data; If the target data is located in the host memory, a first read instruction is sent to the host memory controller to obtain the target data; If the target data is located in the device memory, a second read instruction is sent to the second directory controller to coordinate the retrieval of the target data from the device memory; The target data is received and forwarded to the consistency controller of the request node that initiated the read request, and the position of the target data in the request cache block and its state as exclusive are recorded in the first directory table. The second directory controller is configured to: in response to the second read instruction, send a third read instruction to the device memory controller to obtain the target data from the device memory; After receiving the target data sent by the device memory controller, the target data is forwarded to the first directory controller; The host memory controller is configured to: read the target data from the host memory and send it to the first directory controller in response to the first read instruction; The device memory controller is configured to: in response to the third read instruction, read the target data from the device memory and send it to the second directory controller; The request node consistency controller of the request node is configured to: receive the target data from the first directory controller, instruct the corresponding request cache controller to store the target data into the request cache block, and set the status of the request cache block to an exclusive state for the request processor core to read; The exclusive state indicates that the data in the cache block is consistent with the data in the host memory or device memory, and only this one cache has a copy of the data.

19. The cache maintenance system for a heterogeneous computing system according to claim 17, characterized in that, When the data access request is a write request; The first directory controller is configured to: in response to the data access request received and forwarded by the request node consistency controller, query the first directory table, and after confirming that there is no valid cached copy of the target data on the host, determine the storage domain to which the target data belongs based on the physical address of the target data; If the target data is located in the host memory, a first read instruction is sent to the host memory controller to obtain the target data; If the target data is located in the device memory, a second read instruction is sent to the second directory controller to coordinate the retrieval of the target data from the device memory; The target data is received and forwarded to the consistency controller of the request node that initiated the request, and the position of the target data in the request cache block and its status as modified are recorded in the first directory table; The second directory controller is configured to: in response to the second read instruction, send a third read instruction to the device memory controller to obtain the target data from the device memory; After receiving the target data, the target data is forwarded to the first directory controller; The host memory controller is configured to: read the target data from the host memory and send it to the first directory controller in response to the first read instruction; The device memory controller is configured to: in response to the third read instruction, read the target data from the device memory and send it to the second directory controller; The request node consistency controller of the request node is configured to: receive the target data from the first directory controller, instruct the corresponding request cache controller to store the target data into the request cache block, and set the status of the request cache block to the modified status after performing the write operation, so as to complete the write request operation; The modified state indicates that the data in the corresponding cache block has been changed but not written to the host memory or device memory.

20. An electronic device, characterized in that, Includes a cache maintenance system for heterogeneous computing systems as described in any one of claims 1 to 19.

Images