Heterogeneous computing system, cache consistency maintenance method and device, equipment and medium

Abstract

The invention discloses a heterogeneous computing system, a cache consistency maintenance method and device, equipment and a medium, and relates to the technical field of heterogeneous computing. The method comprises the steps that when a data operation request is detected, if a receiving party and a transmitting party are a cluster cache layer and a global cache layer, the protocol format of the data operation request is converted according to the mapping relation between a group consistency protocol and a global consistency protocol. And when the data is forwarded to the local cache layer, executing corresponding operation based on the memory consistency model of the calculation unit to which the data belongs, and maintaining the consistency of the local cache. And when the data is forwarded to a cluster cache layer, corresponding operation is executed based on the cluster consistency protocol, and the consistency of the cluster shared cache is maintained. And when the data is forwarded to a global cache layer, maintaining the consistency of the shared caches of the clusters based on a global consistency protocol. The method and the device can solve the problem of poor compatibility and expansibility of related technologies, can be compatible with various computing units and different memory consistency models thereof, and are easy to expand.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous computing, and in particular to a heterogeneous computing system and a cache consistency maintenance method, device, electronic device, non-volatile storage medium, and computer program product thereof. Background Art

[0002] Heterogeneous computing systems integrate computing units of different architectures. Each computing unit uses a different memory consistency model, which will affect system performance due to cross-device data competition. Any modification of data by any computing unit will cause system cache inconsistency. The relevant technology cannot flexibly integrate more different types of computing units on the basis of maintaining cache consistency and maintaining system high performance, and is not easy to expand. Summary of the invention

[0003] The present invention provides a heterogeneous computing system and a cache consistency maintenance method, device, electronic device, non-volatile storage medium, and computer program product thereof, which are compatible with multiple computing units and their different memory consistency models, and can more flexibly integrate different hardware components. When a new computing unit or a new cache consistency protocol is added to the heterogeneous computing system, there is no need to perform large-scale modifications to the entire system, and it is easy to expand.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: On one hand, the present invention provides a cache consistency maintenance method, comprising: When a heterogeneous computing system detects a data operation request, it determines whether the data operation request is forwarded to the local cache layer, the cluster cache layer or the global cache layer; computing units in the heterogeneous computing system that meet the same or similar conditions of the memory consistency model constitute a cluster, and each cluster includes a cluster cache layer that constructs the local cache of the computing units in the cluster into a cluster shared cache.

[0005] When the sender and receiver are the cluster cache layer and the global cache layer, the data operation request is converted into a protocol format according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface.

[0006] When forwarded to the target local cache layer, the corresponding operations are performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; when forwarded to the target cluster cache layer, the corresponding operations are performed based on the target group consistency protocol that matches the memory consistency model of each computing unit in the cluster, and the consistency of its cluster shared cache is maintained; when forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol.

[0007] Another aspect of the present invention provides a cache consistency maintenance device, comprising: A request forwarding module, which is used to determine whether to forward a data operation request to the local cache layer, the cluster cache layer, or the global cache layer when the heterogeneous computing system detects a data operation request; in the heterogeneous computing system, computing units that meet the same or similar conditions of the memory consistency model form a cluster, and each cluster includes a cluster cache layer that constructs the local caches of the computing units within the cluster into a cluster-shared cache.

[0008] A protocol conversion module, which is used to perform protocol format conversion on the data operation request through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol when the sending and receiving parties are the cluster cache layer and the global cache layer.

[0009] A request processing module, which is used to perform corresponding operations based on the memory consistency model of the computing unit to which it belongs and maintain the consistency of its local cache when forwarded to the target local cache layer; perform corresponding operations based on the target group consistency protocol that matches the memory consistency models of the computing units within the group and maintain the consistency of its cluster-shared cache when forwarded to the target cluster cache layer; maintain the consistency of the cluster-shared caches of each cluster based on the global consistency protocol when forwarded to the global cache layer.

[0010] The present invention also provides an electronic device, including a memory and a processor, and the processor is used to implement the steps of any of the above cache consistency maintenance methods when executing the computer program stored in the memory.

[0011] The present invention also provides a non-volatile storage medium, on which a computer program is stored, and the computer program is used to implement the steps of any of the above cache consistency maintenance methods when executed by a processor.

[0012] The present invention also provides a computer program product, including a computer program / instructions, and the computer program / instructions are used to implement the steps of any of the above cache consistency maintenance methods when executed by a processor.

[0013] The present invention finally also provides a heterogeneous computing system, at least including a first computing node, a second computing node, and a cache consistency controller, and the cache consistency controller is connected to the first computing node and the second computing node; the first computing node at least includes a first computing unit and a second computing unit, and the second computing node at least includes a third computing unit and a fourth computing unit; the memory consistency models of the first computing unit, the second computing unit, and the third computing unit respectively meet the same or similar conditions of the memory consistency model; Among them, the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit all include a local cache layer for managing their respective local caches. The first computing unit, the second computing unit, and the third computing unit form a first cluster. The first cluster includes a cluster cache layer that constructs the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster-shared cache. Moreover, the first computing unit, the second computing unit, and the third computing unit are connected to the cluster-shared cache through a high-speed interconnect network, and each cluster-shared cache performs consistency maintenance through a global cache layer. Among them, the cache coherence controller is used to implement the steps of any of the above cache coherence maintenance methods when executing a computer program.

[0014] The advantages of the technical solution provided by the present invention are as follows: The cache coherence maintenance of the heterogeneous computing system is divided into a local cache layer, a cluster cache layer, and a global cache layer. The local cache layer uses the cache management strategy natively supported by the computing unit for read and write operations and cache line state maintenance. The cluster cache layer adopts a coherence protocol matching the cluster memory model to manage the cache coherence of the shared cache within the cluster. The global cache layer realizes the cache coherence maintenance between different clusters. In this way, it can not only effectively solve the cache coherence maintenance problem caused by different memory coherence models in the heterogeneous computing system, but also be applicable to the complex and diverse data access patterns of the heterogeneous computing system, as well as the dynamic change of the data access pattern with the change of the application load. It will not be too simple and may lead to the inability to effectively handle complex heterogeneous scenarios, nor will it increase the complexity and management difficulty of the system, effectively improving the performance and reliability of the heterogeneous computing system. Through the hierarchical architecture and the coherence protocol adaptation layer, it can be compatible with a variety of heterogeneous computing units and their different memory coherence models, enabling the heterogeneous computing system to more flexibly integrate different hardware components. When a new computing unit or a new cache coherence protocol is added to the heterogeneous computing system, only configuration and adaptation need to be performed at the corresponding level, without large-scale transformation of the entire heterogeneous computing system, and it has good compatibility and scalability.

[0015] In addition, the present invention also provides a corresponding implementation device, electronic device, non-volatile storage medium, computer program product, and heterogeneous computing system for the cache coherence maintenance method, further making the method more practical, and the device, electronic device, non-volatile storage medium, computer program product, and heterogeneous computing system have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 Schematic diagram of the hardware composition framework applicable to the cache coherence maintenance method provided by the present invention; Figure 2 Flow chart of a cache coherence maintenance method provided by the present invention; Figure 3 Schematic diagram of the structure of the tree to be encoded in a schematic scenario provided by the present invention; Figure 4 Schematic diagram of the structure of the tree to be encoded in another schematic scenario provided by the present invention; Figure 5 Structure framework diagram of an exemplary embodiment of the cache coherence maintenance device provided by the present invention; Figure 6 Structure diagram of an exemplary embodiment of the electronic device provided by the present invention; Figure 7 Structure diagram of an exemplary embodiment of the heterogeneous computing system provided by the present invention; Figure 8 Structure diagram of another exemplary embodiment of the heterogeneous computing system provided by the present invention; Figure 9 Schematic diagram of the framework structure of the heterogeneous computing system provided by the present invention in a schematic scenario. Detailed implementation manners

[0018] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. Among them, the terms "first", "second", "third", "fourth", etc. in the specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. The term "exemplary" means "serving as an example, an embodiment or an illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0019] Heterogeneous computing is a technology that uses multiple different architectures of processors to work together in the same computing system to improve computing efficiency and performance. A heterogeneous computing system combines different types of processors such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), and FPGAs (Field Programmable Gate Arrays), giving full play to their respective advantages to achieve parallel computing. Compared with homogeneous systems, it can provide more computing power (i.e., data processing or computing capabilities) resources and is widely used in the field of AI (Artificial Intelligence). It can not only meet the high-efficiency processing requirements of massive data for AI model training and inference but also solve the problem of low energy efficiency ratio in homogeneous computing systems. When dealing with AI tasks, due to architectural limitations, a large amount of energy is consumed to maintain operations, resulting in high heat dissipation and energy consumption. It can dynamically allocate resources according to the characteristics of tasks, and different processors consume less energy when dealing with suitable tasks, effectively reducing the energy consumption and heat dissipation costs of AI applications. It also supports flexibly selecting and combining different types of processors and architectures according to different AI application scenarios and resource requirements, facilitating expansion and upgrading to adapt to the ever-changing AI technology and business needs.

[0020] Taking the heterogeneous computing system as an example, the heterogeneous computing system integrates multiple heterogeneous processors, accelerators, and memory components. It is usually composed of multiple computing units with significant differences in performance, power consumption, programming models, and memory consistency models, such as CPUs, GPUs, FPGAs, and NPUs (Neural network Processing Units). As the computing power gap between accelerators such as GPUs and NPUs and CPUs shows an order-of-magnitude differentiation, that is, the "memory wall effect" where the data throughput demand of the computing core far exceeds the memory bandwidth supply capacity, it leads to a serious imbalance in the performance of the heterogeneous computing system, such as problems of cross-device data competition and protocol rigidity. Among them, cross-device data competition refers to the difficulty in coordinating read and write operations across devices due to different memory consistency models of different computing units. For example, CPUs use a strong consistency model, while GPUs use a weak consistency model, which easily leads to data competition and affects the performance and data consistency of the entire system. Protocol rigidity means that traditional cache coherence protocols cannot be dynamically adjusted according to load changes, resulting in a decline in system performance. For example, the directory protocol is prone to directory expansion problems in high-write operation scenarios, and the broadcast protocol has excessive overhead in high-read operation scenarios.

[0021] To ensure the stable operation of heterogeneous computing systems, related technologies achieve cache coherence in heterogeneous computing systems through protocol stack innovation and hardware co-design. At the software protocol layer, related technologies maintain cache coherence in the computing load scenario of read-intensive and low-frequency write operations in heterogeneous computing systems through the bus snooping protocol, and maintain cache coherence in scenarios with frequent data updates through the directory protocol. Among them, in the bus snooping protocol, the private caches of each processing unit in the heterogeneous computing system continuously monitor bus transactions. When a coherence request broadcast on the bus is detected, a predefined response logic is triggered based on the local cache line state, and the physical characteristics of the bus medium are used to provide transaction atomicity and sequentiality guarantees, so that the state transition logic can be simplified to a deterministic finite state machine. Cache state synchronization can be achieved with relatively low difficulty through the global broadcast communication mode. However, with the expansion of the scale of multi-core processors, the bus load will increase. The characteristic that only one CPU is allowed to occupy the bus at a single moment leads to frequent blocking of memory access requests, resulting in a significant increase in memory access latency, easy resource waste. In addition, the scalability of the bus snooping protocol is poor and it is difficult to meet the requirements of large-scale processor systems. That is, the cache coherence of small-scale heterogeneous computing systems can be effectively maintained through the bus snooping protocol with relatively low difficulty. Among them, the target protocol achieves cache coherence through a global common directory that records the state information of global cache lines. The global common directory includes the coherence state and the list of replica owners. In the directory protocol, all coherence messages are forwarded through the directory structure, and efficient point-to-point message transmission can be performed by querying the list of replica owners, reducing network communication overhead. In addition, the directory protocol can effectively manage the replicas of shared data. By establishing a directory to track the storage location and state of data, multiple processors can conveniently access shared data, improving data availability and access efficiency. Large-scale multi-core processor systems can effectively coordinate cache interactions in a multi-core environment through centralized directory management, achieving efficient and reliable cache coherence maintenance. However, the storage and update operations of the directory both consume resources, which may affect the overall performance of the system. In addition, the directory protocol may introduce additional communication latency, reducing the speed of data access, which is not conducive to large-scale heterogeneous computing systems with high real-time requirements and limited resources. For the collaborative implementation of cache coherence maintenance between the software protocol layer and the hardware layer, related technologies are based on a heterogeneous coherence protocol based on directory strategies. It designs a composite coherence protocol fusion method for heterogeneous clusters, follows the global composite coherence model while retaining the coherence model of the original cluster, and uses proxy caches to fuse the coherence models of different clusters to ensure the consistency of the entire system. To achieve coherence maintenance, these two methods will introduce additional fusion overhead, with large performance overhead, and the additional overhead may affect system performance.

[0022] In summary, although the above related technologies can achieve cache coherence maintenance for heterogeneous computing systems to a certain extent, the update protocol requires a fixed global directory and cannot dynamically optimize the protocol strategy, resulting in poor flexibility and unsatisfactory performance in complex load scenarios. Protocol synthesis depends on static cluster configurations, lacks runtime adaptability, and cannot dynamically adjust the protocol according to the system state, leading to suboptimal performance in different load scenarios, inability to effectively adapt to diverse heterogeneous architectures, inability to be compatible with more computing units and their different memory coherence models while maintaining cache coherence, and lack of support for adding new computing units or new cache coherence protocols in heterogeneous computing systems, resulting in poor scalability.

[0023] In view of this, the present invention divides the cache coherence maintenance of a heterogeneous computing system into a local cache layer, a cluster cache layer, and a global cache layer. The local cache layer is responsible for cache management within a single computing unit; the cluster cache layer coordinates caches for clusters of computing units with the same or similar memory coherence models; the global cache layer implements cache coherence maintenance between different clusters, and a coherence protocol adaptation layer is introduced between each cluster cache layer and the global cache layer. The personalized cache coherence protocols used by different clusters are converted into a globally unified coherence protocol format to achieve seamless cooperation between different clusters, thereby being able to be compatible with multiple computing units and their different memory coherence models, more flexibly integrating different hardware components. When new computing units or clusters are added to the heterogeneous computing system, there is no need to conduct large-scale transformation of the entire system, making it easy to expand. In combination with the specific application environment architecture or specific hardware architecture on which the execution of the cache coherence maintenance method depends, the specific application environment architecture or specific hardware architecture is described herein. The following combines Figure 1 Some possible application scenarios related to the technical solution of the present invention are introduced by way of example, which may include the following content: In this embodiment, the heterogeneous computing system includes multiple servers 1. Each server 1 serves as a computing node of the heterogeneous computing system. Each server includes at least multiple computing units 10. In addition to the CPU as a computing unit itself, it also has at least built-in computing units of different types from the CPU architecture, such as GPUs, FPGAs, and ASICs. Each computing unit contains a local cache layer that manages its own internal local cache. Computing units with the same or similar memory coherence models are divided into a cluster. The computing units in the same cluster can be connected through a tree topology for the network topology. Each cluster is provided with a cluster cache layer that coordinates the caching of shared caches within the cluster. All clusters correspond to a global cache layer, and a coherence protocol adaptation layer is provided between each cluster cache layer and between the cluster cache layer and the global cache layer. The coherence protocol adaptation layer provides an interface externally.

[0024] To solve the problem of cache coherence maintenance difficulties caused by different memory coherence models in heterogeneous computing systems, improve the performance and reliability of the system, and at the same time have good compatibility and scalability, this embodiment proposes a cache coherence protocol maintenance method based on a hierarchical protocol architecture and cache coherence protocol adaptation. A cache coherence controller with computer program code implementing the following method steps can be built into any of the above servers or cloud servers. The cache coherence controller monitors data operation requests in the heterogeneous computing system, determines whether the data operation request is processed by the local cache layer, the cluster cache layer, or the global cache layer, and forwards it accordingly. When forwarding to the target local cache layer, corresponding operations are performed based on the memory coherence model of the computing unit to which it belongs, and the coherence of its local cache is maintained; when the sender and receiver are the cluster cache layer and the global cache layer, the protocol format of the data operation request is converted through a coherence protocol adaptation layer with an interface according to the mapping relationship between the group coherence protocol and the global coherence protocol; when forwarding to the target cluster cache layer, corresponding operations are performed based on the target group coherence protocol that matches the memory coherence model of each computing unit in the group, and the coherence of its cluster shared cache is maintained; when forwarding to the global cache layer, the coherence of each cluster shared cache is maintained based on the global coherence protocol. It should be noted that this application scenario is only shown for the convenience of understanding the idea and principle of the present invention, and the embodiments of the present invention are not restricted in this regard. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, the various non-limiting embodiments of the present invention will be described in detail below with reference to the drawings and specific embodiments.

[0025] It should be noted that the above application scenario is only shown for the convenience of understanding the idea and principle of the present invention, and the embodiments of the present invention are not restricted in this regard. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, the various non-limiting embodiments of the present invention will be described in detail below with reference to the drawings and specific embodiments. First, please refer to Figure 2 , Figure 2 which is a schematic flowchart of a cache coherence maintenance method provided in this embodiment. This embodiment may include the following content: S201: When the heterogeneous computing system detects a data operation request, determine whether the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer.

[0026] In the present invention, the heterogeneous computing system can be a multi - heterogeneous computing system, a non - multi - heterogeneous computing system, or any type of heterogeneous computing system, as long as it meets the definition of a heterogeneous computing system, and the present invention does not make any limitations in this regard. However, according to the architecture characteristics of different computing units, memory access patterns, and the requirements of application loads, the present invention hierarchically divides the heterogeneous computing system. First, the memory consistency models used by the computing units in the heterogeneous computing system are statistically classified by type, and the computing units with the same or similar memory consistency models, that is, the computing units that meet the conditions of the same or similar memory consistency models, are divided into the same group. These computing units that meet the conditions of the same or similar memory consistency models are a cluster. On this basis, a local cache layer is set locally in the computing units that need to maintain cache consistency in the heterogeneous computing system. The local cache layer is responsible for cache management within a single computing unit; a cluster cache layer is set within the same cluster. The cluster cache layer integrates the local memories of the computing units in the cluster into a shared cache, which is defined as the cluster shared cache for convenience of description. That is, each cluster includes a cluster cache layer that constructs the local caches of the computing units in the cluster into a cluster shared cache; the cluster cache layer coordinates caches for the computing unit clusters with the same or similar memory consistency models; the entire heterogeneous computing system has a global cache layer, which, as the final consistency guarantee node for the data of the entire heterogeneous system, manages the global shared cache using a strong consistency protocol, and the global cache layer realizes cache consistency maintenance between different clusters.

[0027] Among them, the data operation request is a read - write operation request of a computing unit to the memory. The cluster cache layer can be a cache update message or a data operation notification message for each computing unit in the cluster, or a broadcast message for data update, modification, or invalidation forwarded between clusters, or a protocol conversion request between clusters, or a protocol conversion request between a cluster and the global. Whenever a data operation request is detected, it is necessary to determine which local cache layer, which cluster cache layer, or the global cache layer is the processing party of the data operation request.

[0028] S202: When the sending and receiving parties are the cluster cache layer and the global cache layer, the protocol format of the data operation request is converted through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol.

[0029] It can be understood that the cache coherence maintenance of the heterogeneous computing system in this embodiment is divided into multiple levels, and the cache coherence protocols adopted at different levels perform corresponding maintenance on their respective computing units. Considering that there will be problems with inconsistent cache protocols when interacting between different levels. Before sending a data operation request to the processing party, in the case where the sender and receiver use different cache coherence protocols, it is necessary to first perform protocol format conversion. The local cache layer of the present invention performs cache management based on its own cache coherence protocol, and the cache coherence protocols of the local cache layers in the same cluster are the same or similar, so the interaction within the cluster does not involve protocol conversion. The present invention uses the global cache layer to achieve coherence maintenance between different clusters. Correspondingly, there will be data interaction between the cluster cache layer and the global cache layer. Different clusters adopt different cache coherence protocols, and the global cache layer will adopt the global cache coherence protocol. In order to achieve the interaction between the two layers, it is necessary to establish accurate and efficient protocol mapping rules for many different types of cluster local coherence protocols, that is, in this step, according to the differences in state definition, conversion logic, and message format of various protocols, find out their commonalities and differences in semantics and behavior, pre-construct the mapping relationship between different cluster coherence protocols and the global coherence protocol, perform semantic conversion and order adjustment on various memory operations (such as reads, writes, locks, etc.), ensure compliance with the global cache coherence requirements, convert the personalized cache coherence protocols used by different clusters into a globally unified coherence protocol format, ensure seamless cooperation between different clusters, and solve the problem of protocol incompatibility between heterogeneous clusters, providing basic support for global cache coherence maintenance.

[0030] For example, when the cluster cache layer sends a message to the global cache layer, after the coherence protocol adaptation layer receives the local protocol message, it first parses information such as the type, content, and target address of the message, and according to the predefined mapping rules, converts the format, status code, operation instructions, etc. of the message to adapt to the requirements of the global protocol, re-packages the message, generates a new message that conforms to the global protocol format, and sends it to the global cache layer. When the global cache layer sends a message to the cluster cache layer: the coherence protocol adaptation layer parses the global protocol message, extracts key information, and according to the local protocol type of the target cluster, applies the corresponding reverse mapping rules to convert the message into a local protocol message format that the cluster can understand. The converted message is sent to the cluster cache layer to ensure that the cluster can process it correctly. Of course, if data interaction is required between different clusters according to the actual application scenario, then the coherence protocol adaptation layer can also pre-construct the mapping relationship between different cluster coherence protocols, convert the personalized cache coherence protocol used by the cluster of the message sender into the coherence protocol format of the message receiver, realize the conversion between different cluster local coherence protocols, and achieve seamless cooperation between different clusters.

[0031] In addition, to support the scale - out of heterogeneous computing systems, such as adding new computing units or updating the existing cache coherence protocol of heterogeneous computing systems or adding new cache coherence protocols, it is necessary to modify the mapping relationship between the corresponding group coherence protocol and the global coherence protocol again. Based on this, the coherence protocol adaptation layer of the present invention also provides a user interface externally, through which the mapping relationship between the group coherence protocol and the global coherence protocol is allowed to be adjusted accordingly.

[0032] S203: When forwarding to the target local cache layer, perform corresponding operations based on the memory coherence model of the computing unit to which it belongs and maintain the coherence of its local cache; when forwarding to the target cluster cache layer, perform corresponding operations based on the target group coherence protocol that matches the memory coherence models of the computing units within the group and maintain the coherence of its cluster - shared cache; when forwarding to the global cache layer, maintain the coherence of each cluster - shared cache based on the global coherence protocol.

[0033] After determining the processing party based on the sender and receiver of the data operation request or the operation type, for the sake of convenience in description, the processing party being the specified local cache layer is defined as the target local cache layer, and the processing party being the specified cluster cache layer is defined as the target cluster cache layer. The target group consistency protocol is the cache consistency protocol used by the target cluster, that is, the cluster to which the target cluster cache layer belongs. Each computing unit is equipped with an independent local cache layer, and adopts the cache management strategy natively supported by this computing unit. For example, the target computing unit corresponding to the target local cache layer is a CPU adopting the TSO (Total Store Order, memory consistency model) memory model, and the target local cache layer performs read and write operations and maintains cache line states according to the requirements of TSO. The cluster cache layer manages the shared cluster cache using a consistency protocol matching the cluster memory model. For example, the target cluster corresponding to the target cluster cache layer is a GPU cluster adopting the RC (Release Consistency) model, and the target cluster cache layer allocates, invalidates, and updates cache lines for its cluster shared cache following the RC protocol. The computing units within the cluster are connected to the cluster shared cache through a high-speed interconnect network for data interaction. When a computing unit performs a write operation on shared data, it notifies other relevant computing units to update their local cache copies according to the cluster consistency protocol to ensure data consistency within the cluster. For example, the target group consistency protocol is the RC protocol. When a computing unit executes a release write operation, the cluster cache layer ensures that the write operation propagates in order within the cluster, and enables other computing units to see the result of the write operation before obtaining the corresponding acquire read operation. The target group consistency protocol adopts a publish-subscribe mode, and the cluster cache layer propagates cache line update and invalidation messages among the local cache layers within the cluster through the publish-subscribe mode. When a local cache layer performs a write operation on shared data, the cluster cache layer sends a notification to other local cache layers subscribing to this data, informing the computing units within the group that the data has been updated or invalidated, prompting other local cache layers to update or invalidate their corresponding cache copies. The global cache layer, as the ultimate consistency guarantee node for the data of the entire heterogeneous system, manages the global shared cache using a strong consistency protocol.

[0034] In the technical solution provided in this embodiment, the cache coherence maintenance of the heterogeneous computing system is divided into a local cache layer, a cluster cache layer, and a global cache layer. That is, the local cache layer uses the cache management strategy natively supported by the computing unit for read and write operations and cache line state maintenance. The cluster cache layer uses a coherence protocol matching the cluster memory model to manage the cache coherence of the shared cache within the cluster. The global cache layer realizes the cache coherence maintenance between different clusters. This can not only effectively solve the cache coherence maintenance problem caused by different memory coherence models in the heterogeneous computing system, but also adapt to the complex and diverse data access patterns of the heterogeneous computing system, as well as the dynamic change of the data access pattern with the change of the application load. It will not be too simple and may lead to the inability to effectively handle complex heterogeneous scenarios, nor will it increase the complexity and management difficulty of the system, effectively improving the performance and reliability of the heterogeneous computing system. Through the hierarchical architecture and the coherence protocol adaptation layer, it is possible to be compatible with a variety of heterogeneous computing units and their different memory coherence models, enabling the heterogeneous computing system to more flexibly integrate different hardware components. When a new computing unit or a new cache coherence protocol is added to the heterogeneous computing system, only configuration and adaptation need to be performed at the corresponding level, without large-scale transformation of the entire heterogeneous computing system, having good compatibility and scalability.

[0035] In the above embodiment, there is no limitation on how the target local cache layer processes data operation requests. The target local cache layer is the local cache layer of the target computing unit. The target computing unit includes a target local cache and uses a target cache coherence protocol. This embodiment also provides an exemplary processing method, which may include the following: For a data operation request that is a read operation request, that is, the data operation request is a memory read request of the target computing unit. When the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache line where the target read data block is located, and sent to the target computing unit. When the target local cache layer does not find the target read data block in the target local cache, a source cache line in the non-dirty state is selected from the target local cache for eviction, and the target read data block read from the main memory or the next-level storage is written into the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

[0036] In this embodiment, the target read data block is the data block that the target computing unit needs to read. When the computing unit performs a memory access operation, it first checks whether the target read data block exists in its local cache. If the target read data block exists in the local cache, the local cache is hit, and the read operation can be directly performed; if not, the data is obtained from the next-level storage according to the corresponding cache replacement policy. The next-level storage is the main memory or the cluster shared cache, and the local cache status is updated. This embodiment also provides an exemplary process for the local cache layer to process data operation requests: When the computing unit initiates a memory read request, it sequentially searches in the local cache according to the target read data block. If it is hit, the index information of the cache line corresponding to the target read data block in the local cache is located, and then it is determined whether a data read operation can be performed according to the tag of the cache line. The tag includes at least whether it is hit, the cache status, and the data storage address. The cache status includes the dirty state, the non-dirty state, the exclusive state, the modified state, the invalid state, and the shared state. If it is in the invalid state or the dirty state, it can wait or directly return an error prompt. The read target read data block or the prompt information is provided to the computing unit. At the same time, the status information such as the most recently used time of the cache line is updated to reflect its latest access situation. If it is not hit (Miss), according to the cache replacement policy, such as the improved LRU (Least Recently Used) algorithm, a cache line is selected for elimination. For the sake of description, it is defined as the source cache line. If the eliminated cache line is in the dirty state, that is, its data is inconsistent with the main memory or other upper-level storage, the dirty data is first written back to the next-level storage, such as the main memory or the cluster shared cache. The target read data block is read from the next-level storage and loaded into the local cache. The cache line written in the local cache is defined as the destination cache line. Then, the data is provided to the computing unit, and the relevant status information of the local cache is updated, such as marking the cache line as valid and updating its status, such as the exclusive state, etc.

[0037] For a data operation request that is a write operation request, that is, the data operation request is a memory write request of the target computing unit. When the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, according to the data write policy, the target write data block is written into the target local cache according to the tag and index of the target cache line where the target write data block is located, and the status information of the target cache line is updated to the dirty state; when the target local cache layer does not find the target write data block in the target local cache, a source cache line in the non-dirty state is selected from the target local cache for elimination, and the status information of the destination cache line corresponding to the target write data block written into the target local cache is updated to the to-be-written state. The target write data block is written into the destination cache line, and the status information of the destination cache line is updated to the dirty state.

[0038] Among them, the target write data block is the data block that the target computing unit needs to write. When the computing unit performs a write operation, it will first check whether the data block exists in its local cache. If the target write data block exists in the local cache, the local cache is hit; if the target write data block does not exist in the local cache, the local cache is not hit. If it is hit, according to the write policy, such as the write-back policy, the data is written into the local cache, and the status of the cache line is updated, such as being marked as the dirty state. If the write policy adopts the write buffer technology, the write operation data is first temporarily stored in the write buffer, and the target computing unit quickly continues to execute the subsequent tasks. The data in the write buffer is batch-written into the local cache or the next-level storage when certain conditions are met, such as the buffer being full or reaching a certain execution refresh condition. If it is not hit, a cache line, that is, the source cache line, is selected for replacement according to the cache replacement policy. If the replaced cache line is in the dirty state, the data is first written back. Then, the status of the cache line corresponding to the target data block, that is, the destination cache line, is updated to the writable state, and the new data is written into the local cache. At the same time, the dirty state of this cache line is recorded for subsequent write-back operations.

[0039] To better understand the working principle of the local cache layer, this embodiment also provides an example to illustrate the read and write operation processing flow of the local cache layer. In this embodiment, CPU core A is executing a program that needs to frequently access an integer array of size 100, and the memory address of this integer array starts from 0x1000. The configuration parameters of the local cache layer corresponding to CPU core A are: cache size: 32 KB, cache line size: 64 bytes, cache replacement policy: improved LRU algorithm, write policy: write-back policy combined with write buffer technology.

[0040] For the read operation request: The program instruction requests CPU core A to read the 10th element of the array. The local cache layer sequentially checks whether there is a cache line of the 10th element of the array in the L1 / L2 / L3 caches. If it is not in the local cache, the local cache layer selects a non-dirty cache line for replacement according to the improved LRU algorithm, reads the data block containing the 10th element of the array, with a size of 64 bytes, from the main memory, loads it into the local cache, updates the status of the cache line to valid, provides the read data to CPU core A, and updates the status information such as the recent use time of this cache line to make it the most recently used cache line.

[0041] For the write operation request: The program instruction requires CPU core A to modify the value of the 10th element of the array to 200. The local cache layer looks up the corresponding cache line. If the 10th element is already in the local cache and in a valid state. According to the write-back policy, the new value 200 is written to the corresponding location in the local cache, and the status of the cache line is updated to the dirty state. At the same time, the write operation is recorded in the write buffer. After the write buffer accumulates a certain amount of write operations, the data is written back to the main memory in batches. CPU core A quickly completes the write operation and continues to execute subsequent tasks.

[0042] The process for the local cache layer to maintain the consistency of its local cache includes: The target local cache layer monitors the data access of each computing unit in the heterogeneous computing system to the target local cache based on the target cache coherence protocol, and determines whether to update the data copy in the target local cache according to the data access situation and the status information of each cache line in the target local cache; when the target data copy in the target local cache has expired, the target data copy is marked as invalid or a data update operation is performed.

[0043] In this embodiment, the target local cache layer monitors the access of other computing units or clusters to the data in the target local cache of this computing unit by listening to the bus or using the directory method, etc. For example, when the target local cache layer adopts the bus snooping coherence protocol, when other components perform a write operation on the target local cache data, the target local cache layer will receive a corresponding message notification. Then, according to the message content of the received snooping message and the status of the data in the target local cache, it is decided whether to update the data copy in the target local cache. For example, if the data copy in the target local cache has expired, it is marked as invalid or updated to a new data value to ensure that the data in the target local cache is consistent with the data in the upper-level storage or other caches.

[0044] As can be seen from the above, this embodiment realizes the cache management inside a single computing unit through the local cache layer, performs local read and write operations on data and maintains the status of cache lines according to the cache policy natively supported by the computing unit. It can not only improve the local data access efficiency and quickly provide the required data for the computing unit, but also more simply achieve the consistency maintenance of the local cache, ensure data consistency, enable the local cache layer to provide efficient and reliable local data storage and access services for the computing unit in the heterogeneous computing system, support the performance improvement of the entire system, and is beneficial to improving the cache coherence maintenance accuracy of the heterogeneous computing system.

[0045] To further improve the effectiveness of cache coherence maintenance, based on the above embodiment, the present invention also provides an implementation method for dynamically adjusting the configuration parameters of the local cache layer, which may include the following content: Obtain the read / write operation data and access latency data of the target computing unit collected by the target local cache layer; determine the cache hit rate of the target local cache layer according to the read / write operation data, and determine the average access latency according to the access latency data; adjust the configuration parameters of the target local cache layer accordingly based on the cache hit rate and / or the average access latency.

[0046] In this embodiment, a performance monitor can be built into the target local cache layer, and the performance monitor is used to collect performance metric data such as cache hit count, miss count, read / write operation count, access latency, etc. in real time. By statistically analyzing these metrics, key performance parameters such as cache hit rate and average access latency are calculated. According to the change trend of the performance parameters and the set performance goals, the configuration parameters and management strategies of the local cache are adaptively and dynamically adjusted. For example, if it is found that the read operation hit rate continues to decline, the allocation ratio of the read cache in the local cache can be increased, or the weight parameter in the cache replacement strategy can be adjusted to improve the protection degree of the read cache line.

[0047] Exemplarily, an exemplary calculation method for the cache hit rate can be as follows: determine the read hit rate according to the read operation count and the read operation cache hit rate; determine the write hit rate according to the write operation count and the write operation cache hit rate; use the sum of the read hit rate and the write hit rate as the total hit rate, use the sum of the read operation count and the write operation count as the total operation count, and determine the cache hit rate of the target local cache layer according to the total hit rate and the total operation count. As an efficient cache hit rate calculation method, the cache hit rate calculation relation can be pre-stored, and the cache hit rate of the target local cache is calculated by calling the cache hit rate calculation relation. The cache hit rate calculation relation can be expressed as: where, represents the cache hit rate of the target local cache, represents the read operation cache hit rate, represents the write operation cache hit rate, represents the read operation count, represents the write operation count. The cache hit rate can reflect the overall hit efficiency of the local cache layer when processing read / write operations. By increasing the cache hit rate, the number of accesses to the next-level storage can be reduced, thereby reducing the access latency and improving the system performance. For example, in an application scenario dominated by read operations, can be optimized, such as increasing the proportion of the read cache space, thereby improving the overall performance.

[0048] For example, for a certain data block in the target local cache , the read count , the write count , the read hit rate of this data block , write hit rate , then its cache hit rate can be:

[0049] As can be seen from the above, in this embodiment, by evaluating the cache hit rate of the local cache layer, the cache hit conditions of different computing units can be dynamically captured, facilitating the dynamic adjustment of the size of the local cache according to the actual operating conditions, optimizing the cache replacement policy, and improving the local data access efficiency by optimizing the cache hit rate, etc., so that the local cache layer can better meet the memory access requirements of the computing units and improve the execution efficiency of the entire system.

[0050] The above embodiments do not make any limitations on how the target cluster cache layer processes data read and write operation requests. The target cluster cache layer is the cluster cache layer of the target cluster. The target cluster cache layer includes the target cluster shared memory and uses a target group coherence protocol that matches the memory coherence model of each computing unit in the group. For the convenience of description, for the computing units in the group whose read and write operations miss their own local caches, this embodiment defines them as the first target units, that is, the target cluster includes the first computing unit. The independent local cache of the first computing unit is the first local cache, and the first computing unit includes the first local cache layer, that is, the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs. The present invention also provides an exemplary implementation manner, which may include the following content: For a data operation request that is a read request, when the first computing unit performs a memory read operation and the data to be read is not in the first local cache, the first local cache layer forwards its memory read request to the cluster cache layer to which it belongs. That is, when the target cluster cache layer receives a data operation request that is a memory read request of the first computing unit, the target cluster cache layer will search in the target cluster shared memory to check if the data to be read exists. The data to be read is the data block that the first computing unit wants to read. If it exists, the data to be read is sent to the first local cache layer, and the cache line status information of the data to be read in the target cluster shared memory is updated. When the target cluster cache layer does not find the data to be read in the target cluster shared memory, it selects the first source shared cache line in the target cluster shared memory that is in a non-dirty state for eviction, reads the data to be read from the main memory or the next-level storage, writes it into the first destination shared cache line of the target cluster shared memory, sends the data to be read to the first local cache layer, and updates the status information of the first destination shared cache line. For any cluster cache layer of heterogeneous computing units, when the local cache layer of a computing unit within the cluster initiates a memory read request and misses, the request is forwarded to the cluster cache layer. The cluster cache layer searches for the corresponding data block in the cluster shared cache according to the content of the request. If it hits in the cluster shared cache, it immediately returns the data to the requesting local cache layer and updates the status information such as the most recently used time of the data block in the cluster shared cache. If it misses, the cluster cache layer will select a cache line, that is, the first source shared cache line, for eviction according to the preset cache replacement policy, such as an LRU variant suitable for the cluster memory model. If the evicted cache line is in a dirty state, its data needs to be written back to the main memory or other specified next-level storage locations, such as the relevant cache nodes in the global cache layer. The data to be read is read from the main memory or other next-level storage and loaded into the cluster shared cache, the status of this line, that is, the first destination shared cache line, is updated to valid, and then the data is returned to the requesting local cache layer.

[0051] For a data operation request that is a write request, when the first computing unit performs a memory write operation and the data to be written is not in the first local cache, the first local cache layer forwards its memory write request to the cluster cache layer to which it belongs. That is, when the target cluster cache layer receives a data operation request that is a memory write request of the first computing unit, the target cluster cache layer locates the data to be written in the target cluster shared memory. The data to be written is also the data block that the first computing unit wants to write. Then, according to the target group coherence protocol, the target cluster shared memory is updated, and the state of the second shared cache line of the data to be written in the target cluster shared memory is updated to the dirty state. The second shared cache line is the cache line occupied by the data block to be written in the target cluster memory. When the target cluster cache layer fails to locate the data to be written in the target cluster shared memory, it selects a second source shared cache line in the target cluster shared memory that is in a non-dirty state for eviction. The second source shared cache line is the cache line in the target cluster memory that is about to be evicted, and the second destination shared cache line is the cache line occupied by the data block to be written in the target cluster memory that is about to be written. The data to be written is written into the second destination shared cache line of the target cluster shared memory, the state information of the second destination shared cache line is updated, and the target group computing units in the target cluster related to the data to be written are notified based on the target group coherence protocol, so that each target group computing unit updates the data copy in its corresponding local cache. For any cluster cache layer of heterogeneous computing units, when a computing unit initiates a memory write request and the data to be written misses in the local cache layer or needs to be further written to the cluster shared cache layer according to the write policy, the write request is passed to the cluster cache layer. The cluster cache layer searches for the data to be written in the cluster shared cache. If a hit occurs, according to the coherence protocol adopted by the cluster, such as the RC protocol, the data is updated, and the corresponding cache line to be written is marked as the dirty state. At the same time, the local cache layers of other related computing units are notified that the data block has become invalid or needs to be updated. If a miss occurs, a cache line is selected for eviction according to the cache replacement policy. If the cache line is in a dirty state, it is first written to the main memory or the next-level storage and then evicted. The data to be written is written into the cluster shared cache, and the state information is updated and other affected local cache layers are notified.

[0052] To better understand the working principle of the cluster cache layer, this embodiment also provides an example to illustrate the read / write operation processing and cache consistency maintenance process of the cluster cache layer. This embodiment takes a cluster consisting of 4 homogeneous computing units, each using the RC memory model, as an example. The computing units within the cluster frequently access a shared data structure, such as a linked list with a size of 256 bytes, whose memory address range is from 0x2000 to 0x20FF. Computing unit 1 first reads the head of the linked list (address 0x2000 - 0x200F), then computing unit 2 modifies the middle part of the linked list (address 0x2030 - 0x203F), then computing unit 3 reads the tail of the linked list (address 0x20F0 - 0x20FF), and finally computing unit 4 modifies the head of the linked list. The data read / write operation processes of each computing unit include the following content: Among them, the process of computing unit 1 reading the head of the linked list: The cluster cache layer searches the cluster shared cache according to the address 0x2000 - 0x200F. Assuming a miss, the cluster cache layer reads the data block containing this address range from the main memory. This data block is 16 bytes in total and requires 4 cache lines, each cache line being 64 bytes. The data is loaded into the cluster shared cache and returned to the local cache layer of computing unit 1. At this time, the status of the relevant cache lines in the cluster shared cache is updated to valid.

[0053] Among them, the process of computing unit 2 modifying the middle part of the linked list: The write request arrives at the cluster cache layer, and the cache line corresponding to the address 0x2030 - 0x203F is searched. When a hit occurs, the cluster cache layer updates the data according to the RC protocol and marks this cache line as the dirty state. At the same time, a notification is sent to the local cache layers of other computing units, informing them that the data in the middle part of the linked list has been updated. After receiving the notification, the other local cache layers invalidate or update their corresponding cache copies.

[0054] Among them, the process of computing unit 3 reading the tail of the linked list: The cluster cache layer searches for the data corresponding to the address 0x20F0 - 0x20FF. If a miss occurs, the data is read from the main memory and loaded into the cluster shared cache, and the data is returned to the local cache layer of computing unit 3, while updating the corresponding cache status.

[0055] Among them, the process of computing unit 4 modifying the head of the linked list: The write request arrives at the cluster cache layer. The cluster cache layer searches for the cache line corresponding to the head of the linked list based on the address 0x2000 - 0x200F. If a hit occurs and the cache line is in the valid state, the data is updated and marked as the dirty state. An update notification is sent to other computing units to ensure that their cache copies are consistent with the data in the cluster shared cache.

[0056] As can be seen from the above, for a cluster of computing units with the same or similar memory consistency models, the present embodiment uses a coherence protocol that matches the cluster memory model through the cluster cache layer to maintain the state of the cluster shared cache lines, ensuring the consistency of the data within the cluster while reducing the access pressure on the global cache layer.

[0057] To further improve the effectiveness of cache coherence maintenance, based on the above embodiments, the present invention also provides an implementation method for dynamically adjusting the configuration parameters of the target cluster cache layer. The target cluster cache layer belongs to the target cluster and includes the target cluster shared memory, and may include the following content: Obtain the average propagation delay of cache line invalidation messages and the average propagation delay of cache line update messages in the target cluster shared memory; respectively obtain the invalidation weight coefficient corresponding to the cache line invalidation message and the update weight coefficient corresponding to the cache line update message; determine the intra-cluster cache coherence overhead of the target cluster according to the average propagation delay of the cache line invalidation message, the average propagation delay of the cache line update message, the invalidation weight coefficient, and the update weight coefficient; adjust the group configuration parameters and / or the intra-group communication network structure of the target cluster according to the intra-cluster cache coherence overhead of the target cluster.

[0058] In this embodiment, the overhead generated by the cluster cache layer in maintaining coherence is quantified according to the invalidation and update operations of the cache line. The group configuration parameters and / or the intra-group communication network structure can be flexibly adjusted according to the intra-cluster cache coherence overhead in combination with the actual application scenario. The cluster configuration parameters such as the invalidation operation frequency and the update operation frequency, and the intra-group communication network structure such as whether the computing units in the cluster use a tree topology structure or a star network structure for communication. If the data sharing among the computing units in the cluster is frequent and there are many write operations, the impact of the update operation on the overhead will be more significant. The intra-cluster communication network structure can be optimized accordingly to reduce the message propagation delay, or the coherence protocol can be optimized to reduce the frequency of update operations, thereby improving the performance of the cluster cache layer and making the entire system run more smoothly.

[0059] As an efficient implementation method, the calculation relationship formula of the intra-cluster cache coherence overhead can be pre-stored, and the intra-cluster cache coherence overhead of the target cluster can be calculated by calling this formula. The calculation relationship formula of the intra-cluster cache coherence overhead can be expressed as: represents the intra-cluster cache coherence overhead of the target cluster, is the average propagation delay of the cache line invalidation message, is the average propagation delay of the cache line update message, are respectively the weight coefficients of the invalidation and update operations in the cluster coherence maintenance. The two can adopt the default value of 0.5, or can be flexibly adjusted according to the actual cluster operation mode and application load.

[0060] As can be seen from the above, in this embodiment, the cluster cache layer is used to coordinate the memory access requests of multiple computing units, maintain data consistency within the cluster, and dynamically adjust the cache management policy, configuration parameters, or communication architecture according to the intra-cluster overhead to adapt to the access requirements of different computing units, which can effectively improve the data sharing efficiency within the cluster, reduce frequent access to the main memory, and enhance the performance of the entire cluster. At the same time, the consistency maintenance based on the RC protocol ensures the correctness and reliability of data interaction between computing units.

[0061] Considering that in a heterogeneous computing system under a heterogeneous network environment, the communication interfaces and transmission protocols of different clusters may vary, which increases the implementation difficulty of the communication mechanism. To ensure that data operation requests can flow quickly and accurately between different layers without introducing excessive latency and overhead, the present invention comprehensively considers factors such as bandwidth, latency, and reliability, and realizes efficient information transmission through the following embodiments, and has a highly compatible and adaptable communication: Corresponding local message queues are respectively set in each local cache layer, corresponding group message queues are respectively set in each cluster cache layer, and a global message queue is set in the global cache layer; whenever data interaction or status update occurs between each local cache layer and the corresponding cluster cache layer, a corresponding local message is generated, the local message is encapsulated according to the corresponding local message format, and placed in the corresponding local message queue; whenever data interaction or status update occurs between each cluster cache layer and the corresponding local cache layer, other cluster cache layers, or the global cache layer, a corresponding group message is generated, the group message is encapsulated according to the corresponding group message format, and placed in the corresponding group message queue; when the global cache layer and each cluster cache layer perform data interaction or status update, a global message is generated, the global message is encapsulated according to the corresponding global message format, and placed in the corresponding global message queue.

[0062] In this embodiment, when the local cache layer or the cluster cache layer needs to perform data interaction or status update with other layers, corresponding messages will be generated. For example, when the local cache layer has a cache miss and needs to obtain data from the upper-level cache layer, a read request message is generated; after the cluster cache layer updates the shared data, a data update notification message is generated to inform the upper-level global cache layer and other relevant cluster cache layers. After the message is generated, it can be encapsulated according to a predefined message format, such as adding information such as the target address, message type, data content, and sending sequence number, and then the message is placed in the message queue of this layer and waits to be sent.

[0063] As can be seen from the above, in this embodiment, the message queue can efficiently and accurately transfer cache status information and memory access requests between different levels. When each level receives relevant requests or status change notifications, it processes and forwards them according to predefined rules, ensuring that information can flow accurately and timely between levels without introducing excessive delays and overheads, and having highly compatible and adaptable communication.

[0064] Further, based on the above embodiment, a message router can also be set for each cluster cache layer. Using the message router, monitor the local message queue of the computing units within the cluster and the cluster message queue; determine the receiving layer and sending timeliness of the target message according to the receiving address, message type, and message label of the target message, and send the target message according to the receiving layer and sending timeliness; the receiving layer is the local cache layer or cluster cache layer or global cache layer that receives the target message.

[0065] Among them, the message label includes at least a timeliness mark, such as a mark indicating high or low priority, a priority processing mark. In this embodiment, the message router monitors the messages in the message queue in real time and determines the next-hop level or component to which the message should be sent according to the target address and type of the message. For example, when a write-back message is generated by the local cache layer and after the message is written into the message queue of the local cache layer, the message router detects the message and will send it to the message queue of the cluster cache layer according to the target cluster address and message priority of the message, and then the cluster cache layer will perform subsequent processing. For example, if the target of the message is the global cache layer, it will be sent to the communication link leading to the global cache layer; if the message is to notify other cluster cache layers, the message will be sent to the corresponding cluster cache layer according to the cluster address resolution and routing table. During the process of the message router forwarding messages, when there are multiple messages at the same time, the message router will sort the messages based on the message label. For example, messages with higher timeliness requirements such as write requests and data update notifications are preferentially forwarded, while messages with relatively lower delay sensitivity such as read requests are appropriately postponed to optimize the communication efficiency of the entire system.

[0066] As can be seen from the above, this embodiment adopts a communication architecture based on message queues and message routers. Each level is provided with a message queue for temporarily storing messages to be sent and received. The message router forwards the messages to the corresponding levels or components according to the target address and type of the messages, establishing an efficient information transfer channel to ensure that messages flow quickly and accurately between the local cache layer, cluster cache layer, and global cache layer, and guaranteeing the timely interaction and sharing of data and status information between levels.

[0067] Further, to ensure the timeliness of message processing, based on the above embodiments, the present invention also provides a message receiving and processing process: when the receiving layer receives a target message, it verifies the integrity and legality of the target message, and after the verification passes, it executes the target message. Further, the message tag may further include identification information such as a message addition serial number to ensure that the message can be processed in the correct order at the receiving party. When the target message reaches the target level, the message is read from the communication link and preliminarily parsed to check whether the format of the target message is correct, whether the checksum matches, etc., to ensure the legality and integrity of the message. If the target message is legal and complete, it is distributed to the corresponding processing unit for execution according to the type of the target message. For example, a read request message is handed over to the data reading processing unit, and a data update notification message is handed over to the cache update processing unit. The processing unit performs corresponding operations according to the message content, such as reading data, updating the cache status, etc., and generates a response message or a new notification message when necessary. Further, to ensure that messages are not missed or not received, a preset message confirmation time period, such as 10s, can be preset. After the sender sends the message, it can start a timer and wait for the confirmation message from the receiver. After successfully processing the message, the receiver sends a confirmation message to the sender indicating that the message has been correctly received and processed. When the message reception confirmation information from the receiving layer is not received within the preset message confirmation time period, under the condition of meeting the preset maximum retransmission times, the target message is resent until the message reception confirmation information is received. If the sender does not receive the confirmation message within 10s when the timer times out, it is considered that the message may be lost or in error during transmission, and the message can be resent. To ensure communication efficiency, a preset maximum retransmission times, such as 6 times, can also be preset. When the retransmission times of the message reach 6 and the confirmation message is still not received, an error handling process can be triggered, such as reporting a system failure, etc.

[0068] As can be seen from the above, through the message confirmation and retransmission mechanism, this embodiment can ensure the reliability and orderliness of message delivery. At the same time, by adding identification information such as a serial number to the message, it is ensured that the message can be processed in the correct order at the receiving party.

[0069] To further improve the effectiveness of cache consistency maintenance, based on the above embodiments, the present invention also provides an implementation method for dynamically adjusting communication configuration parameters, which may include the following contents: Obtain the message retransmission rate of the heterogeneous computing system and the sending parameters of the target historical message; determine the message sending time of the target historical message according to the occupancy capacity value of the target historical message and the communication link bandwidth; use the communication link delay as the message transmission delay of the target historical message, and determine the retransmission time according to the message retransmission rate, message sending time, message transmission delay, and message processing time; determine the total inter-group communication time of a single message according to the message sending time, message transmission delay, message processing time, and retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

[0070] Among them, the sending parameters at least include the occupancy capacity value of the target historical message, communication link bandwidth, communication link delay, and message processing time. For example, the heterogeneous computing system includes 2 clusters: Cluster A and Cluster B, and each cluster has 2 computing units: Computing Unit 1 and Computing Unit 2. The communication link bandwidth is 10 Gbps, that is, it can transmit bits per second. The communication link delay is 0.1 microseconds, including physical medium transmission delay and interface processing delay. The local cache layer of Cluster A sends a read request message to the cluster cache layer of Cluster B, and the message size is 128 bytes, that is, 1024 bits, to obtain the shared data block. Among them, the sender and the receiver each need an average of 0.05 microseconds to process 1 message, and the message retransmission rate is 0.1%, that is, 1 out of every 1000 messages needs to be retransmitted on average. Then, the average time from the local cache layer of Cluster A sending the read request message to the cluster cache layer of Cluster B and getting a response: . The message transmission delay is the communication link delay = 0.1 microseconds. Message processing time: The sender processing time is 0.05 microseconds, and the receiver processing time is 0.05 microseconds, for a total of 0.1 microseconds. Retransmission time: The retransmission rate is 0.1%, and the average number of retransmissions is 0.001 times. Each retransmission needs to resend the message, including the sending time, transmission delay, and processing time, so the retransmission time = 0.001×(0.1024 + 0.1 + 0.1) = 0.0003124 microseconds. Total time: Message sending time + message transmission delay + message processing time + retransmission time = 0.1024 + 0.1 + 0.1 + 0.0003124 ≈ 0.3027124 microseconds. This time is the average time for a single message to be sent from Cluster A to Cluster B. If the process of Cluster B processing the read request and returning the response message is considered, and if the processing time is the same, the entire round-trip time is approximately 0.3027124×2 ≈ 0.6054248 microseconds.

[0071] As can be seen from the above, by calculating the total inter-group communication time, the performance of the inter-layer communication mechanism can be quantified, and the communication parameters and strategies can be further optimized according to actual needs, such as increasing the bandwidth, reducing the delay, and optimizing the message processing algorithm, etc., so as to improve the inter-layer communication efficiency and enhance the performance of the entire hierarchical cache coherence architecture.

[0072] In order to further improve communication efficiency and reduce communication latency, this embodiment can also perform compression processing on the message to be transmitted or the data operation request, so as to reduce the amount of message transmission, which may include the following content: Construct an initial dictionary according to a preset seed item. Each entry in the initial dictionary has a unique index and a corresponding string; match the target string with the same characters for the input data in the initial dictionary, form a new string by combining the target string with the next input data, and add the corresponding index value to the new string as a new entry. By continuously adding new entries to the initial dictionary, an adaptive message coding dictionary is generated; input each request character of the data operation request into the adaptive message coding dictionary in turn, determine the index information of each request character in the adaptive message coding dictionary, and determine the remaining characters in the data operation request that do not successfully match the adaptive message coding dictionary. The index information and the remaining characters are used as the data to be encoded; encode the data to be encoded according to the frequencies of different characters in the data to be encoded, generate encoded sequence information, and send the encoded sequence information instead of the data operation request. When the receiving party receives the encoded data operation information, use the same adaptive message coding dictionary as that used to generate the data to be encoded, combine the encoding process of the encoded sequence information, decode the received information, look up the corresponding string in the adaptive message coding dictionary according to the index of the data to be encoded, and output it. At the same time, update the adaptive message coding dictionary according to the previous output string and the current output string to keep it consistent with the encoding end.

[0073] Among them, the preset seed item can be, for example, common characters or phrases, such as the ASCII (American Standard Code for Information Interchange, a computer coding system based on the Latin alphabet) character set. Using common characters or phrases as the seed item, a dynamic "dictionary" is constructed: create an empty dictionary and assign the initial seed item to it. Each entry in the dictionary contains a unique index and a corresponding string. Continuously input new data, and expand the initial dictionary according to the matching situation of the new data in the current dictionary. That is, for each character in the input message, starting from the current pointer position, find the most matching string in the current dictionary, that is, the exactly same string. After finding the string, add the matching string and the next character to the dictionary, move the current pointer to the next character position, and repeat the above process to adapt to the new pattern in the message, and obtain an adaptive message coding dictionary that can be used to encode the data operation request. Of course, during the compression process of each request of the heterogeneous computing system using the adaptive message coding dictionary, it can also be continuously expanded according to the above method, so that the probability that the input characters can be matched will increase.

[0074] For the convenience of those skilled in the art to understand, in this embodiment, an adaptive message coding dictionary is taken as an example, where the dictionary D contains lowercase letters a - z in the ASCII character subset, and the index of the dictionary D starts from 0. Taking the data operation request as abababa as an example, the process of using the adaptive coding dictionary to process abababa to obtain the data to be encoded can be shown in Table 1. The dictionary D contains 26 characters, and the index corresponding to each character starts from 0, that is, the index of a is 0, the index of b is 1, and so on, and the index of z is 25. At the same time, the dictionary reserves space for adding new strings later. The pointer initially points to the first character 'a'. The first character 'a' (pointer position 0): Search for a matching string starting with 'a' in the dictionary D. Since the dictionary initially only has a single character, the matching string is 'a', output the index 0 corresponding to 'a', and combine the current matching string 'a' with the next character 'b' to form a new string "ab", and add it to the dictionary D. Because the initial dictionary has 26 characters, the next available index is 26, and add index 26 to it. The pointer moves to the next position, that is, moves to position 1, and the current character pointed to is 'b'. Search for the corresponding matching string in the dictionary D. There is 'b' in the initial dictionary, and the index is 1, so output index 1. At the same time, combine 'b' with the next character 'a' to form "ba", and add it to the dictionary, and the index is 27. The third character 'a' (pointer position 2): Search for "a" in the dictionary and output index 0. Combine 'a' with the next character 'b' to form "ab". At this time, this string already exists in the dictionary, so there is no need to add it again, and the index of "ab" is 26. The pointer continues to move to the next position, that is, moves to position 3. At this time, the current character pointed to by the pointer is 'b'. There is already 'b' (index 1) in the dictionary D, so output index 1. At the same time, combine 'b' and the next character 'a' to form "ba". This combination already exists in the dictionary, and the index is 27. The pointer continues to move to the next position, that is, moves to position 4. At this time, the current character pointed to by the pointer is 'a'. There is 'a' in the dictionary D, and the index is 0, output index 0. Combine 'a' and the next character 'b' to form "ab", and the index is 26. The pointer continues to move to the next position, that is, moves to position 5. At this time, the current character pointed to by the pointer is 'b'. The matching string starting with 'b' in the dictionary D is 'b', and the index is 1, so output index 1. Combine 'b' and 'a' to form "ba", and the index is 27. The pointer continues to move to the next position, that is, moves to position 6. At this time, the current character pointed to by the pointer is 'a'. There is 'a' in the dictionary D, and the index is 0, output index 0. The sequence of the data to be encoded output is: [0, 1, 0, 1, 0, 1, 0]. All characters of the data operation request in this embodiment match the strings, so there are no remaining characters. Table 1 Adaptive Dictionary Coding Table

[0075] The above embodiments do not impose any limitations on the process of encoding the data to be encoded to generate the encoded sequence information. The present invention also provides an exemplary encoding method, which may include the following: Obtain each different character of the data to be encoded and its frequency of occurrence in the data to be encoded as the character information to be encoded; use each different character as a node of the tree structure, and select the first character and the second character with the lowest frequencies therefrom as leaf nodes, and merge the first character and the second character into the parent node of the leaf node, and the frequency of the parent node is the sum of the frequencies of the first character and the second character; select the third character with the lowest frequency from the remaining characters in the character information to be encoded, use the third character and the parent node as sibling sub-nodes at the same level, merge the third character and the parent node to obtain their parent node, and repeat continuously until there are no remaining characters in the character information to be encoded, to obtain the tree to be encoded; assign a first code value to each left sub-node of the tree to be encoded and a second code value to each right sub-node, and starting from the root node, recursively generate the encoded sequence for each character of the character information to be encoded based on the code values of each node, and combine the encoded sequences of each character of the character information to be encoded into the encoded sequence information. When encoding and decoding the encoded sequence information, for the convenience of encoding, an encoding table may be constructed. The encoding table includes at least items such as characters, frequencies of occurrence, and encoded sequences and their corresponding contents, and then the generated encoding table can be used to encode the characters. During decoding, starting from the root node of the tree to be encoded, traverse the tree bit by bit according to the input encoded sequence information until reaching the leaf node, and output the corresponding character.

[0076] In this embodiment, the characters of the data to be encoded are counted, and the frequency of occurrence of each character is calculated. Taking the data to be encoded as [0, 1, 0, 1, 0, 1, 0] as an example, the frequencies of occurrence of 1 and 0 in the data to be encoded are respectively counted. The frequency of the 0 character is 4, and the frequency of the character 1 is 3. Construct the tree to be encoded according to the frequencies: merge the two nodes with the lowest frequencies into a parent node, and the frequency of its parent node is the sum of the frequencies of the two child nodes. Repeat this process until only one root node remains. Starting from the root node, assign a first code value, such as "0", to the left sub-node, and assign a second code value, such as "1", to the right sub-node, and recursively generate the encoded sequence for each character to achieve the compression of a single character. Of course, the first code value and the second code value may also adopt other different numerical values or characters, which do not affect the implementation of the present invention. Still taking the above example as an example, the data to be encoded only has the character 0 and the character 1, and the corresponding tree to be encoded is as Figure 3As shown, character 0 is the left leaf node, and the first code value 0 can be assigned to it. Character 1 is the right leaf node, and the second code value 1 can be assigned to it. Then the path from the root node to character 0 is ->0, so the encoding sequence of character 0 is 0. The path from the root node to character 1 is ->1, so the encoding sequence of character 1 is 1. As shown in Table 2, using the encodings of the above characters to replace the output sequence [0, 1, 0, 1, 0, 1, 0] after adaptive dictionary encoding, its encoding sequence information is still [0, 1, 0, 1, 0, 1, 0]. Table 2 Encoding Table

[0077] For ease of understanding, this embodiment also provides another more complex embodiment. The data to be encoded is [0, 1, 17, 0, 2, 0, 3, 26]. The characters that appear are 3, 17, 2, 1, 26, 0. Count the occurrence frequencies of each character, select the two characters with the lowest frequencies, 3 and 17, as the two bottommost nodes, merge these two nodes into a parent node, and the frequency of its parent node is the sum of the frequencies of the two child nodes. Repeat this process until only one root node remains, obtaining the encoding tree as shown in Figure 4 As shown. Starting from the root node, assign 0 to the left child node and 1 to the right child node. The encoding sequences recursively generated for each character are as follows: The path from the root node to character 3 is: 0->0->0->0->0, so the encoding sequence of character 3 is 00000. The path from the root node to character 17 is: 0->0->0->0->1, so the encoding sequence of character 17 is 00001. The path from the root node to character 2 is: 0->0->0->1, so the encoding sequence of character 2 is 0001. The path from the root node to character 1 is: 0->0->1, so the encoding sequence of character 1 is 001. The path from the root node to character 26 is: 0->1, so the encoding sequence of character 26 is 01. The path from the root node to character 0 is: 1, so the encoding sequence of character 0 is 1. After using the encoding sequences of the above characters to replace the corresponding characters in the data to be encoded [0, 1, 17, 0, 2, 0, 3, 26] after adaptive dictionary encoding, the final encoding sequence information obtained is: 0001000010000100000001.

[0078] To ensure that the index can cover the entire dictionary, based on the above embodiment, the present invention also provides a method for determining the index encoding length: obtain the space capacity value of the adaptive message encoding dictionary; calculate the logarithm of the space capacity value with a preset number as the base, and take the ceiling value of the logarithm as the minimum index encoding length of the adaptive message encoding dictionary.

[0079] As an efficient computing method, the index coding length calculation relationship can be pre-stored, and the minimum index coding length of the adaptive message coding dictionary is calculated by calling the index coding length calculation relationship. The index coding length calculation relationship can be expressed as: Wherein, the index coding length depends on the dictionary size , is the ceiling function. To ensure that the index can cover the entire dictionary, the coding length is usually taken as the smallest integer that satisfies .

[0080] To verify the effectiveness of the coding method provided by the present invention, in this embodiment, the message abababa is taken as an example. The size of abababa is 7 bytes, with 1 byte for each character. The corresponding compression ratio is calculated through a pre-set compression ratio calculation relationship. The compression ratio (CR) is defined as the ratio of the original message size to the compressed message size. The compression ratio calculation relationship can be expressed as: Wherein, represents the original message size, in bytes, is the compressed message size, in bytes, and CR represents the compression ratio. The corresponding coding sequence information for this string is [0, 1, 0, 1, 0, 1, 0], with a length of 7 bits. 8 bits correspond to 1 byte size, and it is 1 byte. The dictionary index coding length bits, where 27 represents the maximum index of the dictionary. Then the total dictionary index coding length is equal to 7 * 5 = 35 bits, that is, 5 bytes. Therefore, the total compressed bytes are 1 (coding sequence information) + 5 (dictionary) = 6 bytes. So the compression ratio indicates the amount of compressed information.

[0081] As can be seen from the above, in this embodiment, the adaptive message coding dictionary efficiently compresses the repeated characters in the data operation request. By replacing the repeated strings with indexes, the data volume is greatly reduced. At the same time, combined with the coding method for the encoding tree to encode the remaining single characters, a higher compression ratio is achieved. The variable-length coding is generated based on the character occurrence frequency, with high-frequency characters corresponding to short codes, further reducing the data volume. The combination of the two can achieve efficient compression of the data operation request, effectively reducing the communication data volume, thereby reducing the communication delay and improving the system efficiency. In addition, the adaptive message coding dictionary can dynamically update its content, effectively capturing newly emerging repeated patterns and effectively improving the compression ratio.

[0082] The above embodiments do not make any limitations on the protocol conversion of the consistency protocol adaptation layer. Based on the above embodiments, the present invention also proposes a schematic protocol format conversion method, which may include the following content: Pre - establish a mapping rule table from each group consistency protocol to the global consistency protocol according to the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; when a new group consistency protocol appears in the heterogeneous computing system, determine the new mapping rule from the new group consistency protocol to the global consistency protocol, and update the mapping rule table accordingly through the interface; whenever an out - of - order data operation request is received, while caching the data operation request, assign a unique message sequence number to the data operation request; and perform protocol format conversion on the data operation requests in sequence according to the order of each message sequence number while preserving data dependency information.

[0083] In this embodiment, in order to complete the protocol conversion work between heterogeneous clusters, the consistency protocol adaptation layer needs to perform protocol mapping rule definition and message conversion processing. For the protocol mapping rule, it is based on establishing a mapping rule table from various local consistency protocols of clusters to the global consistency protocol. For example, for a CPU cluster using the MSI (Modified, Shared, Invalid) protocol and a GPU cluster using the Firefly protocol, define corresponding rules to map the state transitions and message types in the MSI protocol to the format specified by the global protocol, and at the same time perform corresponding conversion processing on the update messages and response messages in the Firefly protocol. Message conversion processing refers to performing message conversion between the cluster cache layer and the global cache layer. When the cluster cache layer sends a message to the global cache layer, parse, convert, and encapsulate the message content according to the predefined mapping rules to generate a message format that meets the requirements of the global protocol; conversely, when the global cache layer sends a message to the cluster cache layer, perform the reverse conversion operation as well to ensure that the message can be correctly understood and processed by the cluster. When the cluster cache layer sends a message to the global cache layer: After receiving the local protocol message, first parse information such as the type, content, and target address of the message. According to the predefined mapping rules, convert the format, status code, operation instructions, etc. of the message to adapt to the requirements of the global protocol. Re - encapsulate the message to generate a new message that conforms to the global protocol format and send it to the global cache layer. When the global cache layer sends a message to the cluster cache layer: Parse the global protocol message, extract key information, and according to the local protocol type of the target cluster, apply the corresponding reverse mapping rule to convert the message into the local protocol message format that the cluster can understand. Send the converted message to the cluster cache layer to ensure that the cluster can process it correctly.

[0084] During the operation of the heterogeneous computing system, when a new cluster type or protocol appears, the user can add a new protocol and update the existing protocol through the interface of the consistency protocol adaptation layer.

[0085] Furthermore, to ensure the consistency of the operation order during the message conversion process, a unique sequence number is assigned to each message, and the conversion and sending are carried out in the order of the sequence numbers. At the same time, the messages that arrive out of order are buffered until they can be processed in the correct order. For operations that depend on a specific order, such as the write-after-write order, the relevant dependency information is retained and passed during the message conversion to ensure that the global cache layer and the cluster cache layer can execute the operations in the correct order.

[0086] To make the technical solution of this embodiment clearer and more understandable to those skilled in the art, this embodiment also provides a schematic example. One cluster uses the MSI protocol, and another cluster uses the Firefly protocol. It is necessary to convert the "Modified" state of the MSI protocol to the "Exclusive" state of the global protocol and convert the update message of the Firefly protocol to the write message of the global protocol. Among them, the MSI protocol state mapping rules are: Modified → Exclusive, Shared → Shared, Invalid → Invalid; the Firefly protocol message mapping rules are: update message (Update) → write message (Write), where the data value in the Update message is mapped to the data value of the Write message, and the target address remains unchanged. Cluster A (MSI protocol) sends a cache line update message with a state of Modified to the global cache layer. The original MSI message: {Type: Update, State: Modified, Address: 0x1000, Data: 0xABC}; the converted global message: {Type: Update, State: Exclusive, Address: 0x1000, Data: 0xABC}. Cluster B (Firefly protocol) sends an update message to the global cache layer: the original Firefly message: {Type: Update, Address: 0x2000, Data: 0xDEF}, the converted global message: {Type: Write, Address: 0x2000, Data: 0xDEF}. The global cache layer sends a write message to Cluster A: the original global message: {Type: Write, Address: 0x1000, Data: 0x123}, the converted MSI message: {Type: Write, State: Modified, Address: 0x1000, Data: 0x123}. The global cache layer sends a read message to Cluster B: the original global message: {Type: Read, Address: 0x2000}, the converted Firefly message: {Type: Read, Address: 0x2000}. When Cluster A sends two consecutive write operation messages to the global cache layer, the sequence number of message 1 is 1, and the sequence number of message 2 is 2. Then the messages can be expressed as: message 1: {Seq: 1, Type: Write, Address: 0x1000, Data: 0xABC}, message 2: {Seq: 2, Type: Write, Address: 0x1000, Data: 0xDEF}.Due to network latency, when Message 2 arrives at the consistency protocol adaptation layer before Message 1, the message buffer of the consistency protocol adaptation layer will temporarily store Message 2 and wait for Message 1 to arrive. After Message 1 arrives, the consistency protocol adaptation layer converts and sends the messages in the sequence number order (1→2) to ensure that the global buffer layer performs write operations in the correct order.

[0087] As can be seen from the above, the protocol mapping rules and message conversion processing of the consistency protocol adaptation layer in this embodiment can achieve deep integration and seamless cooperation between multiple different memory consistency models. During runtime, it can identify the message types of the new protocol and apply the corresponding mapping rules for conversion, thereby realizing the dynamic expansion and compatibility maintenance of the system. This enables the system to be compatible with more types of heterogeneous computing units and provide more efficient and reliable consistency guarantee between different clusters.

[0088] To further optimize the heterogeneous computing system, it is necessary to make arbitration decisions quickly and accurately under the condition of real-time changing system load, reasonably allocate memory access rights to achieve the optimal overall performance of the system, and at the same time avoid introducing excessive computational overhead due to the complexity of the arbitration algorithm itself. Based on the above embodiments, the data processing operations of the global buffer layer can also be optimized, which may include the following contents: monitoring the memory access requests of each cluster's shared buffer layer; dynamically adjusting the access rights and access order of each memory access request according to the load information of the heterogeneous computing system.

[0089] In this embodiment, as the final consistency guarantee node of the data in the entire heterogeneous system, the global shared buffer, on the basis of being managed by a strong consistency protocol that comprehensively considers the characteristics of various memory models, can also monitor the memory access request queues of each cluster in real time. According to factors such as the current system load status, the performance characteristics of each cluster, and the priority of memory access, it dynamically arbitrates and coordinates the memory access requests of different clusters. For example, according to a preset scheduling algorithm, such as weighted round-robin, priority scheduling, etc., it sorts and selects the received requests and reasonably allocates memory access rights. For example, for the cluster where the real-time computing task with high performance requirements is located, it is given a higher access priority; at the same time, it dynamically adjusts the access right allocation of each cluster according to the system load to avoid resource competition and deadlock problems and ensure the efficient use of global memory resources.

[0090] As can be seen from the above, on the basis of reducing the cache consistency overhead in the global scope through the hierarchical cache management method, the dynamic arbitration and coordination in this embodiment can optimize memory access according to real-time situations, thereby improving the overall performance and response speed of the system, optimizing the overall performance of the system, and ensuring the correctness and consistency of data in the global scope.

[0091] It should be noted that there is no strict order of execution for each step in the present invention. As long as it conforms to the logical order, these steps can be executed simultaneously or in a certain preset order. Figure 2 This is just a schematic way and does not mean that it can only be such an execution order. Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0092] The present invention also provides a corresponding device for the cache coherence maintenance method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and the perspective of hardware respectively. The cache coherence maintenance device provided by the present invention will be introduced below. This device is used to implement the cache coherence maintenance method provided by the present invention. In this embodiment, the cache coherence maintenance device may include or be divided into one or more program modules. These one or more program modules are stored in a storage medium and executed by one or more processors to complete the cache coherence maintenance method disclosed in Embodiment 1. The program modules referred to in this embodiment refer to a series of computer program instruction segments that can complete specific functions, and are more suitable for describing the execution process of the cache coherence maintenance device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module in this embodiment. The cache coherence maintenance device described below can be correspondingly referred to the cache coherence maintenance method described above.

[0093] From the perspective of functional modules, see Figure 5 , Figure 5 is the structural diagram of the cache coherence maintenance device provided in this embodiment in a specific implementation manner. This device may include: A request forwarding module 501, configured to, when the heterogeneous computing system detects a data operation request, determine to forward the data operation request to the local cache layer, the cluster cache layer, or the global cache layer; in the heterogeneous computing system, computing units that meet the same or similar conditions of the memory coherence model form a cluster, and each cluster includes a cluster cache layer that constructs the local caches of the computing units within the cluster into a cluster-shared cache.

[0094] A protocol conversion module 502, configured to, when the sending and receiving parties are the cluster cache layer and the global cache layer, perform protocol format conversion on the data operation request through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol.

[0095] The request processing module 503 is configured to, when forwarding to the target local cache layer, perform corresponding operations based on the memory consistency model of the computing unit to which it belongs and maintain the consistency of its local cache; when forwarding to the target cluster cache layer, perform corresponding operations based on the target cluster consistency protocol that matches the memory consistency models of the computing units within the cluster and maintain the consistency of its cluster shared cache; when forwarding to the global cache layer, maintain the consistency of the cluster shared caches based on the global consistency protocol.

[0096] Exemplarily, in some embodiments of this embodiment, the above-mentioned request processing module 503 may also be configured to: the target local cache layer is the local cache layer of the target computing unit, the data operation request is the memory read request of the target computing unit, the target computing unit includes the target local cache. When the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache line where the target read data block is located and sent to the target computing unit; when the target local cache layer does not find the target read data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the target read data block read from the main memory or the next-level storage is written into the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

[0097] Exemplarily, in some other embodiments of this embodiment, the above-mentioned request processing module 503 may also be configured to: the target local cache layer is the local cache layer of the target computing unit, the data operation request is the memory write request of the target computing unit, the target computing unit includes the target local cache. When the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, according to the data write policy, the target write data block is written into the target local cache according to the tag and index of the target cache line where the target write data block is located, and the status information of the target cache line is updated to the dirty state; when the target local cache layer does not find the target write data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the status information of the destination cache line corresponding to the target write data block is updated to the to-be-written state, the target write data block is written into the destination cache line, and the status information of the destination cache line is updated to the dirty state.

[0098] Exemplarily, in some other embodiments of the present embodiment, the above request processing module 503 may further be configured to: the target local cache layer is the local cache layer of the target computing unit, the target computing unit includes a target local cache and uses a target cache coherence protocol, and the target local cache layer monitors the data access conditions of each computing unit in the heterogeneous computing system to the target local cache based on the target cache coherence protocol, and determines whether to update the data copy in the target local cache according to the data access conditions and the status information of each cache line in the target local cache; when the target data copy in the target local cache has become invalid, mark the target data copy as an invalid state or perform a data update operation.

[0099] Exemplarily, in some other embodiments of the present embodiment, the above request processing module 503 may further be configured to: the target local cache layer is the local cache layer of the target computing unit, and the target computing unit obtains the read / write operation data and access latency data of the target computing unit collected by the target local cache layer; determine the cache hit rate of the target local cache layer according to the read / write operation data, and determine the average access latency according to the access latency data; adjust the configuration parameters of the target local cache layer accordingly according to the cache hit rate and / or the average access latency.

[0100] As an exemplary implementation manner of the above embodiment, the above request processing module 503 may further be configured to: determine the read hit rate according to the number of read operations and the read operation cache hit rate; determine the write hit rate according to the number of write operations and the write operation cache hit rate; use the sum value of the read hit rate and the write hit rate as the total hit rate, use the sum value of the number of read operations and the number of write operations as the total number of operations, and determine the cache hit rate of the target local cache layer according to the total hit rate and the total number of operations.

[0101] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to: when the data operation request is for data to be read by the first computing unit and the data is not in the first local cache, forward a read request from the first local cache layer to the cluster cache layer to which it belongs. The target cluster cache layer includes a target cluster shared memory. When the target cluster cache layer finds the data to be read in the target cluster shared memory, it sends the data to be read to the first local cache layer and updates the cache line status information of the data to be read in the target cluster shared memory; when the target cluster cache layer does not find the data to be read in the target cluster shared memory, it selects a first source shared cache line in a non-dirty state from the target cluster shared memory for eviction, writes the data to be read read from the main memory or the next-level storage into the first destination shared cache line of the target cluster shared memory, sends the data to be read to the first local cache layer, and updates the status information of the first destination shared cache line; where the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

[0102] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to: when the data operation request is for data to be written by the first computing unit and the data is not in the first local cache, forward a read request from the first local cache layer to the cluster cache layer to which it belongs. The target cluster cache layer includes a target cluster shared memory. When the target cluster cache layer finds the data to be written in the target cluster shared memory, it updates the target cluster shared memory according to the target group consistency protocol and updates the status of the second shared cache line of the data to be written in the target cluster shared memory to the dirty state; when the target cluster cache layer does not find the data to be written in the target cluster shared memory, it selects a second source shared cache line in a non-dirty state from the target cluster shared memory for eviction, writes the data to be written into the second destination shared cache line of the target cluster shared memory, updates the status information of the second destination shared cache line, and notifies the target group computing units in the target cluster associated with the data to be written based on the target group consistency protocol, so that each target group computing unit updates the data copy in its corresponding local cache; where the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

[0103] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to: the target cluster cache layer belongs to the target cluster and includes a target cluster shared memory, obtain the average propagation delay of cache line invalidation messages and the average propagation delay of cache line update messages of the target cluster shared memory; respectively obtain the invalidation weight coefficient corresponding to the cache line invalidation message and the update weight coefficient corresponding to the cache line update message; determine the intra-cluster cache consistency overhead of the target cluster according to the average propagation delay of the cache line invalidation message, the average propagation delay of the cache line update message, the invalidation weight coefficient, and the update weight coefficient; and adjust the cluster configuration parameters and / or the intra-cluster communication network structure of the target cluster according to the intra-cluster cache consistency overhead of the target cluster.

[0104] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to: respectively set corresponding local message queues in each local cache layer, respectively set corresponding group message queues in each cluster cache layer, and set a global message queue in the global cache layer; whenever data interaction or status update occurs between each local cache layer and the corresponding cluster cache layer, generate a corresponding local message, encapsulate the local message according to the corresponding local message format, and put it into the corresponding local message queue; whenever data interaction or status update occurs between each cluster cache layer and the corresponding local cache layer, other cluster cache layers, or the global cache layer, generate a corresponding group message, encapsulate the group message according to the corresponding group message format, and put it into the corresponding group message queue; when data interaction or status update occurs between the global cache layer and each cluster cache layer, generate a global message, encapsulate the global message according to the corresponding global message format, and put it into the corresponding global message queue.

[0105] As an exemplary implementation of the above embodiment, the above request processing module 503 may further be configured to: use a message router to monitor the local message queue of the computing units within the group to which it belongs and the group message queue to which it belongs; determine the receiving layer and sending timeliness of the target message according to the receiving address, message type, and message label of the target message, and send the target message according to the receiving layer and sending timeliness; the receiving layer is the local cache layer or cluster cache layer or global cache layer that receives the target message, and the message label includes at least a timeliness mark.

[0106] As an exemplary implementation of the above embodiment, the above request processing module 503 may further be configured to: when the receiving layer receives the target message, verify the integrity and legality of the target message, and execute the target message after the verification passes; when the message reception confirmation information of the receiving layer is not received within the preset message confirmation time period, re-send the target message under the condition of meeting the preset maximum retransmission times until the message reception confirmation information is received.

[0107] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to: obtain the message retransmission rate of the heterogeneous computing system and the sending parameters of the target historical message; the sending parameters at least include the occupancy capacity value of the target historical message, the communication link bandwidth, the communication link delay, and the message processing time; determine the message sending time of the target historical message according to the occupancy capacity value of the target historical message and the communication link bandwidth; use the communication link delay as the message transmission delay of the target historical message, and determine the retransmission time according to the message retransmission rate, the message sending time, the message transmission delay, and the message processing time; determine the total inter-group communication time of a single message according to the message sending time, the message transmission delay, the message processing time, and the retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

[0108] Exemplarily, in some other embodiments of this embodiment, the above request processing module 503 may further be configured to construct an initial dictionary according to a preset seed item, each entry of the initial dictionary having a unique index and a corresponding string; match a target string with the same characters for the input data in the initial dictionary, form a new string by combining the target string with the next input data, and add the corresponding index value to the new string as a new entry, and generate an adaptive message coding dictionary by continuously adding new entries to the initial dictionary; sequentially input each request character of the data operation request into the adaptive message coding dictionary, determine the index information of each request character in the adaptive message coding dictionary, and determine the remaining characters in the data operation request that have not been successfully matched with the adaptive message coding dictionary, and use the index information and the remaining characters as the data to be encoded; encode the data to be encoded according to the frequencies of different characters appearing in the data to be encoded, generate encoded sequence information, and send the encoded sequence information instead of the data operation request.

[0109] As an exemplary implementation of the above embodiment, the above request processing module 503 may further be configured to: obtain each different character of the data to be encoded and its frequency of occurrence in the data to be encoded as the character information to be encoded; use each different character as a node of a tree structure, and select the first character and the second character with the lowest frequencies therefrom as leaf nodes, and merge the first character and the second character into the parent node of the leaf node, where the frequency of the parent node is the sum of the frequencies of the first character and the second character; select the third character with the lowest frequency from the remaining characters in the character information to be encoded, use the third character and the parent node as child nodes at the same level, and merge the third character and the parent node to obtain their parent node, and continuously repeat until there are no remaining characters in the character information to be encoded, so as to obtain the encoding tree; assign a first code value to each left child node of the encoding tree and a second code value to each right child node, and starting from the root node, recursively generate an encoding sequence for each character in the character information to be encoded based on the code values of each node, and combine the encoding sequences of each character in the character information to be encoded into encoding sequence information.

[0110] As another exemplary implementation of the above embodiment, the above request processing module 503 may further be configured to: obtain the space capacity value of the adaptive message encoding dictionary; calculate the logarithm of the space capacity value with a preset number as the base, and use the ceiling value of the logarithm value as the minimum index encoding length of the adaptive message encoding dictionary.

[0111] Exemplarily, in some other implementations of this embodiment, the above protocol conversion module 502 may further be configured to: establish a mapping rule table from each group consistency protocol to the global consistency protocol in advance according to the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; when a new group consistency protocol appears in the heterogeneous computing system, determine the new mapping rule from the new group consistency protocol to the global consistency protocol, and update the mapping rule table accordingly through the interface; whenever an out-of-order data operation request is received, while caching the data operation request, assign a unique message sequence number to the data operation request; convert the protocol format of the data operation requests in sequence according to the order of each message sequence number while retaining the data dependency information.

[0112] Exemplarily, in some other implementations of this embodiment, the above request processing module 503 may further be configured to: monitor the memory access requests of each cluster's shared cache layer; dynamically adjust the access permissions and access order of each memory access request according to the load information of the heterogeneous computing system.

[0113] The cache coherence maintenance device mentioned above is described from the perspective of functional modules. Further, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 6Schematic diagram of the structure of the electronic device provided by an embodiment of the present invention. The electronic device includes a memory 601 and a processor 602. A computer program is stored in the memory 601, and the processor 602 is configured to run the computer program to execute the steps in any of the above-described embodiments of the cache coherence maintenance method.

[0114] An embodiment of the present application further provides a non-volatile storage medium. A computer program is stored in the non-volatile storage medium, and the computer program is configured to execute the steps in any of the above-described embodiments of the cache coherence maintenance method when running.

[0115] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk, or optical disc, etc., various media that can store computer programs.

[0116] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the cache coherence maintenance method.

[0117] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the cache coherence maintenance method.

[0118] Finally, the present invention further provides a heterogeneous computing system, please refer to Figure 7, the heterogeneous computing system at least includes a first computing node 701, a second computing node 702, and a cache coherence controller 703. The cache coherence controller 703 is connected to the first computing node 701 and the second computing node 702. The first computing node 701 and the second computing node 702 each include at least two computing units. The cache coherence controller 703 is configured to implement the cache coherence maintenance method described in any of the above cache coherence maintenance method embodiments when executing a computer program. Among them, the first computing node 701 at least includes a first computing unit and a second computing unit, and the second computing node 702 at least includes a third computing unit and a fourth computing unit. The memory coherence models of the first computing unit, the second computing unit, and the third computing unit respectively satisfy the same or similar conditions of the memory coherence model. Among them, the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit all include a local cache layer for managing their respective local caches. The first computing unit, the second computing unit, and the third computing unit form a first cluster. The first cluster includes a cluster cache layer that constructs the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster shared cache. The first computing unit, the second computing unit, and the third computing unit are connected to the cluster shared cache through a high-speed interconnect network, and each cluster shared cache performs coherence maintenance through a global cache layer.

[0119] Exemplarily, such as Figure 8As shown, the cache coherence controller 603 may include a local cache layer component, a cluster cache layer component, an inter-layer communication component, a coherence protocol adaptation layer component, and a global cache layer component. The local cache layer component is used to manage the cache inside a single computing unit, perform local read and write operations on data and maintain the cache line state according to the cache policy natively supported by the computing unit, improve the local data access efficiency by optimizing the cache hit rate, etc., and quickly provide the required data for the computing unit. The cluster cache layer component is for a cluster of computing units with the same or similar memory coherence models, maintains the cluster shared cache, and manages the cache line state using a coherence protocol matching the cluster memory model to ensure data consistency within the cluster and reduce the access pressure on the global cache layer at the same time. The coherence protocol adaptation layer component is used to implement the conversion between different local coherence protocols of clusters and the global coherence protocol. Through protocol mapping rules and message conversion processing units, it ensures seamless cooperation among different clusters, solves the problem of protocol incompatibility between heterogeneous clusters, and provides basic support for global cache coherence maintenance. The global cache layer component, as the final consistency guarantee node for the data of the entire heterogeneous system, manages the global shared cache using a strong coherence protocol, sets up a dynamic arbiter to intelligently arbitrate and coordinate the memory access requests of each cluster, optimizes the overall performance of the system, and ensures the correctness and consistency of data on a global scale. The inter-layer communication component establishes an efficient information transmission channel to ensure the fast and accurate flow of messages between the local cache layer, the cluster cache layer, and the global cache layer. Through means such as message queues, message routers, etc., and corresponding communication protocols, it guarantees the timely interaction and sharing of data and status information between layers; these five components cooperate with each other to jointly complete the cache coherence maintenance process of the heterogeneous system. For example, as Figure 9 shown, the figure includes four groups, and each group adopts the same cache coherence maintenance protocol. Therefore, a local cache layer component is set up on each computing node within each group to be responsible for maintaining the cache state of the node. At the same time, a unified cluster cache component is set up within each group for message synchronization with other groups. Between groups, an efficient inter-layer communication component is required, and various types of information adapted to the inter-group coherence protocol are sent using the communication component. At the global level, a global cache component is set up to uniformly monitor and adjust the cache coherence maintenance process among groups within the system.

[0120] The above has introduced in detail a heterogeneous computing system and its cache coherence maintenance method, device, electronic device, non-volatile storage medium, and computer program product provided by the present invention. Each embodiment in this specification is described in a progressive manner, and the key point of each embodiment is the difference from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. Whether the described units and algorithm steps of each example in the disclosed embodiments are executed in the form of electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A cache coherence maintenance method, characterized in that, Including: When the heterogeneous computing system detects a data operation request, it determines that the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer; The computing units in the heterogeneous computing system that meet the same or similar conditions of the memory consistency model form a cluster. Each cluster includes a cluster cache layer that constructs the local caches of the computing units within the cluster into a cluster-shared cache; When the sender and receiver are the cluster cache layer and the global cache layer, the data operation request is subjected to protocol format conversion through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol; When forwarded to the target local cache layer, corresponding operations are performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; When forwarded to the target cluster cache layer, corresponding operations are performed based on the target group consistency protocol that matches the memory consistency models of the computing units within the group, and the consistency of its cluster-shared cache is maintained; When forwarded to the global cache layer, the consistency of the cluster-shared caches of all clusters is maintained based on the global consistency protocol.

2. The cache coherence maintenance method according to claim 1, wherein The target local cache layer is the local cache layer of the target computing unit, the data operation request is a memory read request of the target computing unit, and the target computing unit includes a target local cache. Performing corresponding operations based on the memory consistency model of the computing unit to which it belongs includes: When the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache line where the target read data block is located, and sent to the target computing unit; When the target local cache layer does not find the target read data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for eviction, and the target read data block read from the main memory or the next-level storage is written into the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

3. The cache coherence maintenance method according to claim 1, wherein The target local cache layer is the local cache layer of the target computing unit, the data operation request is a memory write request of the target computing unit, and the target computing unit includes a target local cache. Performing corresponding operations based on the memory consistency model of the computing unit to which it belongs includes: When the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, according to the data write policy, the target write data block is written into the target local cache according to the tag and index of the target cache line where the target write data block is located, and the status information of the target cache line is updated to the dirty state; When the target local cache layer fails to find the target write data block in the target local cache, it selects a source cache line in the non-dirty state in the target local cache for eviction, updates the status information of the destination cache line corresponding to the target write data block in the target local cache to the to-be-written state, writes the target write data block into the destination cache line, and updates the status information of the destination cache line to the dirty state.

4. The cache coherence maintenance method according to claim 1, wherein The target local cache layer is the local cache layer of a target computing unit. The target computing unit includes a target local cache and uses a target cache coherence protocol. The process of maintaining the coherence of its local cache includes: Based on the target cache coherence protocol, the target local cache layer monitors the data access of each computing unit in the heterogeneous computing system to the target local cache, and determines whether to update the data copy in the target local cache according to the data access and the status information of each cache line in the target local cache; When the target data copy in the target local cache has become invalid, mark the target data copy as the invalid state or perform a data update operation.

5. The cache coherence maintenance method according to claim 1, wherein The target local cache layer is the local cache layer of a target computing unit. The target computing unit includes a target local cache and further includes: Obtain the read / write operation data and access latency data of the target computing unit collected by the target local cache layer; Determine the cache hit rate of the target local cache layer according to the read / write operation data, and determine the average access latency according to the access latency data; Make corresponding adjustments to the configuration parameters of the target local cache layer according to the cache hit rate and / or the average access latency.

6. The cache coherence maintenance method according to claim 5, wherein Determining the cache hit rate of the target local cache layer according to the read / write operation data includes: Determine the read hit rate according to the number of read operations and the read operation cache hit rate; Determine the write hit rate according to the number of write operations and the write operation cache hit rate; Use the sum value of the read hit rate and the write hit rate as the total hit rate, use the sum value of the number of read operations and the number of write operations as the total number of operations, and determine the cache hit rate of the target local cache layer according to the total hit rate and the total number of operations.

7. The cache coherence maintenance method according to claim 1, wherein When the data operation request is a read request that is forwarded by the first local cache layer to the target cluster cache layer when the data to be read by the first computing unit is not in the first local cache, the target cluster cache layer includes a target cluster shared memory and performs corresponding operations based on a target group coherence protocol that matches the memory coherence model of each computing unit in the group, including: When the target cluster cache layer finds the data to be read in the target cluster shared memory, send the data to be read to the first local cache layer and update the cache line status information of the data to be read in the target cluster shared memory; When the target cluster cache layer fails to find the data to be read in the target cluster shared memory, it selects a first source shared cache line in a non-dirty state in the target cluster shared memory for eviction, writes the data to be read read from the main memory or the next-level storage into a first destination shared cache line in the target cluster shared memory, sends the data to be read to the first local cache layer, and updates the status information of the first destination shared cache line; Wherein, the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

8. The cache coherence maintenance method according to claim 1, wherein The data operation request is a write request forwarded by the first local cache layer to the cluster cache layer to which it belongs when the data to be written by the first computing unit is not in the first local cache. The target cluster cache layer includes a target cluster shared memory, and performs corresponding operations based on a target group consistency protocol that matches the memory consistency model of each computing unit in the group, and maintains the consistency of its cluster shared cache, including: When the target cluster cache layer finds the data to be written in the target cluster shared memory, it updates the target cluster shared memory according to the target group consistency protocol, and updates the status of the second shared cache line of the data to be written in the target cluster shared memory to a dirty state; When the target cluster cache layer fails to find the data to be written in the target cluster shared memory, it selects a second source shared cache line in a non-dirty state in the target cluster shared memory for eviction, writes the data to be written into a second destination shared cache line in the target cluster shared memory, updates the status information of the second destination shared cache line, and notifies the target group computing units in the target cluster related to the data to be written based on the target group consistency protocol, so that each target group computing unit updates the data copy in its corresponding local cache; Wherein, the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

9. The cache coherence maintenance method according to claim 1, wherein The target cluster cache layer belongs to the target cluster and includes a target cluster shared memory, and further includes: Obtaining the average propagation delay of cache line invalidation messages and the average propagation delay of cache line update messages in the target cluster shared memory; Respectively obtaining the invalidation weight coefficient corresponding to the cache line invalidation message and the update weight coefficient corresponding to the cache line update message; Determining the in-group cache consistency overhead of the target cluster according to the average propagation delay of the cache line invalidation message, the average propagation delay of the cache line update message, the invalidation weight coefficient, and the update weight coefficient; Adjusting the group configuration parameters and / or the in-group communication network structure of the target cluster according to the in-group cache consistency overhead of the target cluster.

10. The cache coherence maintenance method according to claim 1, wherein After determining that the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer, it further includes: A corresponding local message queue is set in each local cache layer, a corresponding group message queue is set in each cluster cache layer, and a global message queue is set in the global cache layer; Whenever data interaction or status update occurs between each local cache layer and the corresponding cluster cache layer, a corresponding local message is generated, the local message is encapsulated according to the corresponding local message format, and put into the corresponding local message queue; Whenever data interaction or status update occurs between each cluster cache layer and the corresponding local cache layer, other cluster cache layers or the global cache layer, a corresponding group message is generated, the group message is encapsulated according to the corresponding group message format, and put into the corresponding group message queue; When data interaction or status update occurs between the global cache layer and each cluster cache layer, a global message is generated, the global message is encapsulated according to the corresponding global message format, and put into the corresponding global message queue.

11. The cache coherence maintenance method according to claim 10, wherein A message router is set for each cluster cache layer. After determining that the data operation request is forwarded to the local cache layer, cluster cache layer or global cache layer, it further includes: Using the message router to monitor the local message queue of the computing units within the group and the group message queue; According to the receiving address, message type and message label of the target message, determine the receiving layer and sending timeliness of the target message, and send the target message according to the receiving layer and the sending timeliness; the receiving layer is the local cache layer or cluster cache layer or global cache layer that receives the target message, and the message label at least includes a timeliness mark.

12. The cache coherence maintenance method according to claim 11, wherein After determining that the data operation request is forwarded to the local cache layer, cluster cache layer or global cache layer, it further includes: When the receiving layer receives the target message, verify the integrity and legality of the target message, and execute the target message after the verification passes; If the message reception confirmation information of the receiving layer is not received within the preset message confirmation time period, the target message is re-sent under the condition of meeting the preset maximum retransmission times until the message reception confirmation information is received.

13. The cache coherence maintenance method according to claim 1, characterized in that It further includes: Obtain the message retransmission rate of the heterogeneous computing system and the sending parameters of the target historical message; the sending parameters at least include the occupancy capacity value, communication link bandwidth, communication link delay, and message processing time of the target historical message; Determine the message sending time of the target historical message according to the occupancy capacity value of the target historical message and the communication link bandwidth; Take the communication link delay as the message transmission delay of the target historical message, and determine the retransmission time according to the message retransmission rate, the message sending time, the message transmission delay and the message processing time; Determine the total inter-group communication time of a single message according to the message sending time, the message transmission delay, the message processing time and the retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

14. The cache coherence maintenance method according to any one of claims 1 to 13, characterized in that After determining that the data operation request is forwarded to the local cache layer, cluster cache layer or global cache layer, it further includes: Construct an initial dictionary according to preset seed items, where each entry in the initial dictionary has a unique index and a corresponding string; Match a target string with the same characters for the input data in the initial dictionary, form a new string by combining the target string with the next input data, and add a corresponding index value to the new string as a new entry. Generate an adaptive message encoding dictionary by continuously adding new entries to the initial dictionary; Input each request character of the data operation request into the adaptive message encoding dictionary in sequence, determine the index information of each request character in the adaptive message encoding dictionary, and determine the remaining characters in the data operation request that have not been successfully matched with the adaptive message encoding dictionary. Use the index information and the remaining characters as the data to be encoded; Encode the data to be encoded according to the frequencies of different characters in the data to be encoded, generate encoded sequence information, and send the encoded sequence information instead of the data operation request.

15. The cache coherence maintenance method according to claim 14, wherein Encoding the data to be encoded according to the frequencies of different characters in the data to be encoded includes: Obtain each different character in the data to be encoded and its frequency of occurrence in the data to be encoded as character information to be encoded; Use each different character as a node of a tree structure, select the first character and the second character with the lowest frequencies as leaf nodes, and merge the first character and the second character into the parent node of the leaf node. The frequency of the parent node is the sum of the frequencies of the first character and the second character; Select the third character with the lowest frequency from the remaining characters in the character information to be encoded, use the third character and the parent node as child nodes at the same level, merge the third character and the parent node to obtain their parent node, and repeat continuously until there are no remaining characters in the character information to be encoded, obtaining a tree to be encoded; Assign a first code value to each left child node and a second code value to each right child node of the tree to be encoded. Starting from the root node, recursively generate an encoding sequence for each character in the character information to be encoded based on the code values of each node, and combine the encoding sequences of each character in the character information to be encoded into the encoded sequence information.

16. The cache coherence maintenance method according to claim 14, wherein It also includes: Obtain the space capacity value of the adaptive message encoding dictionary; Calculate the logarithm of the space capacity value with a preset number as the base, and use the ceiling value of the logarithm as the minimum index encoding length of the adaptive message encoding dictionary.

17. The cache coherence maintenance method according to any one of claims 1 to 13, characterized in that Perform protocol format conversion on the data operation request through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol, including: Pre-establish a mapping rule table from each group consistency protocol to the global consistency protocol according to the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; When a new group consistency protocol appears in the heterogeneous computing system, determine the new mapping rule from the new group consistency protocol to the global consistency protocol, and update the mapping rule table accordingly through the interface; Whenever a data operation request arrives out of order, while caching the data operation request, a unique message sequence number is assigned to the data operation request; In accordance with the order of each message sequence number, and on the basis of preserving data dependency information, the protocol format conversion of the data operation request is performed sequentially.

18. The cache coherence maintenance method according to any one of claims 1 to 13, characterized in that, Maintain the consistency of each cluster shared cache based on the global consistency protocol, including: Monitor the memory access requests of each cluster shared cache layer; According to the load information of the heterogeneous computing system, dynamically adjust the access permissions and access order of each memory access request.

19. A cache coherence maintenance device, characterized in that Including: A request forwarding module, configured to determine that the data operation request is forwarded to the local cache layer, the cluster cache layer or the global cache layer when the heterogeneous computing system detects a data operation request; In the heterogeneous computing system, the computing units that meet the same or similar conditions of the memory consistency model form a cluster, and each cluster includes a cluster cache layer that constructs the local caches of the computing units within the cluster into a cluster shared cache; A protocol conversion module, configured to, when the sending and receiving parties are the cluster cache layer and the global cache layer, perform protocol format conversion on the data operation request through a consistency protocol adaptation layer with an interface according to the mapping relationship between the group consistency protocol and the global consistency protocol; A request processing module, configured to, when forwarded to the target local cache layer, perform corresponding operations based on the memory consistency model of the computing unit to which it belongs, and maintain the consistency of its local cache; When forwarded to the target cluster cache layer, perform corresponding operations based on the target group consistency protocol that matches the memory consistency model of each computing unit within the group, and maintain the consistency of its cluster shared cache; When forwarded to the global cache layer, maintain the consistency of each cluster shared cache based on the global consistency protocol.

20. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor, configured to implement the steps of the cache consistency maintenance method according to any one of claims 1 to 18 when executing the computer program.

21. A non-volatile storage medium, characterized in that, A computer program is stored on the non-volatile storage medium, and when the computer program is executed by a processor, the steps of the cache consistency maintenance method according to any one of claims 1 to 18 are implemented.

22. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the cache consistency maintenance method according to any one of claims 1 to 18 are implemented.

23. A heterogeneous computing system, characterized in that, At least including a first computing node, a second computing node and a cache consistency controller, the cache consistency controller is connected to the first computing node and the second computing node; the first computing node at least includes a first computing unit and a second computing unit, and the second computing node at least includes a third computing unit and a fourth computing unit; the memory consistency models of the first computing unit, the second computing unit, and the third computing unit each meet the same or similar conditions of the memory consistency model; Among them, the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit all include a local cache layer for managing their respective local caches. The first computing unit, the second computing unit, and the third computing unit form a first cluster. The first cluster includes a cluster cache layer that constructs the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster-shared cache. Moreover, the first computing unit, the second computing unit, and the third computing unit are connected to the cluster-shared cache through a high-speed interconnect network, and each cluster-shared cache performs consistency maintenance through a global cache layer; Among them, when the cache coherence controller executes a computer program, it implements the steps of the cache coherence maintenance method according to any one of claims 1 to 18.

Images