Heterogeneous computing system and cache consistency maintenance method, device, equipment, and medium

Abstract

The present invention discloses a heterogeneous computing system and a cache consistency maintenance method, device, equipment, and medium, which relate to the field of heterogeneous computing technology. The method includes: when a data operation request is detected, if the sending and receiving parties are a cluster cache layer and a global cache layer, the protocol format of the data operation request is converted according to the mapping relationship between the group consistency protocol and the global consistency protocol. When forwarded to the local cache layer, the corresponding operation is performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained. When forwarded to the cluster cache layer, the corresponding operation is performed based on the group consistency protocol, and the consistency of its cluster shared cache is maintained. When forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol. The present invention can solve the problems of poor compatibility and scalability of related technologies, can be compatible with multiple computing units and their different memory consistency models, and is easy to expand.

Description

Technical Field

[0001] The present invention relates to the field of heterogeneous computing, and in particular to a heterogeneous computing system and a cache consistency maintenance method, device, electronic device, non-volatile storage medium, and computer program product thereof. Background Art

[0002] Heterogeneous computing systems integrate computing units with different architectures. Each unit uses a different memory consistency model, which can impact system performance due to cross-device data contention. Data modifications by any unit can also lead to cache inconsistencies. While these technologies maintain cache coherence and high system performance, they lack the flexibility to integrate more diverse computing units and are difficult to scale. Summary of the Invention

[0003] The present invention provides a heterogeneous computing system and its cache consistency maintenance method, device, electronic device, non-volatile storage medium, and computer program product, which are compatible with multiple computing units and their different memory consistency models, can more flexibly integrate different hardware components, and when new computing units or new cache consistency protocols are added to the heterogeneous computing system, there is no need for large-scale transformation of the entire system, and it is easy to expand.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] In one aspect, the present invention provides a cache consistency maintenance method, comprising:

[0006] When a heterogeneous computing system detects a data operation request, it determines whether to forward the data operation request to the local cache layer, cluster cache layer or global cache layer; computing units in the heterogeneous computing system that meet the same similar conditions of the memory consistency model constitute a cluster, and each cluster includes a cluster cache layer that constructs the local cache of the computing units in the cluster into a cluster shared cache.

[0007] When the sending and receiving parties are the cluster cache layer and the global cache layer, the data operation request is converted into a protocol format according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface.

[0008] When forwarded to the target local cache layer, the corresponding operations are performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; when forwarded to the target cluster cache layer, the corresponding operations are performed based on the target group consistency protocol that matches the memory consistency model of each computing unit in the cluster, and the consistency of its cluster shared cache is maintained; when forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol.

[0009] Another aspect of the present invention provides a cache consistency maintenance device, comprising:

[0010] The request forwarding module is used to determine whether the data operation request is forwarded to the local cache layer, cluster cache layer or global cache layer when the heterogeneous computing system detects a data operation request; the computing units in the heterogeneous computing system that meet the same similar conditions of the memory consistency model constitute a cluster, and each cluster includes a cluster cache layer that constructs the local cache of the computing units in the cluster into a cluster shared cache.

[0011] The protocol conversion module is used to convert the protocol format of data operation requests according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface when the sending and receiving parties are the cluster cache layer and the global cache layer.

[0012] The request processing module is used to perform corresponding operations based on the memory consistency model of the computing unit to which it belongs and maintain the consistency of its local cache when forwarded to the target local cache layer; perform corresponding operations based on the target group consistency protocol that matches the memory consistency model of each computing unit in the cluster and maintain the consistency of its cluster shared cache when forwarded to the target cluster cache layer; maintain the consistency of each cluster shared cache based on the global consistency protocol when forwarded to the global cache layer.

[0013] The present invention also provides an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of any of the above-mentioned cache consistency maintenance methods when executing a computer program stored in the memory.

[0014] The present invention also provides a non-volatile storage medium having a computer program stored thereon, which implements the steps of any of the above-mentioned cache consistency maintenance methods when executed by a processor.

[0015] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above cache consistency maintenance methods when executed by a processor.

[0016] Finally, the present invention also provides a heterogeneous computing system, comprising at least a first computing node, a second computing node, and a cache consistency controller, wherein the cache consistency controller is connected to the first computing node and the second computing node; the first computing node comprises at least a first computing unit and a second computing unit, and the second computing node comprises at least a third computing unit and a fourth computing unit; the memory consistency models of the first computing unit, the second computing unit, and the third computing unit respectively satisfy the same similarity condition of the memory consistency model;

[0017] Among them, the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit all include a local cache layer for managing their respective local caches, the first computing unit, the second computing unit, and the third computing unit constitute a first cluster, the first cluster includes a cluster cache layer that constructs the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster shared cache, and the first computing unit, the second computing unit, and the third computing unit are connected to the cluster shared cache through a high-speed Internet network, and the shared caches of each cluster are maintained consistent through the global cache layer; wherein, the cache consistency controller is used to implement the steps of any of the above-mentioned cache consistency maintenance methods when executing a computer program.

[0018] The advantage of the technical solution provided by the present invention is that the cache consistency maintenance of the heterogeneous computing system is divided into a local cache layer, a cluster cache layer and a global cache layer. The local cache layer uses the cache management strategy natively supported by the computing unit to perform read and write operations and cache line state maintenance. The cluster cache layer uses a consistency protocol that matches the cluster memory model to manage cache consistency for the shared cache within the cluster. The global cache layer implements cache consistency maintenance between different clusters. This not only effectively solves the cache consistency maintenance problem caused by different memory consistency models in the heterogeneous computing system, but also adapts to the complex and diverse data access modes of the heterogeneous computing system, as well as the dynamic changes of the data access mode with the change of application load. It is not too simple and may lead to the inability to effectively handle complex heterogeneous scenarios, nor does it increase the complexity and management difficulty of the system, effectively improving the performance and reliability of the heterogeneous computing system. Through the hierarchical architecture and consistency protocol adaptation layer, it can be compatible with multiple heterogeneous computing units and their different memory consistency models, so that the heterogeneous computing system can more flexibly integrate different hardware components. When a new computing unit or a new cache consistency protocol is added to the heterogeneous computing system, it only needs to be configured and adapted at the corresponding level, without the need for large-scale transformation of the entire heterogeneous computing system, and has good compatibility and scalability.

[0019] In addition, the present invention also provides corresponding implementation devices, electronic devices, non-volatile storage media, computer program products and heterogeneous computing systems for the cache consistency maintenance method, further making the method more practical, and the devices, electronic devices, non-volatile storage media, computer program products and heterogeneous computing systems have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of a hardware composition framework applicable to the cache consistency maintenance method provided by the present invention;

[0022] Figure 2 A schematic diagram of a process for maintaining cache consistency provided by the present invention;

[0023] Figure 3 A schematic diagram of the structure of a tree to be encoded in an exemplary scenario provided by the present invention;

[0024] Figure 4 A schematic diagram of the structure of a tree to be encoded in another exemplary scenario provided by the present invention;

[0025] Figure 5 A structural framework diagram of an exemplary embodiment of the cache consistency maintenance device provided by the present invention;

[0026] Figure 6 A structural diagram of an exemplary embodiment of an electronic device provided by the present invention;

[0027] Figure 7 A structural diagram of an exemplary embodiment of a heterogeneous computing system provided by the present invention;

[0028] Figure 8 A structural diagram of another exemplary embodiment of the heterogeneous computing system provided by the present invention;

[0029] Figure 9 A schematic diagram of the framework structure of the heterogeneous computing system provided by the present invention in an illustrative scenario. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. The terms "first," "second," "third," "fourth," etc. in the specification and the accompanying drawings are used to distinguish different objects rather than to describe a specific order. Furthermore, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0031] Heterogeneous computing is a technology that uses multiple processors with different architectures to work together in the same computing system to improve computing efficiency and performance. Heterogeneous computing systems combine different types of processors, such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), and FPGAs (Field Programmable Gate Arrays), leveraging their respective strengths to achieve parallel computing. Compared to homogeneous systems, heterogeneous computing systems can provide more computing power (i.e., data processing capability or computational capacity). They are widely used in the field of artificial intelligence (AI). They not only meet the demand for efficient processing of massive amounts of data for AI model training and inference, but also address the low energy efficiency of homogeneous computing systems. When processing AI tasks, due to architectural limitations, large amounts of energy are consumed to maintain computing, resulting in high heat dissipation and energy consumption. Heterogeneous computing systems can dynamically allocate resources based on task characteristics, allowing different processors to consume less energy when handling appropriate tasks, effectively reducing the energy consumption and cooling costs of AI applications. They also support the flexible selection and combination of different processor types and architectures based on different AI application scenarios and resource requirements, facilitating expansion and upgrades to adapt to the ever-changing needs of AI technology and business.

[0032] Take multi-heterogeneous computing systems, for example. These systems integrate a variety of heterogeneous processors, accelerators, and memory components. These systems are typically composed of multiple computing units (such as CPUs, GPUs, FPGAs, and Neural Network Processing Units (NPUs)) with significant differences in performance, power consumption, programming models, and memory consistency models. However, as the computing power gap between accelerators like GPUs and NPUs and CPUs has widened by orders of magnitude, the data throughput demands of the computing cores have far outstripped the memory bandwidth available, leading to a "memory wall effect." This has led to significant performance imbalances in multi-heterogeneous computing systems, such as cross-device data contention and protocol rigidity. Cross-device data contention refers to the difficulty in coordinating read and write operations across devices due to differing memory consistency models (e.g., strong consistency for CPUs and weak consistency for GPUs). This can easily lead to data contention, impacting overall system performance and data consistency. Protocol rigidity refers to the inability of traditional cache coherence protocols to dynamically adapt to load fluctuations, resulting in reduced system performance. For example, directory protocols can easily cause directory bloat in scenarios with high write traffic, while broadcast protocols can be prohibitively expensive in scenarios with high read traffic.

[0033] To ensure the stable operation of heterogeneous computing systems, related technologies achieve cache coherence through protocol stack innovation and hardware co-design. At the software protocol layer, these technologies maintain cache coherence in heterogeneous computing systems in read-intensive, write-infrequent workloads using a bus snooping protocol, and in scenarios with frequent data updates using a directory protocol. In this bus snooping protocol, the private cache of each processing unit in the heterogeneous computing system continuously monitors bus transactions. Upon detecting a coherence request broadcast from the bus, predefined response logic is triggered based on the local cache line state. This leverages the physical properties of the bus medium to ensure transaction atomicity and ordering, simplifying the state transition logic into a deterministic finite state machine. Cache state synchronization can be achieved with minimal effort through a global broadcast communication model. However, as multi-core processors scale, bus loads increase. The fact that only one CPU is allowed to occupy the bus at a time leads to frequent blocking of memory access requests, significantly increasing memory access latency and potentially wasting resources. Furthermore, the bus snooping protocol has poor scalability, making it difficult to adapt to the needs of large-scale processor systems. In other words, the bus snooping protocol can effectively maintain cache coherence in small-scale heterogeneous computing systems with minimal effort. The target protocol achieves cache coherence through a global public directory that records the state information of global cache lines. This global public directory includes the coherence state and a list of replica owners. In the directory protocol, all coherence messages are forwarded through the directory structure. By querying the replica owner list, efficient point-to-point message transmission is achieved, reducing network communication overhead. Furthermore, the directory protocol effectively manages replicas of shared data. By establishing a directory to track the storage location and status of data, multiple processors can conveniently access shared data, improving data availability and access efficiency. Large-scale multi-core processor systems can effectively coordinate cache interactions in a multi-core environment through centralized directory management, achieving efficient and reliable cache coherence maintenance. However, storing and updating the directory consumes resources, which may affect overall system performance. Furthermore, the directory protocol may introduce additional communication latency, slowing data access, making it unsuitable for large-scale heterogeneous computing systems with high real-time requirements and limited resources. For cache consistency maintenance, the software protocol layer and the hardware layer collaborate to achieve this. Related technologies use heterogeneous consistency protocols based on directory policies. This approach designs a composite consistency protocol fusion method for heterogeneous clusters. This method adheres to a global composite consistency model while retaining the consistency model of the original cluster. It utilizes proxy caches to fuse the consistency models of different clusters to ensure consistency across the entire system. These two methods incur additional fusion overhead to achieve consistency maintenance, resulting in significant performance overhead, which may impact system performance.

[0034] In summary, while the aforementioned technologies can achieve cache consistency maintenance for heterogeneous computing systems to a certain extent, updating the protocol requires a fixed global directory, making it impossible to dynamically optimize protocol policies. This results in poor flexibility and poor performance in complex load scenarios. Protocol synthesis relies on static cluster configurations, lacks runtime adaptability, and cannot dynamically adjust the protocol based on system status. This results in suboptimal performance in different load scenarios, an inability to effectively adapt to diverse heterogeneous architectures, an inability to maintain cache consistency while being compatible with more computing units and their different memory consistency models, and a lack of support for adding new computing units or new cache consistency protocols to heterogeneous computing systems, resulting in poor scalability.

[0035] In view of this, the present invention divides the cache consistency maintenance of heterogeneous computing systems into local cache layer, cluster cache layer and global cache layer. The local cache layer is responsible for the cache management within a single computing unit; the cluster cache layer performs cache coordination for computing unit clusters with the same or similar memory consistency models; the global cache layer implements cache consistency maintenance between different clusters, and introduces a consistency protocol adaptation layer between each cluster cache layer and the global cache layer. The personalized cache consistency protocols used by different clusters are converted into a globally unified consistency protocol format to achieve seamless collaboration between different clusters, thereby being compatible with multiple computing units and their different memory consistency models, and more flexibly integrating different hardware components. When adding new computing units or clusters to heterogeneous computing systems, there is no need to make large-scale modifications to the entire system, and it is easy to expand. In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the cache consistency maintenance method depends, the specific application environment architecture or specific hardware architecture is described here. The following is combined with Figure 1 Some possible application scenarios involved in the technical solution of the present invention are introduced by way of example, which may include the following:

[0036] In this embodiment, the heterogeneous computing system includes a plurality of servers 1, each server 1 serving as a computing node of the heterogeneous computing system, and each server including at least a plurality of computing units 10. In addition to the CPU computing unit itself, it also has at least a built-in computing unit of a different type from the CPU architecture, such as a GPU, FPGA, and ASIC. Each computing unit includes a local cache layer for managing its own internal local cache, and the computing units with the same or similar memory consistency models are divided into a cluster. The computing units in the same cluster can be connected in a network topology structure through a tree topology. A cluster cache layer is set up for each cluster, and the cluster cache layer coordinates the cache of the shared cache within the cluster. All clusters correspond to a global cache layer. A consistency protocol adaptation layer is set up between the cluster cache layers and between the cluster cache layer and the global cache layer. The consistency protocol adaptation layer provides an interface to the outside.

[0037] To address the cache consistency maintenance challenges caused by different memory consistency models in heterogeneous computing systems, improve system performance and reliability, and maintain good compatibility and scalability, this embodiment proposes a cache consistency protocol maintenance method based on a hierarchical protocol architecture and cache consistency protocol adaptation. A cache consistency controller containing computer program code implementing the following method steps can be embedded in any of the aforementioned servers or cloud servers. The cache consistency controller monitors data operation requests from the heterogeneous computing systems, determines whether the data operation requests are processed by the local cache layer, the cluster cache layer, or the global cache layer, and forwards them accordingly. When forwarded to the target local cache layer, the corresponding operation is performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; when the sender and receiver are the cluster cache layer and the global cache layer, the data operation request is converted into a protocol format according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface; when forwarded to the target cluster cache layer, the corresponding operation is performed based on the target group consistency protocol that matches the memory consistency model of each computing unit in the group, and the consistency of its cluster shared cache is maintained; when forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol. It should be noted that this application scenario is only shown to facilitate understanding of the ideas and principles of the present invention, and the embodiments of the present invention are not subject to any restrictions in this respect. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, various non-restrictive embodiments of the present invention are described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0038] It should be noted that the above application scenarios are only shown to facilitate understanding of the ideas and principles of the present invention, and the embodiments of the present invention are not limited in this respect. On the contrary, the embodiments of the present invention can be applied to any applicable scenario. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments. First, please refer to Figure 2 , Figure 2 This is a flow chart of a cache consistency maintenance method provided in this embodiment. This embodiment may include the following contents:

[0039] S201: When a heterogeneous computing system detects a data operation request, it determines whether to forward the data operation request to a local cache layer, a cluster cache layer, or a global cache layer.

[0040] In the present invention, a heterogeneous computing system can be a multi-component heterogeneous computing system, a non-multi-component heterogeneous computing system, or any other type of heterogeneous computing system, as long as it meets the definition of a heterogeneous computing system. The present invention does not impose any restrictions on this. However, the present invention hierarchically divides the heterogeneous computing system based on the architectural characteristics, memory access patterns, and application load requirements of different computing units. First, the memory consistency models used by the computing units in the heterogeneous computing system are counted by type, and computing units with the same or similar memory consistency models, that is, computing units that meet the same or similar conditions for the memory consistency models, are divided into the same group. These computing units that meet the same or similar conditions for the memory consistency models are a cluster. On this basis, the computing units in the heterogeneous computing system that need to maintain cache consistency set up a local cache layer locally, which is responsible for the cache management within a single computing unit; a cluster cache layer is set up in the same cluster, and the cluster cache layer will integrate the local memory of the computing units in the cluster into a shared cache. For the sake of convenience, it is defined as a cluster shared cache, that is, each cluster includes a cluster cache layer that builds the local cache of the computing units in the cluster into a cluster shared cache; the cluster cache layer coordinates caches for computing unit clusters with the same or similar memory consistency models; the entire heterogeneous computing system has a global cache layer, which serves as the final consistency guarantee node for the data of the entire heterogeneous system, and adopts a strong consistency protocol to manage the global shared cache. The global cache layer realizes cache consistency maintenance between different clusters.

[0041] Among them, the data operation request is the read and write operation request of the computing unit to the memory, the cache update message or data operation notification message of the cluster cache layer to each computing unit in the group, or the broadcast message of data update, modification or invalidation forwarded between clusters, or the protocol conversion request between clusters, or the protocol conversion request between clusters and the global. Whenever a data operation request is detected, it will be determined which local cache layer, cluster cache layer or global cache layer is responsible for processing the data operation request.

[0042] S202: When the sending and receiving parties are the cluster cache layer and the global cache layer, the data operation request is converted into a protocol format according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface.

[0043] It is understandable that this embodiment divides the cache consistency maintenance of heterogeneous computing systems into multiple levels, and the cache consistency protocols adopted by different levels perform corresponding maintenance on the computing units to which they belong. Considering that when different levels interact, there will be problems with inconsistent cache protocols. Before sending the data operation request to the processor, for the case where a cache consistency protocol does not exist for both sending and receiving, it is necessary to convert the protocol format first. The local cache layer of the present invention performs cache management based on its own cache consistency protocol, and the cache consistency protocols of each local cache layer in the same cluster are the same or similar, so the interaction within the cluster does not involve protocol conversion. The present invention uses a global cache layer to achieve consistency maintenance between different clusters. Accordingly, there will be data interaction between the cluster cache layer and the global cache layer. Different clusters use different cache consistency protocols, and the global cache layer uses a global cache consistency protocol. In order to achieve interaction between the two layers, it is necessary to establish accurate and efficient protocol mapping rules for many different types of cluster local consistency protocols. That is, this step finds out the commonalities and differences in semantics and behaviors of various protocols based on the differences in state definition, conversion logic and message format, and pre-builds the mapping relationship between different group consistency protocols and the global consistency protocol. Various memory operations (such as reading, writing, locking, etc.) are semantically converted and sequenced to ensure compliance with global cache consistency requirements. The personalized cache consistency protocols used by different clusters are converted into a globally unified consistency protocol format to ensure that different clusters can collaborate seamlessly, solve the problem of protocol incompatibility between heterogeneous clusters, and provide basic support for global cache consistency maintenance.

[0044] For example, when the cluster cache layer sends a message to the global cache layer, the consistency protocol adaptation layer receives the local protocol message and first parses the message type, content, and destination address. Based on predefined mapping rules, it converts the message format, status code, and operation instructions to conform to the global protocol requirements. It then repackages the message, generates a new message conforming to the global protocol format, and sends it to the global cache layer. When the global cache layer sends a message to the cluster cache layer, the consistency protocol adaptation layer parses the global protocol message, extracts key information, and applies the corresponding inverse mapping rules based on the local protocol type of the target cluster to convert the message into a local protocol message format that the cluster understands. The converted message is then sent to the cluster cache layer to ensure correct processing by the cluster. Of course, if data exchange between different clusters is required in actual application scenarios, the consistency protocol adaptation layer can also pre-establish mappings between different clusters' coherence protocols. This allows for conversion between the personalized cache coherence protocol used by the message sender's cluster to the coherence protocol format of the message receiver. This enables seamless collaboration between different clusters.

[0045] In addition, in order to support the scale expansion of heterogeneous computing systems, such as adding new computing units, or updating the existing cache consistency protocol of the heterogeneous computing system or adding a new cache consistency protocol, it is necessary to modify the corresponding mapping relationship between the group consistency protocol and the global consistency protocol. Based on this, the consistency protocol adaptation layer of the present invention also provides a user interface to the outside world, through which the mapping relationship between the group consistency protocol and the global consistency protocol can be adjusted accordingly.

[0046] S203: When forwarded to the target local cache layer, the corresponding operations are performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; when forwarded to the target cluster cache layer, the corresponding operations are performed based on the target group consistency protocol that matches the memory consistency model of each computing unit in the cluster, and the consistency of its cluster shared cache is maintained; when forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol.

[0047] After determining the processor based on the sender and receiver of a data operation request or the operation type, for ease of description, the designated local cache layer is defined as the target local cache layer, and the designated cluster cache layer is defined as the target cluster cache layer. The target cluster coherence protocol is the cache coherence protocol used by the target cluster, that is, the cluster to which the target cluster cache layer belongs. Each compute unit is equipped with an independent local cache layer, which uses the cache management policy natively supported by that compute unit. For example, if the target compute unit corresponding to the target local cache layer is a CPU using the Total Store Order (TSO) memory model, the target local cache layer performs read and write operations and cache line state maintenance in accordance with TSO requirements. The cluster cache layer manages the shared cluster cache using a coherence protocol that matches the cluster's memory model. For example, if the target cluster cache layer corresponds to a GPU cluster using the Release Consistency (RC) model, the target cluster cache layer allocates, invalidates, and updates cache lines in its cluster shared cache according to the RC protocol. Computing units within the cluster are connected to the cluster shared cache via a high-speed interconnect network for data exchange. When a computing unit writes to shared data, it notifies other relevant computing units to update their local cache copies according to the cluster consistency protocol, ensuring data consistency within the cluster. For example, if the target cluster consistency protocol is the RC protocol, when a computing unit performs a release write operation, the cluster cache layer ensures that the write operation is propagated in order across the cluster and that other computing units can see the results of the write operation before obtaining the corresponding read operation. The target cluster consistency protocol adopts a publish-subscribe model, through which the cluster cache layer propagates cache line update and invalidation messages among the various local cache layers within the cluster. When a local cache layer writes to shared data, the cluster cache layer sends a notification to other local cache layers that have subscribed to the data, informing the computing units in the cluster that the data has been updated or invalidated, prompting other local cache layers to update or invalidate their corresponding cache copies. The global cache layer, as the node that ensures the eventual consistency of data across the entire heterogeneous system, uses a strong consistency protocol to manage the global shared cache.

[0048] In the technical solution provided in this embodiment, the cache consistency maintenance of the heterogeneous computing system is divided into a local cache layer, a cluster cache layer, and a global cache layer. The local cache layer uses the cache management strategy natively supported by the computing unit to perform read and write operations and cache line state maintenance, the cluster cache layer uses a consistency protocol that matches the cluster memory model to manage cache consistency for the shared cache within the cluster, and the global cache layer implements cache consistency maintenance between different clusters. This not only effectively solves the cache consistency maintenance problem caused by different memory consistency models in heterogeneous computing systems, but also adapts to the complex and diverse data access patterns of heterogeneous computing systems, as well as the dynamic changes in data access patterns as application loads change. It is not too simple and may not be able to effectively handle complex heterogeneous scenarios, nor does it increase the complexity and management difficulty of the system, effectively improving the performance and reliability of the heterogeneous computing system. Through the hierarchical architecture and consistency protocol adaptation layer, it is compatible with multiple heterogeneous computing units and their different memory consistency models, allowing heterogeneous computing systems to more flexibly integrate different hardware components. When a new computing unit or a new cache consistency protocol is added to a heterogeneous computing system, it only needs to be configured and adapted at the corresponding level, without the need for large-scale transformation of the entire heterogeneous computing system, and has good compatibility and scalability.

[0049] In the above embodiment, there is no limitation on how the target local cache layer processes the data operation request. The target local cache layer is the local cache layer of the target computing unit, the target computing unit includes a target local cache, and uses the target cache consistency protocol. This embodiment also provides an exemplary processing method, which may include the following:

[0050] For a data operation request that is a read operation request, that is, a memory read request for a target computing unit, when the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache line where the target read data block is located, and sent to the target computing unit; when the target local cache layer does not find the target read data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the target read data block read from the main memory or the next level storage is written to the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

[0051] In this embodiment, the target read data block is the data block that the target computing unit needs to read. When the computing unit performs a memory access operation, it will first search its local cache to see if the target read data block exists. If the target read data block exists in the local cache, the local cache is hit and the read operation can be performed directly. If it does not hit, the data is obtained from the next level of storage according to the corresponding cache replacement strategy. The next level of storage is the main memory or cluster shared cache, and the local cache status is updated. This embodiment also provides an exemplary process for the local cache layer to process data operation requests:

[0052] When a compute unit initiates a memory read request, it searches the local cache for the target read data block. If a hit is found, the cache line index information corresponding to the target read data block is located in the local cache. The cache line tag then determines whether the data read operation can proceed. The tag includes at least whether the read request was successful, the cache state, and the data storage address. Cache states include dirty, not dirty, exclusive, modified, invalid, and shared. If the cache state is invalid or dirty, the read operation can wait or directly return an error message. The read target read data block or the message is provided to the compute unit. Simultaneously, the cache line's state information, such as the last use time, is updated to reflect its most recent access. If a miss is found, a cache line is selected for eviction based on a cache replacement strategy, such as a modified LRU (Least Recently Used) algorithm. For ease of description, this cache line is defined as the source cache line. If the evicted cache line is dirty, meaning its data is inconsistent with main memory or other higher-level storage, the dirty data is first written back to the next-level storage, such as main memory or a cluster shared cache. The target read data block is read from the next level of storage and loaded into the local cache. The cache line written in the local cache is defined as the destination cache line. Then, the data is provided to the computing unit and the relevant status information of the local cache is updated, such as marking the cache line as valid and updating its status, such as exclusive status.

[0053] For data operation requests that are write operation requests, that is, data operation requests that are memory write requests for the target computing unit, when the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, then according to the data write strategy, based on the tag and index of the target cache line where the target write data block is located, the target write data block is written to the target local cache, and the status information of the target cache line is updated to a dirty state; when the target local cache layer does not find the target write data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the status information of the destination cache line corresponding to the target write data block written to the target local cache is updated to a pending write state, the target write data block is written to the destination cache line, and the status information of the destination cache line is updated to a dirty state.

[0054] The target write data block is the data block that the target compute unit needs to write. When a compute unit performs a write operation, it first checks its local cache to see if the target write data block exists. If the target write data block exists in the local cache, it is considered a local cache hit. If not, it is considered a local cache miss. If a hit occurs, the data is written to the local cache according to a write strategy, such as a write-back strategy, and the cache line status is updated, such as marking it dirty. If the write strategy utilizes write buffering, the write operation data is temporarily stored in the write buffer, allowing the target compute unit to quickly continue executing subsequent tasks. The data in the write buffer is batch-written to the local cache or next-level storage when it meets certain conditions, such as when the cache is full or certain refresh conditions are met. If a miss occurs, a cache line, namely the source cache line, is selected for eviction according to the cache replacement strategy. If the evicted cache line is dirty, the data is first written back. Then, the cache line corresponding to the target data block, namely the destination cache line, is updated to a writable state, and the new data is written to the local cache. The dirty state of the cache line is also recorded for subsequent write-back operations.

[0055] To better understand the working principle of the local cache layer, this embodiment also provides an example to illustrate the read and write operation processing flow of the local cache layer. This embodiment assumes that CPU core A is executing a program that frequently accesses an integer array of size 100, whose memory address starts at 0x1000. The configuration parameters of the local cache layer corresponding to CPU core A are: cache size: 32KB, cache line size: 64 bytes, cache replacement strategy: improved LRU algorithm, and write strategy: write-back strategy combined with write buffering technology.

[0056] For a read request: A program instruction requests CPU core A to read the 10th element of an array. The local cache layer sequentially searches the L1 / L2 / L3 caches to see if there is a cache line containing the 10th element of the array. If not, the local cache layer selects a non-dirty cache line for eviction based on the improved LRU algorithm. It then reads a 64-byte data block containing the 10th element of the array from main memory, loads it into the local cache, and updates the cache line's status to valid. The read data is then provided to CPU core A, and the cache line's status, such as its last used time, is updated, making it the most recently used cache line.

[0057] For a write operation request: The program instructions require CPU core A to modify the value of the 10th element of the array to 200. The local cache layer searches the corresponding cache line. If the 10 elements are already in the local cache and are valid, according to the write-back policy, the new value 200 is written to the corresponding location in the local cache, and the cache line status is updated to dirty. At the same time, the write operation is recorded in the write buffer. After accumulating a certain number of write operations, the write buffer writes the data back to the main memory in batches. CPU core A quickly completes the write operation and continues to execute subsequent tasks.

[0058] The process of maintaining the consistency of the local cache layer includes: the target local cache layer monitors the data access of each computing unit of the heterogeneous computing system to the target local cache based on the target cache consistency protocol, and determines whether to update the data copy in the target local cache according to the data access situation and the status information of each cache line of the target local cache; when the target data copy in the target local cache has expired, the target data copy is marked as invalid or a data update operation is performed.

[0059] In this embodiment, the target local cache layer monitors the access of other computing units or clusters to the data in the target local cache of the computing unit by monitoring the bus or using a directory method. For example, when the target local cache layer adopts a bus monitoring consistency protocol, when other components perform write operations on the target local cache data, the target local cache layer will receive a corresponding message notification. Then, based on the message content of the received monitoring message and the status of the data in the target local cache, it is decided whether the data copy in the target local cache needs to be updated. For example, if the data copy in the target local cache has expired, it will be marked as invalid or updated to a new data value to ensure that the data in the target local cache remains consistent with the data in the upper storage or other caches.

[0060] From the above, it can be seen that this embodiment implements cache management within a single computing unit through a local cache layer, and performs local data read and write operations and cache line status maintenance according to the cache strategy natively supported by the computing unit. This not only improves the efficiency of local data access and quickly provides the required data to the computing unit, but also more simply implements consistency maintenance of the local cache to ensure data consistency. The local cache layer can provide efficient and reliable local data storage and access services for computing units in heterogeneous computing systems, provide support for performance improvement of the entire system, and help improve the cache consistency maintenance accuracy of heterogeneous computing systems.

[0061] To further improve the effectiveness of cache consistency maintenance, based on the above embodiments, the present invention further provides an implementation method for dynamically adjusting the configuration parameters of the local cache layer, which may include the following:

[0062] Obtain read and write operation data and access delay data of the target computing unit collected by the target local cache layer; determine the cache hit rate of the target local cache layer based on the read and write operation data, and determine the average access delay based on the access delay data; and adjust the configuration parameters of the target local cache layer accordingly based on the cache hit rate and / or average access delay.

[0063] In this embodiment, a performance monitor can be built into the target local cache layer. This monitor collects real-time performance metrics such as cache hits, misses, read and write operations, and access latency. By analyzing and analyzing these metrics, key performance parameters such as cache hit rate and average access latency can be calculated. Based on the changing trends of these performance parameters and the set performance goals, the configuration parameters and management strategies of the local cache can be adaptively and dynamically adjusted. For example, if the read operation hit rate continues to decline, the read cache allocation ratio in the local cache can be increased, or the weight parameters in the cache replacement strategy can be adjusted to improve the protection of read cache lines.

[0064] Exemplarily, an exemplary method for calculating the cache hit rate may be: determining the read hit rate based on the number of read operations and the read operation cache hit rate; determining the write hit rate based on the number of write operations and the write operation cache hit rate; taking the sum of the read hit rate and the write hit rate as the total hit rate, taking the sum of the number of read operations and the number of write operations as the total number of operations, and determining the cache hit rate of the target local cache layer based on the total hit rate and the total number of operations. As an efficient cache hit rate calculation method, a cache hit rate calculation formula may be pre-stored, and the cache hit rate of the target local cache may be calculated by calling the cache hit rate calculation formula. The cache hit rate calculation formula may be expressed as:

[0065]

[0066] in, Indicates the cache hit ratio of the target local cache. Indicates the cache hit rate of read operations, Indicates the cache hit rate of write operations, Indicates the number of read operations, Indicates the number of write operations. The cache hit rate reflects the overall hit efficiency of the local cache layer when processing read and write operations. By improving the cache hit rate, the number of times the next level of storage is accessed can be reduced, thereby reducing access latency and improving system performance. For example, in an application scenario dominated by read operations, you can optimize , such as increasing the proportion of read cache space to improve overall performance.

[0067] For example, a data block in the target local cache , Number of reads , write times , the read hit rate of the data block , write hit rate , then its cache hit rate can be:

[0068]

[0069] From the above, it can be seen that this embodiment can dynamically capture the cache hit situation of different computing units by evaluating the cache hit rate of the local cache layer, facilitate dynamic adjustment of the local cache size according to actual operating conditions, optimize the cache replacement strategy, and improve local data access efficiency by optimizing the cache hit rate, so that the local cache layer can better meet the memory access requirements of the computing unit and improve the execution efficiency of the entire system.

[0070] The above embodiment does not impose any restrictions on how the target cluster cache layer processes data read and write operation requests. The target cluster cache layer is the cluster cache layer of the target cluster. The target cluster cache layer includes the target cluster shared memory and uses a target group consistency protocol that matches the memory consistency model of each computing unit in the cluster. For ease of description, for a computing unit in the cluster whose read and write operations do not hit its own local cache, this embodiment defines it as the first target unit, that is, the target cluster includes the first computing unit, the independent local cache of the first computing unit is the first local cache, and the first computing unit includes the first local cache layer, that is, the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs. The present invention also provides an exemplary implementation method, which may include the following content:

[0071] If the data operation request is a read request, when the first computing unit performs a memory read operation, when the to-be-read data to be read is not in the first local cache, the first local cache layer forwards its memory read request to the cluster cache layer to which it belongs. That is, when the target cluster cache layer receives the memory read request of the first computing unit, the target cluster cache layer searches for the existence of the to-be-read data in the target cluster shared memory. The to-be-read data is the data block that the first computing unit wants to read. If it exists, the to-be-read data is sent to the first local cache layer, and the cache line status information of the to-be-read data in the target cluster shared memory is updated; when the target cluster cache layer does not find the to-be-read data in the target cluster shared memory, it selects the first source shared cache line in a non-dirty state from the target cluster shared memory for elimination, and writes the to-be-read data read from the main memory or the next-level storage to the first destination shared cache line of the target cluster shared memory, sends the to-be-read data to the first local cache layer, and updates the status information of the first destination shared cache line. For example, when a local cache layer within a cluster of heterogeneous computing units initiates a memory read request and misses, the request is forwarded to the cluster cache layer. The cluster cache layer searches for the corresponding data block in the cluster shared cache based on the request. If a hit is found in the cluster shared cache, the data is immediately returned to the requesting local cache layer and the data block's state, such as its last used time, is updated in the cluster shared cache. If a miss is found, the cluster cache layer selects a cache line (the first source shared cache line) for eviction based on a pre-defined cache replacement strategy, such as an LRU variant suitable for the cluster memory model. If the evicted cache line is dirty, its data is first written back to main memory or other designated next-level storage locations, such as the relevant cache node in the global cache layer. The data to be read is then read from main memory or other next-level storage and loaded into the cluster shared cache. The state of the line (the first destination shared cache line) is updated to valid before the data is returned to the requesting local cache layer.

[0072] For data operation requests that are write requests, when the first computing unit performs a memory write operation, when the data to be written is not in the first local cache, the first local cache layer forwards its memory write request to the cluster cache layer to which it belongs, that is, when the target cluster cache layer receives the data operation request that is the memory write request of the first computing unit, the target cluster cache layer finds the data to be written in the target cluster shared memory, and the data to be written is the data block that the first computing unit wants to write, then the target cluster shared memory is updated according to the target group consistency protocol, and the second shared cache line state of the data to be written in the target cluster shared memory is updated to a dirty state; the second shared cache line is the cache occupied by the data block to be written in the target cluster memory. When the target cluster cache layer does not find the data to be written in the target cluster shared memory, it selects the second source shared cache line in a non-dirty state from the target cluster shared memory for elimination. The second source shared cache line is the cache line in the target cluster memory that is about to be eliminated, and the second destination shared cache line is the cache line in the target cluster memory occupied by the data block to be written in the target cluster memory. The data to be written is written to the second destination shared cache line in the target cluster shared memory, and the status information of the second destination shared cache line is updated. Based on the target group consistency protocol, the target group computing units associated with the data to be written in the target cluster are notified so that each target group computing unit updates the corresponding local cached data copy. For any cluster cache layer of a heterogeneous computing unit, when a computing unit initiates a memory write request and the data to be written does not hit in the local cache layer, or needs to be further written to the cluster shared cache layer according to the write strategy, the write request will be passed to the cluster cache layer. The cluster cache layer searches for the data to be written in the cluster shared cache. If a hit occurs, the data is updated according to the cluster's coherence protocol, such as the RC protocol, and the corresponding cache line is marked as dirty. The local cache layers of other relevant compute units are notified that the data block has expired or needs to be updated. If a miss occurs, the cache line is selected for eviction according to the cache replacement policy. If the cache line is dirty, it is first written to main memory or the next level of storage before being evicted. The data to be written is written to the cluster's shared cache, and the status information is updated and other affected local cache layers are notified.

[0073] To better understand the working principle of the cluster cache layer, this embodiment also provides an example to illustrate the read and write operation processing and cache consistency maintenance process of the cluster cache layer. This embodiment takes a cluster containing four isomorphic computing units, each of which adopts the RC memory model as an example. The computing units in the cluster frequently access a shared data structure, such as a 256-byte linked list with a memory address range from 0x2000 to 0x20FF. Computing unit 1 first reads the head of the linked list (address 0x2000-0x200F), then computing unit 2 modifies the middle part of the linked list (address 0x2030-0x203F), then computing unit 3 reads the tail of the linked list (address 0x20F0-0x20FF), and finally computing unit 4 modifies the head of the linked list. The data read and write operation process of each computing unit includes the following:

[0074] Compute unit 1 reads the linked list header as follows: The cluster cache layer searches the cluster shared cache for addresses 0x2000-0x200F. If a hit is found, the cluster cache layer reads the data block containing the address range from main memory. This data block is 16 bytes, requiring four cache lines, each 64 bytes. The data is loaded into the cluster shared cache and returned to the local cache layer of compute unit 1. At this point, the status of the corresponding cache line in the cluster shared cache is updated to valid.

[0075] Compute unit 2 modifies the middle portion of the linked list: The write request reaches the cluster cache layer, which searches for the cache line corresponding to addresses 0x2030-0x203F. If a match is found, the cluster cache layer updates the data according to the RC protocol and marks the cache line as dirty. Simultaneously, a notification is sent to the local cache layers of the other compute units, informing them of the updated data in the middle portion of the linked list. Upon receiving the notification, the other local cache layers invalidate or update their corresponding cache copies.

[0076] When computing unit 3 reads the tail of the linked list, the cluster cache layer searches for the data corresponding to addresses 0x20F0-0x20FF. If a hit is found, the data is read from main memory and loaded into the cluster shared cache. The data is then returned to computing unit 3's local cache layer, and the corresponding cache status is updated.

[0077] Compute unit 4 modifies the linked list header as follows: A write request arrives at the cluster cache layer. The cluster cache layer searches for the cache line corresponding to the linked list header at addresses 0x2000-0x200F. If a match is found and the cache line is valid, the data is updated and marked as dirty. An update notification is sent to other compute units to ensure that their cache copies are consistent with the data in the cluster's shared cache.

[0078] From the above, it can be seen that this embodiment targets a cluster of computing units with the same or similar memory consistency models. The cluster cache layer adopts a consistency protocol that matches the cluster memory model to maintain the cluster shared cache line state, ensuring the consistency of data within the cluster while reducing the access pressure on the global cache layer.

[0079] To further improve the effectiveness of cache consistency maintenance, based on the above embodiments, the present invention further provides an implementation method for dynamically adjusting configuration parameters of a target cluster cache layer. The target cluster cache layer belongs to the target cluster and includes the target cluster shared memory, which may include the following:

[0080] Obtain the average propagation delay of cache line invalidation messages and the average propagation delay of cache line update messages of the target cluster's shared memory; obtain the invalidation weight coefficient corresponding to the cache line invalidation messages and the update weight coefficient corresponding to the cache line update messages respectively; determine the intra-cluster cache consistency overhead of the target cluster based on the average propagation delay of the cache line invalidation messages, the average propagation delay of the cache line update messages, the invalidation weight coefficient and the update weight coefficient; adjust the group configuration parameters and / or the intra-cluster communication network structure of the target cluster based on the intra-cluster cache consistency overhead of the target cluster.

[0081] This embodiment quantifies the overhead generated by the cluster cache layer in maintaining consistency based on the invalidation and update operations of cache lines. The cluster configuration parameters and / or the intra-cluster communication network structure can be flexibly adjusted based on the intra-cluster cache consistency overhead combined with the actual application scenario. Cluster configuration parameters include invalidation operation frequency, update operation frequency, and intra-cluster communication network structure, such as whether the computing units in the cluster use a tree topology or a star network structure for communication. If data is shared frequently between computing units in the cluster and there are more write operations, the update operation will have a more significant impact on the overhead. The communication network structure within the cluster can be optimized in a targeted manner to reduce message propagation delay, or the consistency protocol can be optimized to reduce the frequency of update operations, thereby improving the performance of the cluster cache layer and making the entire system run more smoothly.

[0082] As an efficient implementation method, a calculation formula for the cache consistency overhead within the cluster can be pre-stored and called to calculate the cache consistency overhead within the target cluster. The calculation formula for the cache consistency overhead within the cluster can be expressed as: Indicates the cache consistency overhead within the target cluster, is the average propagation delay of cache line invalidation messages, is the average propagation delay of cache line update messages, They are the weight coefficients of failure and update operations in cluster consistency maintenance, respectively. Both can use the default value of 0.5, and can also be flexibly adjusted according to the actual cluster operation mode and application load.

[0083] As can be seen from the above, this embodiment utilizes a cluster cache layer to coordinate memory access requests from multiple computing units, maintain data consistency within the cluster, and dynamically adjust cache management policies, configuration parameters, or communication architectures based on cluster overhead to accommodate the access needs of different computing units. This effectively improves data sharing efficiency within the cluster, reduces frequent accesses to main memory, and enhances overall cluster performance. Furthermore, consistency maintenance based on the RC protocol ensures the correctness and reliability of data exchange between computing units.

[0084] Considering that heterogeneous computing systems operate in heterogeneous network environments, different clusters may have different communication interfaces and transmission protocols, which increases the difficulty of implementing communication mechanisms. To ensure that data operation requests can flow quickly and accurately between different layers without introducing excessive delays and overhead, the present invention comprehensively considers factors such as bandwidth, latency, and reliability, and implements efficient information transmission through the following embodiments, with highly compatible and adaptable communication:

[0085] A corresponding local message queue is set up in each local cache layer, a corresponding group message queue is set up in each cluster cache layer, and a global message queue is set up in the global cache layer; whenever each local cache layer interacts with the corresponding cluster cache layer for data or updates its status, a corresponding local message is generated, the local message is encapsulated in the corresponding local message format, and placed in the corresponding local message queue; whenever each cluster cache layer interacts with the corresponding local cache layer, other cluster cache layers or the global cache layer for data or updates its status, a corresponding group message is generated, the group message is encapsulated in the corresponding group message format, and placed in the corresponding group message queue; when the global cache layer interacts with each cluster cache layer for data or updates its status, a global message is generated, the global message is encapsulated in the corresponding global message format, and placed in the corresponding global message queue.

[0086] In this embodiment, when a local cache layer or a cluster cache layer needs to exchange data or update status with other layers, a corresponding message is generated. For example, when a cache miss occurs and the local cache layer needs to obtain data from a higher-level cache layer, it generates a read request message. After updating shared data, the cluster cache layer generates a data update notification message to inform the higher-level global cache layer and other relevant cluster cache layers. After the message is generated, it can be encapsulated according to a predefined message format, such as adding information such as the destination address, message type, data content, and a sending sequence number. The message is then placed in the message queue of that layer, awaiting delivery.

[0087] As can be seen from the above, this embodiment can efficiently and accurately transmit cache status information and memory access requests between different levels through message queues. When each level receives relevant requests or status change notifications, it processes and forwards them according to predefined rules, ensuring that information can flow accurately and promptly between levels without introducing excessive delays and overhead, and with highly compatible and adaptable communication.

[0088] Furthermore, based on the above embodiments, a message router can be set up for each cluster cache layer, and the message router can be used to monitor the local message queues of the computing units in the group and the message queues of the group to which they belong; according to the receiving address, message type and message tag of the target message, the receiving layer and sending time limit of the target message are determined, and the target message is sent according to the receiving layer and sending time limit; the receiving layer is the local cache layer, cluster cache layer or global cache layer that receives the target message.

[0089] Message tags include at least a timeliness tag, such as a priority tag and a priority processing tag. In this embodiment, a message router monitors messages in the message queue in real time and determines the next-hop layer or component to which the message should be sent based on the target address and type of the message. For example, when a local cache layer generates a writeback message and writes it to the local cache layer's message queue, the message router detects the message and, based on the target cluster address and message priority, sends it to the cluster cache layer's message queue for subsequent processing. For example, if the message is destined for the global cache layer, it is sent to the communication link leading to the global cache layer. If the message is intended for notification to other cluster cache layers, it is sent to the corresponding cluster cache layer based on cluster address resolution and routing tables. When the message router forwards messages, if multiple messages are present simultaneously, it prioritizes them based on their tags. For example, messages with high timeliness requirements, such as write requests and data update notifications, are forwarded first, while messages with relatively low latency sensitivity, such as read requests, are appropriately postponed, optimizing communication efficiency across the entire system.

[0090] As can be seen from the above, this embodiment adopts a communication architecture based on message queues and message routers. Each layer has a message queue for temporarily storing incoming and outgoing messages. The message router forwards messages to the appropriate layer or component based on the target address and type, establishing an efficient information delivery channel. This ensures that messages flow quickly and accurately between the local cache layer, the cluster cache layer, and the global cache layer, and ensures the timely exchange and sharing of data and status information between layers.

[0091] Furthermore, to ensure timely message processing, based on the above-mentioned embodiments, the present invention also provides a message reception and processing process: When the receiving layer receives the target message, it verifies the integrity and legitimacy of the target message and, if verification is successful, executes the target message. Furthermore, the message tag may include identifying information such as a sequence number to ensure that the messages are processed in the correct order at the receiving end. When the target message arrives at the target layer, it is read from the communication link and initially parsed to check for correct formatting and checksum matching to ensure legitimacy and integrity. If the target message is valid and complete, it is dispatched to the corresponding processing unit for execution based on the target message type. For example, a read request message is dispatched to the data read processing unit, and a data update notification message is dispatched to the cache update processing unit. The processing unit performs corresponding operations based on the message content, such as reading data and updating cache status, and generates a response message or a new notification message as necessary. Furthermore, to prevent messages from being missed or not received, a preset message confirmation period, such as 10 seconds, can be set. After sending a message, the sender can start a timer to wait for a confirmation message from the receiver. After successfully processing a message, the receiver sends a confirmation message to the sender, indicating that the message has been correctly received and processed. If no confirmation message is received from the receiving layer within the preset message confirmation period, the target message is resent, subject to the preset maximum number of retransmissions, until a confirmation message is received. If the sender does not receive a confirmation message within 10 seconds of the timer expiration, it is assumed that the message may have been lost or an error occurred during transmission and the message may be resent. To ensure communication efficiency, a preset maximum number of retransmissions, such as 6, can be pre-set. If a confirmation message is still not received after the message has been retransmitted 6 times, an error handling process can be triggered, such as reporting a system failure.

[0092] As can be seen from the above, this embodiment can ensure the reliability and order of message delivery through the message confirmation and retransmission mechanism. At the same time, by adding identification information such as serial numbers to messages, it ensures that the messages can be processed in the correct order at the receiver.

[0093] To further improve the effectiveness of cache consistency maintenance, based on the above embodiments, the present invention further provides an implementation method for dynamically adjusting communication configuration parameters, which may include the following:

[0094] Obtain the message retransmission rate of the heterogeneous computing system and the sending parameters of the target historical message; determine the message sending time of the target historical message according to the occupied capacity value of the target historical message and the communication link bandwidth; use the communication link delay as the message transmission delay of the target historical message, and determine the retransmission time according to the message retransmission rate, message sending time, message transmission delay and message processing time; determine the total inter-group communication time of a single message according to the message sending time, message transmission delay, message processing time and retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

[0095] The sending parameters include at least the occupied capacity value of the target historical message, the communication link bandwidth, the communication link delay, and the message processing time. For example, a heterogeneous computing system includes two clusters: cluster A and cluster B, each cluster has two computing units: computing unit 1 and computing unit 2. The communication link bandwidth is 10Gbps, which means that it can transmit bits. The communication link latency is 0.1 microseconds, including physical media transmission latency and interface processing latency. Cluster A's local cache layer sends a 128-byte (1024-bit) read request message to Cluster B's cluster cache layer to obtain a shared data block. The sender and receiver each take an average of 0.05 microseconds to process each message, and the message retransmission rate is 0.1%, meaning that on average, 1 in 1000 messages requires retransmission. The average time it takes to send a read request message from Cluster A's local cache layer to Cluster B's cluster cache layer and receive a response is: The message transmission delay is the communication link delay = 0.1 microseconds. Message processing time: 0.05 microseconds for the sender and 0.05 microseconds for the receiver, for a total of 0.1 microseconds. Retransmission time: The retransmission rate is 0.1%, with an average of 0.001 retransmissions. Each retransmission requires resending the message, which includes the sending time, transmission delay, and processing time. Therefore, the retransmission time = 0.001 × (0.1024 + 0.1 + 0.1) = 0.0003124 microseconds. Total time: Message sending time + Message transmission delay + Message processing time + Retransmission time = 0.1024 + 0.1 + 0.1 + 0.0003124 ≈ 0.3027124 microseconds. This is the average time it takes for a single message to be sent from cluster A to cluster B. If we consider the process of cluster B processing the read request and returning the response message, if the processing time is the same, the entire round-trip time is approximately 0.3027124×2≈0.6054248 microseconds.

[0096] As can be seen from the above, by calculating the total inter-group communication time, the performance of the inter-layer communication mechanism can be quantified. Communication parameters and strategies can be further optimized according to actual needs, such as increasing bandwidth, reducing latency, and optimizing message processing algorithms. This can improve the efficiency of inter-layer communication and enhance the performance of the entire hierarchical cache consistency architecture.

[0097] In order to further improve communication efficiency and reduce communication delay, this embodiment may also compress the messages to be transmitted or the data operation requests to reduce the amount of message transmission, which may include the following:

[0098] An initial dictionary is constructed based on a preset seed item, where each entry in the initial dictionary has a unique index and a corresponding string. A target string with the same characters as the input data is matched in the initial dictionary. The target string is combined with the next input data to form a new string, and the corresponding index value is added to the new string as a new entry. By continuously adding new entries to the initial dictionary, an adaptive message coding dictionary is generated. Each request character in the data operation request is sequentially input into the adaptive message coding dictionary, and the index information of each request character in the adaptive message coding dictionary is determined. The remaining characters in the data operation request that do not successfully match the adaptive message coding dictionary are determined, and the index information and remaining characters are used as the data to be encoded. The data to be encoded is encoded according to the frequency of occurrence of different characters in the data to be encoded, generating encoding sequence information, and sending the encoding sequence information in place of the data operation request. Upon receiving the encoded data operation information, the receiver uses the same adaptive message coding dictionary used to generate the data to be encoded, combined with the encoding process of the encoding sequence information, to decode the received information. Based on the index of the data to be encoded, the receiver searches for the corresponding string in the adaptive message coding dictionary and outputs it. The adaptive message coding dictionary is also updated based on the previously output string and the currently output string to maintain consistency with the encoding end.

[0099] The preset seed items can be common characters or phrases, such as the ASCII (American Standard Code for Information Interchange) character set, a computer encoding system based on the Latin alphabet. Using these common characters or phrases as seeds, a dynamic "dictionary" is constructed: an empty dictionary is created and assigned an initial seed item. Each entry in the dictionary contains a unique index and a corresponding string. New data is continuously input, and the initial dictionary is expanded based on the matches of the new data in the current dictionary. Specifically, for each character in the input message, starting from the current pointer position, the current dictionary is searched for the best matching string (i.e., an identical string). Once the string is found, the matching string plus the next character is added to the dictionary, and the current pointer is moved to the next character position. This process is repeated to adapt to new patterns in the message, resulting in an adaptive message coding dictionary that can be used to encode data operation requests. Of course, when using the adaptive message coding dictionary to compress requests from heterogeneous computing systems, the dictionary can also be continuously expanded using the above method, thereby increasing the probability that the input characters will be matched.

[0100] To facilitate understanding for those skilled in the art, this embodiment uses an adaptive message coding dictionary, D, containing the lowercase letters az in ASCII characters, indexed starting at 0. For example, a data operation request is abababa. The process of processing abababa using the adaptive coding dictionary to obtain the encoded data is shown in Table 1. Dictionary D contains 26 characters, and the index corresponding to each character starts at 0: a is indexed at 0, b is indexed at 1, and so on, up to z's index of 25. The dictionary also reserves space for adding new strings. The pointer initially points to the first character 'a' (pointer position 0). A search is performed in dictionary D for a matching string beginning with 'a'. Since the dictionary initially contains only a single character, the matching string is 'a'. The index corresponding to 'a' is output as 0. The matching string 'a' is then combined with the next character 'b' to form the new string "ab", which is then added to dictionary D. Since the initial dictionary contains 26 characters, the next available index is 26, so index 26 is added. The pointer moves to the next position, position 1. The current character it points to is 'b', and a matching string is searched for in dictionary D. Initially, 'b' exists in the dictionary, with index 1, so index 1 is output. 'b' is combined with the next character, 'a', to form "ba", which is added to the dictionary at index 27. The third character, 'a' (pointer position 2): "a" is found in the dictionary, and index 0 is output. 'a' is combined with the next character, 'b', to form "ab". This string already exists in the dictionary, so there's no need to add it again. "ab" has an index of 26. The pointer moves to the next position, position 3. The current character it points to is 'b', which already exists in dictionary D (index 1), so index 1 is output. 'b' is combined with the next character, 'a', to form "ba". This combination already exists in the dictionary, with index 27. The pointer moves to the next position, position 4. The current character it points to is 'a', which exists in dictionary D at index 0, so index 0 is output. Combine 'a' and the next character 'b' to form "ab", with an index of 26. The pointer continues to move to the next position, that is, to position 5. At this time, the current character pointed to by the pointer is 'b'. The matching string starting with 'b' in dictionary D is 'b', with an index of 1, so index 1 is output. Combine 'b' and 'a' to form "ba", with an index of 27. The pointer continues to move to the next position, that is, to position 6. At this time, the current character pointed to by the pointer is 'a'. There is 'a' in dictionary D, with an index of 0, so index 0 is output. The output data sequence to be encoded is: [0, 1, 0, 1, 0, 1, 0]. All characters in the data operation request of this embodiment match the string, so there are no remaining characters.

[0101] Table 1 Adaptive dictionary encoding table

[0102]

[0103] The above embodiment does not limit the process of encoding the data to be encoded and generating the encoding sequence information. The present invention also provides an exemplary encoding method, which may include the following:

[0104] The invention relates to a method for encoding a character tree. The method comprises the following steps: obtaining different characters of the data to be encoded and their frequencies of occurrence in the data to be encoded as character information to be encoded; treating the different characters as nodes of a tree structure, selecting the first and second characters with the lowest frequencies as leaf nodes, merging the first and second characters into the parent node of the leaf node, wherein the frequency of the parent node is the sum of the frequencies of the first and second characters; selecting the third character with the lowest frequency from the remaining characters in the character information to be encoded, treating the third character and the parent node as child nodes at the same level, merging the third character and the parent node to obtain the parent node of the two, and repeating the process until there are no remaining characters in the character information to be encoded, thereby obtaining a tree to be encoded; assigning a first code value to each left child node of the tree to be encoded, and a second code value to each right child node, starting from the root node, recursively generating a code sequence for each character of the character information to be encoded based on the code value of each node, and combining the code sequences of each character of the character information to be encoded into code sequence information. When encoding and decoding the code sequence information, a code table can be constructed to facilitate encoding. The code table includes at least items such as character, frequency of occurrence, and code sequence, and their corresponding contents, and the generated code table can be used to encode the characters. During decoding, starting from the root node of the tree to be encoded, the tree is traversed bit by bit according to the input encoding sequence information until a leaf node is reached and the corresponding character is output.

[0105] In this embodiment, the characters of the data to be encoded are counted, and the frequency of occurrence of each character is calculated. Taking the data to be encoded as [0, 1, 0, 1, 0, 1, 0] as an example, the frequencies of occurrence of 1 and 0 in the data to be encoded are counted respectively, the frequency of character 0 is 4, and the frequency of character 1 is 3. The tree to be encoded is constructed according to the frequency: the two nodes with the lowest frequency are merged into a parent node, and the frequency of the parent node is the sum of the frequencies of the two child nodes. Repeat this process until only one root node remains. Starting from the root node, the first code value, such as "0", is assigned to the left child node, and the second code value, such as "1", is assigned to the right child node. The encoding sequence of each character is recursively generated to achieve compression of a single character. Of course, the first code value and the second code value can also use other different values ​​or characters, which does not affect the implementation of the present invention. Taking the above example again, the data to be encoded only has characters 0 and 1, and the corresponding tree to be encoded is as follows Figure 3As shown in Table 2, character 0 is a left leaf node and can be assigned a first code value of 0. Character 1 is a right leaf node and can be assigned a second code value of 1. The path from the root node to character 0 is ->0, so the encoding sequence of character 0 is 0. The path from the root node to character 1 is ->1, so the encoding sequence of character 1 is 1. As shown in Table 2, the encoding of each character above is used to replace the output sequence [0, 1, 0, 1, 0, 1, 0] after adaptive dictionary encoding. The encoding sequence information remains [0, 1, 0, 1, 0, 1, 0].

[0106] Table 2 Coding table

[0107]

[0108] For ease of understanding, this embodiment also provides another more complex embodiment. The data to be encoded is [0, 1, 17, 0, 2, 0, 3, 26], and the characters that appear are 3, 17, 2, 1, 26, and 0. The frequency of occurrence of each character is counted, and the lowest frequencies, 3 and 17, are selected as the two nodes at the bottom. These two nodes are merged into a parent node, and the frequency of the parent node is the sum of the frequencies of the two child nodes. This process is repeated until only one root node remains, and the result is as follows: Figure 4 The tree to be encoded shown starts from the root node, assigns 0 to the left child node and 1 to the right child node. The encoding sequence recursively generated for each character is as follows:

[0109] The path from the root node to character 3 is: 0->0->0->0->0->0, so the encoding sequence for character 3 is 00000. The path from the root node to character 17 is: 0->0->0->0->1, so the encoding sequence for character 17 is 00001. The path from the root node to character 2 is: 0->0->0->1, so the encoding sequence for character 2 is 0001. The path from the root node to character 1 is: 0->0->1, so the encoding sequence for character 1 is 001. The path from the root node to character 26 is: 0->1, so the encoding sequence for character 26 is 01. The path from the root node to character 0 is: 1, so the encoding sequence for character 0 is 1. After replacing the corresponding characters in the data to be encoded [0, 1, 17, 0, 2, 0, 3, 26] after adaptive dictionary encoding with the encoding sequences of each of the above characters, the final encoding sequence information is: 0001000010000100000001.

[0110] In order to ensure that the index can cover the entire dictionary, based on the above embodiments, the present invention also provides a method for determining the index coding length: obtaining the spatial capacity value of the adaptive message coding dictionary; calculating the logarithm of the spatial capacity value with a preset number as the base, and taking the rounded-up value of the logarithm as the minimum index coding length of the adaptive message coding dictionary.

[0111] As an efficient calculation method, the index coding length calculation formula can be pre-stored and the minimum index coding length of the adaptive message coding dictionary can be calculated by calling the index coding length calculation formula. The index coding length calculation formula can be expressed as:

[0112]

[0113] Among them, the index encoding length Depends on the dictionary size , To round up, in order to ensure that the index can cover the entire dictionary, the encoding length is usually set to meet The smallest integer.

[0114] In order to verify the effectiveness of the encoding method provided by the present invention, this embodiment takes the message abababa as an example. The size of abababa is 7 bytes, and each character is 1 byte. The corresponding compression ratio is calculated using a preset compression ratio calculation formula. The compression ratio (CR) is defined as the ratio of the original message size to the compressed message size. The compression ratio calculation formula can be expressed as:

[0115]

[0116] in, Indicates the original message size in bytes. The size of the compressed message in bytes, CR represents the compression ratio. The encoding sequence information corresponding to this string is [0, 1, 0, 1, 0, 1, 0], with a length of 7 bits. 8 bits correspond to 1 byte, which is 1 byte. Dictionary index encoding length bits, where 27 represents the maximum index of the dictionary. Then the total length of the dictionary index code is equal to 7*5=35 bits, that is, 5 bytes. Therefore, the total bytes after compression are 1 (coding sequence information) + 5 (dictionary) = 6 bytes. Therefore, the compression rate is Indicates the amount of compressed information.

[0117] As can be seen from the above, this embodiment uses an adaptive message coding dictionary to efficiently compress repeated characters in data operation requests. By replacing repeated character strings with indexes, the amount of data is greatly reduced. At the same time, the remaining single characters are encoded in combination with the encoding method for the coding tree to achieve a higher compression rate. Variable-length codes are generated based on the frequency of character occurrence, and high-frequency characters correspond to short codes, further reducing the amount of data. The combination of the two can achieve efficient compression of data operation requests, effectively reducing the amount of communication data, thereby reducing communication delays and improving system efficiency. In addition, the adaptive message coding dictionary can dynamically update content, effectively capture newly emerging repeated patterns, and effectively improve the compression rate.

[0118] The above embodiment does not impose any limitation on the protocol conversion of the consistency protocol adaptation layer. Based on the above embodiment, the present invention also proposes an exemplary protocol format conversion method, which may include the following contents:

[0119] A mapping rule table from each group consistency protocol to the global consistency protocol is established in advance based on the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; when a new group consistency protocol appears in the heterogeneous computing system, a new mapping rule from the new group consistency protocol to the global consistency protocol is determined, and the mapping rule table is updated accordingly through the interface; whenever a data operation request is received out of order, a unique message sequence number is assigned to the data operation request while caching the data operation request; the data operation request is converted into the protocol format in sequence according to the order of each message sequence number and on the basis of retaining the data dependency information.

[0120] In this embodiment, the coherence protocol adaptation layer needs to define protocol mapping rules and perform message conversion to complete protocol conversion between heterogeneous clusters. The protocol mapping rules are based on establishing a mapping rule table from various cluster-local coherence protocols to the global coherence protocol. For example, for a CPU cluster using the MSI (Modified, Shared, Invalid) protocol and a GPU cluster using the Firefly protocol, corresponding rules are defined to map the state transitions and message types in the MSI protocol to the format specified by the global protocol. Update messages and response messages in the Firefly protocol are also converted accordingly. Message conversion involves converting messages between the cluster cache layer and the global cache layer. When the cluster cache layer sends a message to the global cache layer, the message content is parsed, converted, and encapsulated according to predefined mapping rules to generate a message format that meets the requirements of the global protocol. Conversely, when the global cache layer sends a message to the cluster cache layer, the reverse conversion is also performed to ensure that the message is correctly understood and processed by the cluster. When the cluster cache layer sends a message to the global cache layer: After receiving the local protocol message, the message type, content, and destination address are first parsed. Based on predefined mapping rules, the message format, status code, and operation instructions are converted to conform to the global protocol. The message is repackaged to generate a new message conforming to the global protocol format and sent to the global cache layer. When the global cache layer sends a message to the cluster cache layer, it parses the global protocol message, extracts key information, and applies the corresponding inverse mapping rules based on the local protocol type of the target cluster to convert the message into the local protocol message format understood by the cluster. The converted message is then sent to the cluster cache layer to ensure that the cluster can process it correctly.

[0121] During the operation of heterogeneous computing systems, when new cluster types or protocols appear, users can add new protocols and update existing protocols through the interface of the consistency protocol adaptation layer.

[0122] Furthermore, to ensure consistent ordering during message conversion, each message is assigned a unique sequence number, and conversion and delivery are performed in sequence. Out-of-order messages are buffered until they can be processed in the correct order. For operations that rely on a specific order, such as write-after-write ordering, this dependency information is retained and passed during message conversion, ensuring that operations are executed in the correct order at both the global and cluster cache layers.

[0123] To help those skilled in the art better understand the technical solution of this embodiment, this embodiment also provides an illustrative example. In this example, a cluster uses the MSI protocol, while another cluster uses the Firefly protocol. The MSI protocol's "Modified" state needs to be converted to the global protocol's "Exclusive" state, and Firefly protocol update messages need to be converted to global protocol write messages. The MSI protocol state mapping rules are: Modified → Exclusive, Shared → Shared, Invalid → Invalid; the Firefly protocol message mapping rules are: Update message → Write message. The data value in the Update message is mapped to the data value in the Write message, and the target address remains unchanged. Cluster A (MSI protocol) sends a cache line update message with the Modified state to the global cache layer. The original MSI message is: {Type:Update, State:Modified, Address:0x1000, Data:0xABC}; the converted global message is: {Type:Update, State:Exclusive, Address:0x1000, Data:0xABC}. Cluster B (Firefly protocol) sends an update message to the global cache layer: Original Firefly message: {Type:Update, Address: 0x2000, Data: 0xDEF}, converted global message: {Type:Write, Address: 0x2000, Data: 0xDEF}. The global cache layer sends a write message to cluster A: Original global message: {Type:Write, Address: 0x1000, Data: 0x123}, converted MSI message: {Type:Write, State:Modified, Address: 0x1000, Data: 0x123}. The global cache layer sends a read message to cluster B: Original global message: {Type:Read, Address: 0x2000}, converted Firefly message: {Type:Read, Address: 0x2000}. When cluster A sends two consecutive write operation messages to the global cache layer, the sequence number of message 1 is 1 and the sequence number of message 2 is 2, the messages can be represented as: Message 1: {Seq: 1, Type: Write, Address: 0x1000, Data: 0xABC}, Message 2: {Seq: 2, Type: Write, Address: 0x1000, Data: 0xDEF}.Due to network latency, when Message 2 arrives at the consistency protocol adaptation layer before Message 1, the consistency protocol adaptation layer's message buffer temporarily stores Message 2 and waits for Message 1 to arrive. When Message 1 arrives, the consistency protocol adaptation layer converts and sends the messages in sequence (1 → 2), ensuring that the global cache layer performs write operations in the correct order.

[0124] As can be seen from the above, the protocol mapping rules and message conversion processing of the consistency protocol adaptation layer of this embodiment can achieve deep integration and seamless collaboration between multiple different memory consistency models. At runtime, it can identify the message type of the new protocol and apply the corresponding mapping rules for conversion, thereby achieving dynamic expansion and compatibility maintenance of the system. This makes the system compatible with a wider variety of heterogeneous computing units and provides more efficient and reliable consistency guarantees between different clusters.

[0125] In order to further optimize the heterogeneous computing system, it is possible to make arbitration decisions quickly and accurately under real-time changing system load conditions, reasonably allocate memory access rights, and achieve optimal overall system performance, while avoiding excessive computing overhead due to the complexity of the arbitration algorithm itself. Based on the above embodiment, the data processing operations of the global cache layer can also be optimized, which may include the following: monitoring memory access requests of each cluster shared cache layer; dynamically adjusting the access rights and access order of each memory access request based on the load information of the heterogeneous computing system.

[0126] In this embodiment, the global shared cache serves as the final consistency guarantee node for data in the entire heterogeneous system. On the basis of adopting a strong consistency protocol that comprehensively considers the characteristics of various memory models for management, it can also monitor the memory access request queues of each cluster in real time, and dynamically arbitrate and coordinate the memory access requests of different clusters based on factors such as the current system load status, the performance characteristics of each cluster, and the priority of memory access. For example, according to preset scheduling algorithms such as weighted round-robin and priority scheduling, the received requests can be sorted and selected, and memory access rights can be reasonably allocated. For example, for clusters where real-time computing tasks with higher performance requirements are located, higher access priority is given to them. At the same time, the access rights allocation of each cluster is dynamically adjusted according to the system load to avoid resource competition and deadlock problems, thereby ensuring efficient utilization of global memory resources.

[0127] From the above, it can be seen that on the basis of reducing the cache consistency overhead on a global scale through a hierarchical cache management method, the dynamic arbitration and coordination of this embodiment can optimize memory access according to real-time conditions, thereby improving the overall performance and response speed of the system, optimizing the overall performance of the system, and ensuring the correctness and consistency of data on a global scale.

[0128] It should be noted that there is no strict order in which the steps in the present invention are performed. As long as they conform to a logical order, the steps can be performed simultaneously or in a predetermined order. Figure 2 This is just an illustrative example and does not necessarily mean that this is the only execution order. Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0129] The present invention also provides a corresponding device for the cache consistency maintenance method, which further makes the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The cache consistency maintenance device provided by the present invention is introduced below. The device is used to implement the cache consistency maintenance method provided by the present invention. In this embodiment, the cache consistency maintenance device may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to complete the cache consistency maintenance method disclosed in Example 1. The program module referred to in this embodiment refers to a series of computer program instruction segments that can perform specific functions, which is more suitable for describing the execution process of the cache consistency maintenance device in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The cache consistency maintenance device described below and the cache consistency maintenance method described above can be referenced to each other.

[0130] From the perspective of functional modules, see Figure 5 , Figure 5 This is a structural diagram of a cache consistency maintenance device provided in this embodiment in a specific implementation manner. The device may include:

[0131] The request forwarding module 501 is used to determine whether the data operation request is forwarded to the local cache layer, cluster cache layer or global cache layer when the heterogeneous computing system detects a data operation request; the computing units that meet the same similar conditions of the memory consistency model in the heterogeneous computing system constitute a cluster, and each cluster includes a cluster cache layer that constructs the local cache of the computing units in the cluster into a cluster shared cache.

[0132] The protocol conversion module 502 is used to convert the protocol format of the data operation request according to the mapping relationship between the group consistency protocol and the global consistency protocol through the consistency protocol adaptation layer with an interface when the sending and receiving parties are the cluster cache layer and the global cache layer.

[0133] The request processing module 503 is used to perform corresponding operations based on the memory consistency model of the computing unit to which it belongs and maintain the consistency of its local cache when forwarded to the target local cache layer; perform corresponding operations based on the target group consistency protocol that matches the memory consistency model of each computing unit in the cluster and maintain the consistency of its cluster shared cache when forwarded to the global cache layer, and maintain the consistency of each cluster shared cache based on the global consistency protocol when forwarded to the global cache layer.

[0134] Exemplarily, in some implementation schemes of this embodiment, the above-mentioned request processing module 503 can also be used for: the target local cache layer is the local cache layer of the target computing unit, the data operation request is the memory read request of the target computing unit, the target computing unit includes a target local cache, and when the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache line where the target read data block is located, and sent to the target computing unit; when the target local cache layer does not find the target read data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the target read data block read from the main memory or the next level storage is written to the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

[0135] Exemplarily, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used for: the target local cache layer is the local cache layer of the target computing unit, the data operation request is the memory write request of the target computing unit, and the target computing unit includes a target local cache. When the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, the target write data block is written to the target local cache according to the data write strategy and the tag and index of the target cache line where the target write data block is located, and the status information of the target cache line is updated to a dirty state; when the target local cache layer does not find the target write data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the status information of the destination cache line corresponding to the target write data block written to the target local cache is updated to a to-be-written state, the target write data block is written to the destination cache line, and the status information of the destination cache line is updated to a dirty state.

[0136] Exemplarily, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used for: the target local cache layer is the local cache layer of the target computing unit, the target computing unit includes a target local cache, and uses a target cache consistency protocol, the target local cache layer monitors the data access of each computing unit of the heterogeneous computing system to the target local cache based on the target cache consistency protocol, and determines whether to update the data copy in the target local cache based on the data access status and the status information of each cache line of the target local cache; when the target data copy in the target local cache has expired, the target data copy is marked as invalid or a data update operation is performed.

[0137] Illustratively, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used for: the target local cache layer is the local cache layer of the target computing unit, and the target computing unit includes a target local cache to obtain the read and write operation data and access delay data of the target computing unit collected by the target local cache layer; determine the cache hit rate of the target local cache layer based on the read and write operation data, and determine the average access delay based on the access delay data; and adjust the configuration parameters of the target local cache layer accordingly based on the cache hit rate and / or average access delay.

[0138] As an exemplary implementation of the above embodiment, the above request processing module 503 can also be further used to: determine the read hit rate based on the number of read operations and the read operation cache hit rate; determine the write hit rate based on the number of write operations and the write operation cache hit rate; take the sum of the read hit rate and the write hit rate as the total hit rate, take the sum of the number of read operations and the number of write operations as the total number of operations, and determine the cache hit rate of the target local cache layer based on the total hit rate and the total number of operations.

[0139] Exemplarily, in some other implementations of this embodiment, the request processing module 503 may also be used for: when the data operation request is a read request of the first computing unit and the data to be read is not in the first local cache, the first local cache layer forwards the read request to the corresponding cluster cache layer, the target cluster cache layer includes the target cluster shared memory, and when the target cluster cache layer finds the data to be read in the target cluster shared memory, the data to be read is sent to the first local cache layer, and the cache line status information of the data to be read in the target cluster shared memory is updated; when the target cluster cache layer does not find the data to be read in the target cluster shared memory, the first source shared cache line in the non-dirty state is selected from the target cluster shared memory for elimination, and the data to be read read from the main memory or the next-level storage is written to the first destination shared cache line of the target cluster shared memory, the data to be read is sent to the first local cache layer, and the status information of the first destination shared cache line is updated; wherein, the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

[0140] Exemplarily, in some other implementations of this embodiment, the request processing module 503 may also be used for: when the data operation request is a read request forwarded by the first local cache layer to the corresponding cluster cache layer of the first computing unit and the data to be written is not in the first local cache, the target cluster cache layer includes a target cluster shared memory, and when the target cluster cache layer finds the data to be written in the target cluster shared memory, it updates the target cluster shared memory according to the target group consistency protocol, and updates the state of the second shared cache line of the data to be written in the target cluster shared memory to a dirty state; when the target cluster cache layer does not find the data to be written in the target cluster shared memory, it selects a second source shared cache line in a non-dirty state from the target cluster shared memory for elimination, writes the data to be written to the second destination shared cache line of the target cluster shared memory, updates the state information of the second destination shared cache line, and notifies the target group computing units associated with the data to be written in the target cluster based on the target group consistency protocol, so that each target group computing unit updates the corresponding local cached data copy; wherein, the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

[0141] Exemplarily, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used for: the target cluster cache layer belongs to the target cluster and includes the target cluster shared memory, obtaining the average propagation delay of the cache line invalidation message and the average propagation delay of the cache line update message of the target cluster shared memory; obtaining the invalidation weight coefficient corresponding to the cache line invalidation message and the update weight coefficient corresponding to the cache line update message respectively; determining the intra-cluster cache consistency overhead of the target cluster based on the average propagation delay of the cache line invalidation message, the average propagation delay of the cache line update message, the invalidation weight coefficient and the update weight coefficient; adjusting the group configuration parameters and / or the intra-cluster communication network structure of the target cluster based on the intra-cluster cache consistency overhead of the target cluster.

[0142] Exemplarily, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used to: set up a corresponding local message queue in each local cache layer, set up a corresponding group message queue in each cluster cache layer, and set up a global message queue in the global cache layer; whenever each local cache layer interacts with the corresponding cluster cache layer for data or updates status, a corresponding local message is generated, the local message is encapsulated in the corresponding local message format, and placed in the corresponding local message queue; whenever each cluster cache layer interacts with the corresponding local cache layer, other cluster cache layers or global cache layers for data or updates status, a corresponding group message is generated, the group message is encapsulated in the corresponding group message format, and placed in the corresponding group message queue; when the global cache layer interacts with each cluster cache layer for data or updates status, a global message is generated, the global message is encapsulated in the corresponding global message format, and placed in the corresponding global message queue.

[0143] As an exemplary implementation of the above embodiment, the above request processing module 503 can also be used to: use a message router to monitor the local message queue of the computing unit in the group to which it belongs and the message queue of the group to which it belongs; determine the receiving layer and sending timeliness of the target message according to the receiving address, message type and message tag of the target message, and send the target message according to the receiving layer and sending timeliness; the receiving layer is the local cache layer or cluster cache layer or global cache layer that receives the target message, and the message tag includes at least a timeliness mark.

[0144] As an exemplary implementation of the above embodiment, the above request processing module 503 can also be used for: when the receiving layer receives the target message, verifying the integrity and legitimacy of the target message, and executing the target message after the verification is passed; when the message reception confirmation information of the receiving layer is not received within the preset message confirmation time period, the target message is resent under the condition that the preset maximum number of retransmissions is met until the message reception confirmation information is received.

[0145] Exemplarily, in some other implementations of this embodiment, the above-mentioned request processing module 503 can also be used to: obtain the message retransmission rate of the heterogeneous computing system and the sending parameters of the target historical message; the sending parameters include at least the occupied capacity value, communication link bandwidth, communication link delay, and message processing time of the target historical message; determine the message sending time of the target historical message based on the occupied capacity value and communication link bandwidth of the target historical message; use the communication link delay as the message transmission delay of the target historical message, and determine the retransmission time based on the message retransmission rate, message sending time, message transmission delay, and message processing time; determine the total inter-group communication time of a single message based on the message sending time, message transmission delay, message processing time, and retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

[0146] Illustratively, in some other implementations of this embodiment, the request processing module 503 may also be used to construct an initial dictionary based on a preset seed item, wherein each entry of the initial dictionary has a unique index and a corresponding string; matching a target string having the same characters as the input data in the initial dictionary, forming a new string with the target string and the next input data, and adding a corresponding index value to the new string as a new entry, thereby generating an adaptive message coding dictionary by continuously adding new entries to the initial dictionary; inputting each request character of the data operation request into the adaptive message coding dictionary in sequence, determining the index information of each request character in the adaptive message coding dictionary, and determining the remaining characters in the data operation request that have not been successfully matched with the adaptive message coding dictionary, and using the index information and the remaining characters as the data to be encoded; encoding the data to be encoded according to the frequency of occurrence of different characters in the data to be encoded, generating encoding sequence information, and sending the encoding sequence information instead of the data operation request.

[0147] As an exemplary implementation of the above embodiment, the request processing module 503 can also be used to: obtain different characters of the data to be encoded and the frequency of their occurrence in the data to be encoded as character information to be encoded; use different characters as nodes of a tree structure, and select the first and second characters with the lowest frequencies as leaf nodes, and merge the first and second characters into the parent node of the leaf node, where the frequency of the parent node is the sum of the frequencies of the first and second characters; select the third character with the lowest frequency from the remaining characters in the character information to be encoded, use the third character and the parent node as child nodes at the same level, merge the third character and the parent node to obtain their parent node, and repeat until there are no remaining characters in the character information to be encoded, and obtain a tree to be encoded; assign a first code value to each left child node of the tree to be encoded, and assign a second code value to each right child node, and starting from the root node, recursively generate a coding sequence for each character of the character information to be encoded based on the code value of each node, and combine the coding sequences of each character of the character information to be encoded into coding sequence information.

[0148] As another exemplary implementation of the above embodiment, the above request processing module 503 can also be used to: obtain the spatial capacity value of the adaptive message coding dictionary; calculate the logarithm of the spatial capacity value with a preset number as the base, and use the rounded-up value of the logarithm as the minimum index coding length of the adaptive message coding dictionary.

[0149] Exemplarily, in some other implementations of this embodiment, the above-mentioned protocol conversion module 502 can also be used to: establish a mapping rule table from each group consistency protocol to the global consistency protocol in advance based on the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; when a new group consistency protocol appears in the heterogeneous computing system, determine a new mapping rule from the new group consistency protocol to the global consistency protocol, and update the mapping rule table accordingly through the interface; whenever an out-of-order data operation request is received, a unique message sequence number is assigned to the data operation request while caching the data operation request; the data operation request is converted into the protocol format in sequence according to the order of each message sequence number and on the basis of retaining the data dependency information.

[0150] Exemplarily, in some other implementations of this embodiment, the request processing module 503 may also be used to: monitor memory access requests of each cluster shared cache layer; and dynamically adjust the access rights and access order of each memory access request based on the load information of the heterogeneous computing system.

[0151] The cache consistency maintenance device mentioned above is described from the perspective of functional modules. Furthermore, the present invention also provides an electronic device, which is described from the perspective of hardware. Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention in one implementation manner. The electronic device includes a memory 601 and a processor 602. The memory 601 stores a computer program, and the processor 602 is configured to run the computer program to perform the steps of any of the above cache consistency maintenance method embodiments.

[0152] An embodiment of the present application further provides a non-volatile storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above cache consistency maintenance method embodiments when running.

[0153] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0154] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above cache consistency maintenance method embodiments are implemented.

[0155] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned cache consistency maintenance method embodiments are implemented.

[0156] Finally, the present invention also provides a heterogeneous computing system, see Figure 7The heterogeneous computing system includes at least a first computing node 701, a second computing node 702 and a cache consistency controller 703. The cache consistency controller 703 is connected to the first computing node 701 and the second computing node 702. The first computing node 701 and the second computing node 702 include at least two computing units. The cache consistency controller 703 is used to implement the cache consistency maintenance method described in any of the above cache consistency maintenance method embodiments when executing a computer program. Among them, the first computing node 701 includes at least a first computing unit and a second computing unit, and the second computing node 702 includes at least a third computing unit and a fourth computing unit; the memory consistency models of the first computing unit, the second computing unit, and the third computing unit meet the same similar conditions of the memory consistency model; wherein the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit all include a local cache layer for managing their respective local caches, the first computing unit, the second computing unit, and the third computing unit constitute a first cluster, the first cluster includes a cluster cache layer that constructs the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster shared cache, and the first computing unit, the second computing unit, and the third computing unit are connected to the cluster shared cache through a high-speed Internet network, and the shared caches of each cluster are maintained consistent through the global cache layer.

[0157] For example, Figure 8As shown, the cache consistency controller 603 may include a local cache layer component, a cluster cache layer component, an inter-layer communication component, a consistency protocol adaptation layer component, and a global cache layer component. The local cache layer component is used to manage the cache within a single computing unit, perform local data read and write operations and cache line status maintenance according to the cache strategy natively supported by the computing unit, improve local data access efficiency by optimizing cache hit rate, etc., and quickly provide the required data to the computing unit. The cluster cache layer component maintains the cluster shared cache for computing unit clusters with the same or similar memory consistency models, and uses a consistency protocol that matches the cluster memory model to manage cache line status, ensuring the consistency of data within the cluster while reducing access pressure on the global cache layer. The consistency protocol adaptation layer component is used to implement conversion between local consistency protocols and global consistency protocols of different clusters. Through protocol mapping rules and message conversion processing units, it ensures that different clusters can collaborate seamlessly, solves the problem of protocol incompatibility between heterogeneous clusters, and provides basic support for global cache consistency maintenance. The global cache layer component serves as the node that ensures the final consistency of data in the entire heterogeneous system. It uses a strong consistency protocol to manage the global shared cache and sets a dynamic arbitrator to intelligently arbitrate and coordinate memory access requests of each cluster, optimize the overall performance of the system, and ensure the correctness and consistency of data globally. The inter-layer communication component establishes an efficient information transmission channel to ensure that messages flow quickly and accurately between the local cache layer, cluster cache layer, and global cache layer. Through message queues, message routers and other means as well as corresponding communication protocols, it ensures timely interaction and sharing of data and status information between layers. These five components work together to complete the cache consistency maintenance process of the heterogeneous system. For example, Figure 9 As shown in the figure, there are four groups, each of which uses the same cache consistency maintenance protocol. Therefore, each computing node within each group is equipped with a local cache layer component responsible for maintaining the node's cache status. At the same time, a unified cluster cache component is set up within each group to synchronize messages with other groups. Between groups, an efficient inter-layer communication component is required, and this communication component is used to send various types of information adapted by the inter-group consistency protocol. At the global level, a global cache component is set up to uniformly monitor and adjust the cache consistency maintenance process between groups within the system.

[0158] The above is a detailed introduction to a heterogeneous computing system and its cache consistency maintenance method, device, electronic device, non-volatile storage medium, and computer program product provided by the present invention. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. Whether the units and algorithm steps of each example described in each disclosed embodiment are executed in electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, and such implementation should not be considered to exceed the scope of the present invention. Without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A cache consistency maintenance method, characterized in that: include: When the heterogeneous computing system detects a data operation request, determining whether the data operation request is forwarded to a local cache layer, a cluster cache layer, or a global cache layer; The computing units in the heterogeneous computing system that meet the same or similar conditions of the memory consistency model form a cluster, and each cluster includes a cluster cache layer that constructs the local caches of the computing units in the cluster into a cluster shared cache; When the sender is a cluster cache layer and the receiver is a global cache layer, or the sender is a global cache layer and the receiver is a cluster cache layer, or both the sender and the receiver are cluster cache layers, the data operation request is converted into a protocol format according to the mapping relationship between the group consistency protocol and the global consistency protocol through a consistency protocol adaptation layer with an interface; When forwarded to the target local cache layer, the corresponding operation is performed based on the memory consistency model of the computing unit to which it belongs, and the consistency of its local cache is maintained; When forwarded to the target cluster cache layer, the corresponding operation is performed based on the target cluster consistency protocol that matches the memory consistency model of each computing unit in the cluster, and the consistency of its cluster shared cache is maintained; When forwarded to the global cache layer, the consistency of the shared cache of each cluster is maintained based on the global consistency protocol; Among them, when the data operation request is a read request forwarded by the first local cache layer to the cluster cache layer to which it belongs when the to-be-read data of the first computing unit is not in the first local cache; the target cluster cache layer includes the target cluster shared memory; when the target cluster cache layer finds the to-be-read data in the target cluster shared memory, the to-be-read data is sent to the first local cache layer, and the cache line status information of the to-be-read data in the target cluster shared memory is updated; when the target cluster cache layer does not find the to-be-read data in the target cluster shared memory, the first source shared cache line in a non-dirty state is selected from the target cluster shared memory for elimination, and the to-be-read data read from the main memory or the next-level storage is written to the first destination shared cache line of the target cluster shared memory, the to-be-read data is sent to the first local cache layer, and the status information of the first destination shared cache line is updated; the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

2. The cache consistency maintenance method according to claim 1, wherein: The target local cache layer is a local cache layer of a target computing unit, the data operation request is a memory read request of the target computing unit, the target computing unit includes a target local cache, and a corresponding operation is performed based on a memory consistency model of the computing unit to which it belongs, including: When the target local cache layer finds the target read data block corresponding to the memory read request in the target local cache, the target read data block is read from the target local cache according to the tag and index of the target cache row where the target read data block is located, and is sent to the target computing unit; When the target local cache layer does not find the target read data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the target read data block read from the main memory or the next level storage is written to the destination cache line of the target local cache, the target read data block is sent to the target computing unit, and the status information of the destination cache line is updated.

3. The cache consistency maintenance method according to claim 1, wherein: The target local cache layer is a local cache layer of a target computing unit, the data operation request is a memory write request of the target computing unit, the target computing unit includes a target local cache, and a corresponding operation is performed based on a memory consistency model of the computing unit to which it belongs, including: When the target local cache layer finds the target write data block corresponding to the memory write request in the target local cache, the target write data block is written to the target local cache according to the data write strategy and the tag and index of the target cache line where the target write data block is located, and the status information of the target cache line is updated to a dirty state; When the target local cache layer does not find the target write data block in the target local cache, a source cache line in a non-dirty state is selected from the target local cache for elimination, and the status information of the destination cache line corresponding to the target write data block written to the target local cache is updated to a pending write state, the target write data block is written to the destination cache line, and the status information of the destination cache line is updated to a dirty state.

4. The cache consistency maintenance method according to claim 1, wherein: The target local cache layer is a local cache layer of a target computing unit. The target computing unit includes a target local cache and uses a target cache consistency protocol. The process of maintaining the consistency of its local cache includes: The target local cache layer monitors data access to the target local cache by each computing unit of the heterogeneous computing system based on the target cache coherence protocol, and determines whether to update the data copy in the target local cache according to the data access status and status information of each cache line of the target local cache; When the target data copy in the target local cache is invalid, the target data copy is marked as invalid or a data update operation is performed.

5. The cache consistency maintenance method according to claim 1, wherein: The target local cache layer is a local cache layer of a target computing unit, and the target computing unit includes a target local cache and further includes: Obtaining read and write operation data and access latency data of the target computing unit collected by the target local cache layer; Determine a cache hit rate of the target local cache layer according to the read and write operation data, and determine an average access delay according to the access delay data; According to the cache hit rate and / or the average access delay, configuration parameters of the target local cache layer are adjusted accordingly.

6. The cache consistency maintenance method according to claim 5, characterized in that: Determining a cache hit rate of the target local cache layer according to the read and write operation data includes: Determine the read hit rate based on the number of read operations and the read operation cache hit rate; Determine a write hit rate based on the number of write operations and the write operation cache hit rate; The sum of the read hit rate and the write hit rate is taken as the total hit rate, the sum of the number of read operations and the number of write operations is taken as the total number of operations, and the cache hit rate of the target local cache layer is determined based on the total hit rate and the total number of operations.

7. The cache consistency maintenance method according to claim 1, wherein: When the data operation request is a write request forwarded by the first local cache layer to the corresponding cluster cache layer when the data to be written by the first computing unit is not in the first local cache, the target cluster cache layer includes a target cluster shared memory, performs corresponding operations based on a target group consistency protocol that matches the memory consistency model of each computing unit in the cluster, and maintains the consistency of its cluster shared cache, including: When the target cluster cache layer finds the to-be-written data in the target cluster shared memory, it updates the target cluster shared memory according to the target group consistency protocol, and updates the state of the to-be-written data in the second shared cache line of the target cluster shared memory to a dirty state; When the target cluster cache layer fails to find the data to be written in the target cluster shared memory, a second source shared cache line in a non-dirty state is selected from the target cluster shared memory for elimination, the data to be written is written to a second destination shared cache line in the target cluster shared memory, the state information of the second destination shared cache line is updated, and the target group computing units associated with the data to be written in the target cluster are notified based on the target group consistency protocol, so that each target group computing unit updates the corresponding locally cached data copy; The first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

8. The cache consistency maintenance method according to claim 1, wherein: The target cluster cache layer belongs to the target cluster and includes the target cluster shared memory, and also includes: Obtaining an average propagation delay of cache line invalidation messages and an average propagation delay of cache line update messages of the target cluster shared memory; Respectively obtaining an invalidation weight coefficient corresponding to the cache line invalidation message and an update weight coefficient corresponding to the cache line update message; Determining the intra-cluster cache coherence overhead of the target cluster according to the average propagation delay of the cache line invalidation message, the average propagation delay of the cache line update message, the invalidation weight coefficient, and the update weight coefficient; According to the intra-cluster cache coherence overhead of the target cluster, the cluster configuration parameters and / or the intra-cluster communication network structure of the target cluster are adjusted.

9. The cache consistency maintenance method according to claim 1, wherein: After determining that the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer, the method further includes: A corresponding local message queue is set in each local cache layer, a corresponding group message queue is set in each cluster cache layer, and a global message queue is set in the global cache layer; Whenever each local cache layer interacts with the corresponding cluster cache layer for data or status updates, a corresponding local message is generated, the local message is encapsulated in the corresponding local message format, and placed in the corresponding local message queue; Whenever each cluster cache layer interacts with the corresponding local cache layer, other cluster cache layers, or the global cache layer for data exchange or status update, a corresponding group message is generated, the group message is encapsulated in a corresponding group message format, and placed in a corresponding group message queue; When the global cache layer interacts with each cluster cache layer for data or status updates, a global message is generated, the global message is encapsulated according to a corresponding global message format, and is placed in a corresponding global message queue.

10. The cache consistency maintenance method according to claim 9, characterized in that: After setting a message router for each cluster cache layer and determining whether the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer, the following steps are further included: Using the message router, monitoring the local message queue of the computing unit in the group and the message queue of the group to which it belongs; According to the receiving address, message type and message tag of the target message, the receiving layer and sending time limit of the target message are determined, and the target message is sent according to the receiving layer and the sending time limit; the receiving layer is the local cache layer, cluster cache layer or global cache layer that receives the target message, and the message tag includes at least a time limit mark.

11. The cache consistency maintenance method according to claim 10, characterized in that: After determining that the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer, the method further includes: When the receiving layer receives the target message, it verifies the integrity and legitimacy of the target message and executes the target message after passing the verification; When the message reception confirmation information of the receiving layer is not received within the preset message confirmation time period, the target message is resent under the condition that the preset maximum number of retransmissions is met until the message reception confirmation information is received.

12. The cache consistency maintenance method according to claim 1, wherein: Also includes: Obtaining a message retransmission rate of the heterogeneous computing system and sending parameters of a target historical message; the sending parameters at least include an occupied capacity value of the target historical message, a communication link bandwidth, a communication link delay, and a message processing time; Determining a message sending time of the target historical message according to the occupied capacity value of the target historical message and the communication link bandwidth; using the communication link delay as the message transmission delay of the target historical message, and determining a retransmission time according to the message retransmission rate, the message sending time, the message transmission delay, and the message processing time; The total inter-group communication time of a single message is determined according to the message sending time, the message transmission delay, the message processing time, and the retransmission time, so as to adjust the communication configuration parameters of the heterogeneous computing system according to the total inter-group communication time.

13. The cache consistency maintenance method according to any one of claims 1 to 12, characterized in that: After determining that the data operation request is forwarded to the local cache layer, the cluster cache layer, or the global cache layer, the method further includes: Constructing an initial dictionary based on a preset seed item, wherein each item in the initial dictionary has a unique index and a corresponding string; Matching a target string having the same characters as the input data in the initial dictionary, forming a new string with the target string and the next input data, and adding a corresponding index value to the new string as a new entry, thereby generating an adaptive message coding dictionary by continuously adding new entries to the initial dictionary; inputting each requested character of the data operation request into the adaptive message coding dictionary in sequence, determining index information of each requested character in the adaptive message coding dictionary, and determining remaining characters in the data operation request that are not successfully matched with the adaptive message coding dictionary, and using the index information and the remaining characters as data to be encoded; The data to be encoded is encoded according to the frequencies of occurrence of different characters in the data to be encoded, encoding sequence information is generated, and the encoding sequence information is sent instead of the data operation request.

14. The cache consistency maintenance method according to claim 13, wherein: Encoding the data to be encoded according to the frequencies of occurrence of different characters in the data to be encoded, including: Acquire different characters of the data to be encoded and the frequencies of their occurrence in the data to be encoded as character information to be encoded; Using different characters as nodes of a tree structure, selecting the first character and the second character with the lowest frequency as leaf nodes, and merging the first character and the second character into a parent node of the leaf node, where the frequency of the parent node is the sum of the frequencies of the first character and the second character; Selecting a third character with the lowest frequency from the remaining characters in the character information to be encoded, taking the third character and the parent node as child nodes at the same level, merging the third character and the parent node to obtain their parent node, and repeating this process until there are no remaining characters in the character information to be encoded, thereby obtaining a tree to be encoded; A first code value is assigned to each left child node of the tree to be encoded, and a second code value is assigned to each right child node. Starting from the root node, a coding sequence is recursively generated for each character of the character information to be encoded based on the code value of each node, and the coding sequences of each character of the character information to be encoded are combined into the coding sequence information.

15. The cache consistency maintenance method according to claim 13, wherein: Also includes: Obtaining a space capacity value of the adaptive message coding dictionary; The logarithmic value of the space capacity value with a preset number as the base is calculated, and the rounded-up value of the logarithmic value is used as the minimum index coding length of the adaptive message coding dictionary.

16. The cache consistency maintenance method according to any one of claims 1 to 12, characterized in that: The protocol format conversion of the data operation request is performed according to the mapping relationship between the group consistency protocol and the global consistency protocol by a consistency protocol adaptation layer having an interface, including: Establishing a mapping rule table from each group consistency protocol to the global consistency protocol in advance based on the group consistency protocol and the global consistency protocol used by each cluster of the heterogeneous computing system; When a new group consistency protocol appears in the heterogeneous computing system, determining a new mapping rule from the new group consistency protocol to the global consistency protocol, and updating the mapping rule table accordingly through the interface; Whenever a data operation request that arrives out of order is received, a unique message sequence number is assigned to the data operation request while caching the data operation request; According to the order of each message sequence number and on the basis of retaining the data dependency information, the protocol format conversion is performed on the data operation request in sequence.

17. The cache consistency maintenance method according to any one of claims 1 to 12, characterized in that: Maintaining the consistency of shared caches across clusters based on a global consistency protocol includes: Monitor memory access requests for each cluster's shared cache layer; The access rights and access sequence of each memory access request are dynamically adjusted according to the load information of the heterogeneous computing system.

18. A cache consistency maintenance device, characterized in that: include: A request forwarding module is configured to, when a heterogeneous computing system detects a data operation request, determine whether the data operation request is forwarded to a local cache layer, a cluster cache layer, or a global cache layer; The computing units in the heterogeneous computing system that meet the same or similar conditions of the memory consistency model form a cluster, and each cluster includes a cluster cache layer that constructs the local caches of the computing units in the cluster into a cluster shared cache; A protocol conversion module is configured to convert the protocol format of the data operation request according to the mapping relationship between the group consistency protocol and the global consistency protocol through a consistency protocol adaptation layer having an interface when the sender is a cluster cache layer and the receiver is a global cache layer, or when the sender is a global cache layer and the receiver is a cluster cache layer, or when both the sender and the receiver are cluster cache layers; The request processing module is used to perform corresponding operations based on the memory consistency model of the computing unit to which it belongs and maintain the consistency of its local cache when forwarding to the target local cache layer; When forwarded to the target cluster cache layer, the corresponding operation is performed based on the target cluster consistency protocol that matches the memory consistency model of each computing unit in the cluster, and the consistency of its cluster shared cache is maintained; when forwarded to the global cache layer, the consistency of each cluster shared cache is maintained based on the global consistency protocol; Wherein, the request processing module is further used to: when the target cluster cache layer finds the to-be-read data in the target cluster shared memory, send the to-be-read data to the first local cache layer, and update the cache line status information of the to-be-read data in the target cluster shared memory; when the target cluster cache layer does not find the to-be-read data in the target cluster shared memory, select the first source shared cache line in a non-dirty state from the target cluster shared memory for elimination, and write the to-be-read data read from the main memory or the next-level storage to the first destination shared cache line of the target cluster shared memory, send the to-be-read data to the first local cache layer, and update the status information of the first destination shared cache line; wherein, the target cluster cache layer includes the target cluster shared memory, and when the data operation request is a read request forwarded by the first local cache layer to the cluster cache layer to which it belongs when the to-be-read data of the first computing unit is not in the first local cache, the first local cache is the local cache of the first computing unit, and the first local cache layer is the local cache layer of the first computing unit in the target cluster to which the target cluster cache layer belongs.

19. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the cache consistency maintenance method according to any one of claims 1 to 17 when executing the computer program.

20. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, which, when executed by a processor, implements the steps of the cache consistency maintenance method according to any one of claims 1 to 17.

21. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the cache consistency maintenance method according to any one of claims 1 to 17 are implemented.

22. A heterogeneous computing system, characterized in that: The system comprises at least a first computing node, a second computing node, and a cache consistency controller, wherein the cache consistency controller is connected to the first computing node and the second computing node; the first computing node comprises at least a first computing unit and a second computing unit, and the second computing node comprises at least a third computing unit and a fourth computing unit; the memory consistency models of the first computing unit, the second computing unit, and the third computing unit respectively satisfy the same similarity condition of the memory consistency model; wherein the first computing unit, the second computing unit, the third computing unit, and the fourth computing unit each include a local cache layer for managing their respective local caches; the first computing unit, the second computing unit, and the third computing unit constitute a first cluster; the first cluster includes a cluster cache layer for constructing the local caches of the first computing unit, the second computing unit, and the third computing unit into a cluster shared cache; the first computing unit, the second computing unit, and the third computing unit are connected to the cluster shared cache via a high-speed interconnection network; and the shared caches of each cluster are maintained consistent through a global cache layer; The cache consistency controller is configured to implement the steps of the cache consistency maintenance method according to any one of claims 1 to 17 when executing a computer program.

Images