Distributed cache consistency system, method, device, storage medium and product

Abstract

The invention provides a distributed cache consistency system, method and device, a storage medium and a product, and is applied to a CXL switch, and a connection physical device comprises a first cache device and a CXL memory protocol-based memory extension device comprising a second cache device and a memory device. The system is composed of a virtualization aggregation layer and a cache consistency control module, the virtualization aggregation layer aggregates physical equipment as logic equipment, and the cache consistency control module intercepts operation requests, performs protocol conversion and queries directories to determine cache line states and equipment where the cache line states are located and feeds back target data. According to the scheme, the problems of delay, expansibility, communication efficiency and resource waste of the existing centralized architecture are solved. A distributed system is deployed on a CXL switch, so that processing skips and steps are reduced, and delay is reduced; a distributed architecture is adopted to avoid resource bottlenecks and improve expansibility; inter-device communication intermediate links are reduced, and the communication efficiency is improved; intermediate nodes of a data transmission path are reduced, transmission overhead is reduced, resource utilization rate is improved, and resource waste is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a distributed cache consistency system, method, device, storage medium, and product. Background Art

[0002] In the CXL (Compute Express Link) 3.0 architecture, cross-device cache coherence is a key mechanism for achieving efficient multi-device collaborative computing. However, in the current architecture, the Home Agent (HA), acting as a global coherence controller, must reside on the host, responsible for managing all device cache directories and coordinating listener requests. While this centralized control approach simplifies system design to a certain extent, it also introduces significant performance bottlenecks and scalability limitations.

[0003] First, latency is a core challenge in existing technologies. Each cache coherence operation requires multiple hops to complete. Furthermore, processing a request requires parsing message headers, querying directories, and encapsulating the response message. These additional steps increase processing time, especially in large-scale distributed systems, where frequent operations can become a critical bottleneck to system performance.

[0004] Second, existing technologies are severely limited in scalability. As the number of devices increases, the number of requests that need to be processed increases exponentially. However, limited resources make it difficult to support the consistency requirements across a massive number of devices. This scalability limitation not only impacts the system's ability to scale but also risks performance degradation or even system crashes under high load.

[0005] Another significant issue is the limitations of the topology. In existing architectures, communication between all devices must be indirectly coordinated through an intermediary. While this approach simplifies implementation, it sacrifices communication efficiency, making it particularly inefficient in scenarios where devices need to frequently exchange data.

[0006] Finally, existing technologies also suffer from resource waste. Even if devices natively support direct communication, control flows still need to be coordinated through an intermediary. This design leads to a circuitous data path and adds unnecessary transmission overhead. Summary of the Invention

[0007] The present disclosure provides a distributed cache consistency system, method, device, storage medium and product to at least solve the above technical problems existing in the prior art.

[0008] According to a first aspect of the present disclosure, a distributed cache coherence system is provided, which is applied to a computing express link (CXL) switch, wherein the CXL switch is connected to multiple physical devices, wherein the physical devices include a first cache device and a memory expansion device based on the CXL memory protocol, wherein the memory expansion device includes a second cache device and a memory device; the system includes: a virtualization aggregation layer and a cache coherence control module;

[0009] A virtualization aggregation layer, configured to aggregate the multiple physical devices into multiple logical devices;

[0010] A cache consistency control module is used to intercept operation requests sent by the first cache device to the target address and perform protocol conversion; query the cache consistency directory and determine the cache line status of the target address and the physical device where the cache line is located; determine the target data and provide feedback based on the cache line status and the physical device where the cache line is located.

[0011] In one embodiment, the virtualization aggregation layer is specifically used to:

[0012] aggregating cache devices in the physical device into at least one logical cache device, wherein the logical cache device is presented to the host as a CXL cache endpoint;

[0013] Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

[0014] In one embodiment, the CXL switch is connected to a physical device via a downstream port, including:

[0015] The logical memory device and the logical cache device are connected to corresponding memory expansion devices via a first downstream port;

[0016] The logical cache device is connected to the first cache device through a second downstream port.

[0017] In one embodiment, the cache consistency control module is specifically configured to:

[0018] intercepting an operation request initiated by the first cache device to the logical device, and converting the operation request into a target request for the corresponding memory expansion device;

[0019] Querying a locally maintained cache coherence directory to determine a third cache device and a cache line status where a cache line corresponding to a target address indicated by the target request is located;

[0020] According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

[0021] In one possible implementation manner, determining target data and providing feedback based on the cache line state and the third cache device where the cache line is located includes:

[0022] If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, directly feeding back the target data to the first cache device;

[0023] If the cache line exists in the third cache device but its status does not meet the consistency requirement of the target request, generate and send a snoop request to the third cache device, and select a target mode to process data transfer according to the scenario;

[0024] The cache line states of the first cache device and the third cache device in the coherence directory are updated.

[0025] In one possible implementation, selecting a target mode for processing data transmission according to a scenario includes:

[0026] In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device;

[0027] In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

[0028] According to a second aspect of the present disclosure, a distributed cache consistency method is provided, which is applied to a computing express link (CXL) switch. The CXL switch is connected to multiple physical devices, wherein the physical devices include a first cache device and a memory expansion device based on the CXL memory protocol. The memory expansion device includes a second cache device and a memory device. The method includes:

[0029] Aggregating the multiple physical devices into multiple logical devices through a virtualization aggregation layer;

[0030] The cache consistency control module intercepts an operation request sent by the first cache device to the target address and performs protocol conversion on the operation request; queries the cache consistency directory to determine the cache line status of the target address and the physical device where the cache line is located; and determines the target data based on the cache line status and the physical device where the cache line is located and provides feedback.

[0031] In one possible implementation manner, aggregating the multiple physical devices into multiple logical devices through a virtualization aggregation layer includes:

[0032] aggregating cache devices in the physical device into at least one logical cache device, wherein the logical cache device is presented to the host as a CXL cache endpoint;

[0033] Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

[0034] In one embodiment, a CXL switch connects to a physical device via a downstream port, including:

[0035] Connecting the logical memory device and the logical cache device to corresponding memory expansion devices through a first downstream port;

[0036] The logical cache device is connected to the first cache device through the second downstream port.

[0037] In one possible implementation, the method includes utilizing a cache coherence control module to intercept an operation request sent by a first cache device to a target address and perform protocol conversion on the operation request; querying a cache coherence directory to determine a cache line status of the target address and a physical device where the cache line is located; and determining target data and providing feedback based on the cache line status and the physical device where the cache line is located, including:

[0038] intercepting an operation request initiated by the first cache device to the logical device, and converting the operation request into a target request for the corresponding memory expansion device;

[0039] Querying a locally maintained cache coherence directory to determine a third cache device and a cache line status where a cache line corresponding to a target address indicated by the target request is located;

[0040] According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

[0041] In one possible implementation manner, determining the target data and providing feedback based on the cache line state and the third cache device where the cache line is located specifically includes:

[0042] If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, directly feeding back the target data to the first cache device;

[0043] If the cache line exists in the third cache device but its status does not meet the consistency requirement of the target request, generate and send a snoop request to the third cache device, and select a target mode to process data transfer according to the scenario;

[0044] The cache line states of the first cache device and the third cache device in the coherence directory are updated.

[0045] In one possible implementation, selecting a target mode for processing data transmission according to a scenario specifically includes:

[0046] In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device;

[0047] In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

[0048] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0049] at least one processor; and

[0050] a memory communicatively connected to the at least one processor; wherein,

[0051] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0052] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0053] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program or instructions for causing a processor to implement the method when executing the computer program or instructions.

[0054] The distributed cache consistency system, method, device, storage medium and product disclosed in the present invention are applied to computing fast link CXL switches. The physical devices connected to the system include a first cache device and a memory expansion device based on the CXL memory protocol (including a second cache device and a memory device). The system is mainly composed of a virtualization aggregation layer and a cache consistency control module. The virtualization aggregation layer is responsible for aggregating multiple physical devices into multiple logical devices; the cache consistency control module is responsible for intercepting operation requests sent by the first cache device to the target address, performing protocol conversion, querying the cache consistency directory to determine the cache line status of the target address and its physical device, and finally determining the target data based on this information and feedback. In terms of latency, it solves the performance bottleneck problem caused by multiple jumps and additional processing steps caused by the centralized architecture of the existing technology. By deploying a distributed cache consistency system on the CXL switch, the processing jumps and steps are reduced, the latency is greatly reduced, and the system performance is improved. In terms of scalability, it avoids the resource bottleneck under the centralized architecture. The distributed architecture allows each switch to manage a certain number of devices. As the number of devices increases, only additional switches need to be added, so that the system can better adapt to the growth of devices and improve scalability. In terms of communication efficiency, this reduces the number of intermediate links in inter-device communication, allowing cache coherence operations to be processed directly on the switch, improving communication efficiency in scenarios with frequent data exchange. In terms of resource utilization, this reduces the number of intermediate nodes in the data transmission path, avoiding detours and reducing unnecessary transmission overhead, thereby improving resource utilization and effectively minimizing resource waste.

[0055] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0057] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0058] Figure 1 A schematic diagram of the architecture of a distributed cache consistency system according to an embodiment of the present disclosure is shown;

[0059] Figure 2 A schematic diagram of the implementation process of the distributed cache consistency method according to an embodiment of the present disclosure is shown;

[0060] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0062] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of the present disclosure. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0064] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are explained. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.

[0065] The Compute Express Link (CXL) protocol is an open, high-speed interconnect standard based on the PCIe physical layer, designed to enable efficient collaboration between CPUs, accelerators, and memory devices. Through three protocols, CXL.io, CXL.cache, and CXL.mem, it supports low-latency, high-bandwidth data transfer, hardware-level cache coherence maintenance, and memory pooling. It is also compatible with heterogeneous computing architectures, significantly improving resource utilization.

[0066] The CXL Switch is a dedicated switching device that supports the CXL protocol. Its core function is to efficiently route CXL transactions. As a key component of the CXL ecosystem, it enables high-speed communication between the CPU and accelerators and memory expansion devices through hardware-level forwarding mechanisms.

[0067] The Home Agent (HA) is the core controller in the cache coherence protocol, responsible for maintaining the global cache directory and coordinating Snoop operations across devices. It ensures data access consistency across multiple devices by centrally managing the status of each cache line (such as Modified, Shared, and Invalid) and the device to which it belongs. When a device initiates a request, the HA acts as an intermediary, handling directory queries, Snoop request distribution, and data transmission, ensuring system state synchronization. It is a key component in maintaining multi-device collaboration.

[0068] The Snoop operation is a mechanism used in cache coherence protocols to monitor bus transactions. It listens to cache access requests from other devices, checks whether the local cache holds a copy of the target data, and performs invalidation, update, or sharing operations based on the protocol status (such as MESI) to ensure the consistency of data copies between multiple devices.

[0069] In current computing architectures, cache coherence management is a crucial foundation for enabling multi-device collaboration. Existing technologies primarily employ a host-centric Home Agent architecture, whose core characteristic lies in its centralized directory management. Specifically, Host HA maintains a global cache directory that records the status of each cache line and the specific device to which it belongs. This allows the system to coordinate data access across multiple devices and ensure data consistency.

[0070] In this architecture, when cache coherence operations are required between devices, the entire process relies heavily on Host HA as an intermediary. For example, assume there are three devices in the system: cache_2 (the CPU or accelerator's local cache), cache_3 (another accelerator's local cache), and mem_cache_1 (a memory expansion device based on the CXL.mem protocol). If cache_2 needs to read data in mem_cache_1, which is already cached and marked as Modified by cache_3, a complex series of steps are required to achieve coherence. First, cache_2 initiates a read request to mem_cache_1, but mem_cache_1 discovers that the target data is actually stored in cache_3. At this point, mem_cache_1 sends a proxy Snoop request to Host HA via the CXL.mem protocol interface. Host HA then issues a standard snoop request (following the CXL.cache protocol) to cache_3 based on information from the global cache directory. After receiving the request, cache_3 returns the data to Host HA and writes the data back to mem_cache_1. Finally, mem_cache_1 responds to cache_2's original request and returns the data to cache_2.

[0071] While this process ensures data consistency, it also introduces significant latency. Because all Snoop requests and data transfers must be coordinated through Host HA, the entire process involves multiple round-trips across the CXL switches. For example, from cache_2 initiating a request to ultimately obtaining data, the entire path involves six round-trips across the CXL switches: cache_2 → Switch → mem_cache_1 → Switch → Host HA → Switch → cache_3 → Switch → Host HA / mem_cache_1 → cache_2. This high frequency of cross-switch communication not only increases system latency but also significantly impacts overall performance.

[0072] Although this centralized design simplifies system implementation to a certain extent, it also exposes obvious shortcomings: on the one hand, Host HA becomes a key bottleneck of the entire system, limiting the system's scalability and efficiency; on the other hand, frequent cross-Switch communication further exacerbates the latency problem, especially in large-scale distributed systems, where this impact is more significant.

[0073] Therefore, in order to solve the defects of high latency and low scalability in traditional system architecture, the present disclosure proposes a distributed cache consistency system for computing fast link CXL switches. Figure 1 As shown:

[0074] The CXL switch is a switching device that supports the CXL protocol and is responsible for routing CXL transactions (such as data requests and state synchronization). The core optimization of this system relies on the hardware capabilities of the CXL switch. By deploying some cache coherence control logic in the switch (rather than on the traditional host side), it solves the problems of high host load and high cross-device communication latency.

[0075] Specifically, the CXL switch connects to multiple physical devices, the physical devices include a first cache device and a memory expansion device based on the CXL memory protocol, the memory expansion device includes a second cache device and a memory device; the system includes: a virtualization aggregation layer and a cache consistency control module.

[0076] The first cache device refers to the local cache of computing devices such as CPU and accelerator (such as Figure 1 cache_2 and cache_3) for temporary storage of frequently accessed data to accelerate computing;

[0077] Memory extension device based on CXL memory protocol (CXL.mem device): distributed memory resources extended by CXL protocol (such as Figure 1 The mem_cache_1 in the memory device includes a second cache device (such as cache1 in mem_cache_1, which is used to cache memory data) and a memory device (mem part, which provides large-capacity storage).

[0078] The virtualization aggregation layer is used to aggregate the multiple physical devices into multiple logical devices.

[0079] In this example, the virtualized aggregation layer is based on the MDCT (Multi-Device Composite Topology) feature of CXL 3.0, which abstracts multiple physical devices into functional modules of logical devices (such as Figure 1 Logical devices refer to virtual devices abstracted by virtualization technology. From the host's perspective, only logical devices are perceived, without paying attention to the underlying physical devices.

[0080] Specifically, the MDCT feature is used to aggregate physical cache devices into a logical cache device Vcache, and to map physical memory devices into a logical memory device Vmem (as a CXL.mem device), simplifying the device management process of the host.

[0081] A cache consistency control module is used to intercept operation requests sent by the first cache device to the target address and perform protocol conversion; query the cache consistency directory and determine the cache line status of the target address and the physical device where the cache line is located; determine the target data and provide feedback based on the cache line status and the physical device where the cache line is located.

[0082] In this example, the cache coherence control module is Figure 1 The Switch-HA module is integrated into the CXL switch and replaces the traditional host-side Home Agent (HA) to manage cache consistency. The specific implementation process of cache consistency management is as follows:

[0083] An operation request refers to a read / write request from the first cache device to the target address; protocol conversion refers to converting an access request from a logical device into an access request to an actual physical memory device, which involves the conversion of the CXL.mem and CXL.cache protocols. By intercepting and converting requests, it ensures that the host's access to the logical device can be correctly routed to the underlying physical device, while providing a basis for subsequent consistency management. The cache consistency directory is the core data structure in the technical briefing, which records the status of each cache line in the logical cache device and the physical device to which it belongs. By querying the directory, it is determined whether the cache line at the target address exists, its current status, and its storage location, providing a decision-making basis for subsequent consistency operations. If the cache line status meets the request requirements, the data is directly fed back; if it does not meet the requirements, a Snoop request is triggered, and the mode is selected according to the scenario, and the directory status is finally updated to maintain global consistency.

[0084] The present disclosure proposes a distributed cache consistency system for use in computing fast link CXL switches. The physical devices connected to the system include a first cache device and a memory expansion device based on the CXL memory protocol (including a second cache device and a memory device). The system is mainly composed of a virtualization aggregation layer and a cache consistency control module. The virtualization aggregation layer is responsible for aggregating multiple physical devices into multiple logical devices; the cache consistency control module is responsible for intercepting operation requests sent by the first cache device to the target address, performing protocol conversion, querying the cache consistency directory to determine the cache line status of the target address and the physical device where it is located, and finally determining the target data based on this information and feeding it back. In terms of latency, the system solves the performance bottleneck problem caused by multiple jumps and additional processing steps caused by the centralized architecture of the existing technology. By deploying a distributed cache consistency system on the CXL switch, the processing jumps and steps are reduced, latency is greatly reduced, and system performance is improved. In terms of scalability, the system avoids the resource bottleneck under the centralized architecture. The distributed architecture allows each switch to manage a certain number of devices. As the number of devices increases, only additional switches need to be added, so that the system can better adapt to the growth of devices and improve scalability. In terms of communication efficiency, this reduces the number of intermediate links in inter-device communication, allowing cache coherence operations to be processed directly on the switch, improving communication efficiency in scenarios with frequent data exchange. In terms of resource utilization, this reduces the number of intermediate nodes in the data transmission path, avoiding detours and reducing unnecessary transmission overhead, thereby improving resource utilization and effectively minimizing resource waste.

[0085] In one example, the virtualization aggregation layer is specifically used to:

[0086] Cache devices in the physical device are aggregated into at least one logical cache device, and the logical cache device is presented to the host as a CXL cache endpoint.

[0087] In this example, the cache device in the physical device refers to the local cache of the computing device such as CPU and accelerator (for example Figure 1 cache_2, cache_3, and mem_cache_1 are used to temporarily store high-frequency access data to accelerate computing. A logical cache device is a virtual device that abstracts multiple physical cache devices into a virtual device (e.g. Figure 1 The Vcache in the CXL protocol is presented to the host as a CXL.cache endpoint, which complies with the interface specification of the cache device in the CXL protocol.

[0088] Aggregation eliminates the need for hosts to understand the specific distribution of multiple underlying physical cache devices; they only interact with a unified logical cache device, simplifying host device management. For example, after multiple physical cache devices are aggregated into a single logical cache device, host access to that logical device is automatically routed to the corresponding physical cache device, eliminating the complexity of managing each cache device individually.

[0089] Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

[0090] In this example, the memory device in the physical device refers to the distributed memory resources extended by the CXL.mem protocol (for example Figure 1 The mem part in mem_cache_1 shown in the figure provides large-capacity storage; the logical memory device directly maps the physical memory device to a virtual device (such as Vmem in actual scenarios), which is presented to the host as a CXL.mem device and complies with the interface specification of memory devices in the CXL protocol.

[0091] Independent mapping ensures that the resources of physical memory devices remain independent at the logical level. Host access to logical memory devices is directly mapped to the corresponding physical memory devices, avoiding confusion with cache device resources. For example, after a physical memory device is mapped to a logical memory device, when the host accesses the logical device, the system routes the request directly to the actual physical memory device, ensuring data read and write accuracy.

[0092] By aggregating physical cache devices into logical cache devices and mapping physical memory devices into logical memory devices, the complexity of host device management is significantly reduced. At the same time, logical devices comply with the CXL protocol specification, ensuring compatibility with existing hosts and devices, and can be deployed without modifying existing hardware or software.

[0093] Downstream ports refer to the communication interface between the CXL switch and the underlying physical devices (memory expansion devices, first cache devices), responsible for the physical connection of data requests and state synchronization. In one example, the CXL switch connects to the physical devices through downstream ports, including:

[0094] The logical memory device and the logical cache device are connected to corresponding memory expansion devices through a first downstream port.

[0095] In this example, the first downstream port is a physical interface that complies with the CXL.type2 port standard and supports the cxl.io, cxl.mem, and cxl.cache protocols. It is used to connect the logical device to the memory expansion device. The memory expansion device contains both memory and cache (for example, mem_cache_1 in the actual scenario contains mem and cache1). The logical memory device Vmem and the logical cache device Vcache must be connected through the multi-protocol CXL.type2 port, or the first downstream port, to simultaneously handle memory access and cache coherence requests, ensuring proper interaction between the two types of logical devices and the physical memory expansion device.

[0096] The logical cache device is connected to the first cache device through a second downstream port.

[0097] In this example, the second downstream port refers to a physical interface that complies with the CXL.type3 port standard, supports the cxl.io and cxl.cache protocols, and is used to connect the logical cache device and the first cache device.

[0098] The first cache device of the local cache of the computing device is mainly used to handle cache consistency and basic input and output of computing tasks, so there is no need to support the memory protocol cxl.mem. Therefore, it is connected through the CXL.type3 port, that is, the second downstream port, which can not only meet functional requirements but also avoid resource waste caused by redundant protocol support.

[0099] Connection requirements are differentiated by different types of ports. The first downstream port supports multiple protocols to adapt to the complex functions of memory expansion devices, and the second downstream port supports streamlined protocols to adapt to the local cache requirements of computing devices, which not only ensures the efficiency of data interaction but also optimizes the utilization of switch port resources.

[0100] In one example, the cache consistency control module is specifically configured to:

[0101] An operation request initiated by the first cache device to the logical device is intercepted, and the operation request is converted into a target request for the corresponding memory expansion device.

[0102] In this example, when accessing data, the first cache device typically initiates a request to the logical device. In traditional architectures, such requests must be coordinated by the host's Home Agent, resulting in a lengthy path (e.g., cache_2 → Switch → HostHA → Switch → target device). This module intercepts these requests to shift control flow from the host to the CXL switch, preventing the request from detouring around the host and thus reducing the number of communication hops. After interception, the request is no longer sent directly to the host, but is instead handled by the cache coherence control module within the switch.

[0103] The logical device is the virtual address space after virtualization. The access address from the host perspective (such as 0xC000 of Vmem) needs to be mapped to the physical address of the actual physical memory expansion device (such as 0xC000 of mem_cache_1). The conversion process consists of two parts:

[0104] Address mapping: Logical addresses are converted to physical addresses using preconfigured address matching rules at the virtualization aggregation layer. Protocol translation: If a request involves cache coherency, memory access requests using the CXL.mem protocol are translated into cache query requests using the CXL.cache protocol. For example, if mem_cache_1 discovers that data is actually stored in cache_3, it triggers the CXL.cache snoop process. After translation, the target request is directly directed to the physical memory expansion device, ensuring that subsequent operations accurately locate the underlying physical resources.

[0105] A locally maintained cache coherence directory is queried to determine a third cache device where a cache line corresponding to a target address indicated by the target request is located and a cache line status.

[0106] In this example, the cache coherence directory is a core data structure maintained within the switch, recording metadata about all cache lines within the logical cache device, including cache line address, cache line status, and the physical device to which it belongs.

[0107] After the target request address is translated, the module queries the directory to answer two key questions: whether the cache line at that address exists in a physical cache device; and if so, whether the current state allows direct access. For example, if the cache coherence directory shows that the cache line at the target address is stored in cache_3 and is in the Modified state (data has been modified but not written back to memory), reading directly from memory will result in stale data, and a Snoop request must be triggered to synchronize the latest data.

[0108] According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

[0109] In this example, if the cache line status meets the request requirements, the data is directly obtained from the third cache device or memory expansion device and fed back to the first cache device. If the status does not meet the requirements, a snoop request is triggered to request data synchronization. Finally, write-back mode or P2P direct transfer mode is selected according to the scenario, and the directory status is updated to ensure global cache consistency. The specific implementation process will be described in detail in subsequent embodiments.

[0110] In one example, determining target data and providing feedback based on the cache line state and the third cache device where the cache line is located includes:

[0111] If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, the target data is directly fed back to the first cache device.

[0112] In this example, the consistency requirement refers to the cache line validity state defined by the operation request type (read / write). For example, the consistency requirement for a read request is typically "Shared" (data is consistent with memory and can be shared by multiple devices) or "Exclusive" (data is consistent with memory and only held by the current device); the consistency requirement for a write request is typically "Modified" (data has been modified but not synchronized with memory) or "Exclusive" (data can be modified exclusively by the current device).

[0113] When a cache line exists and its status meets the requirements, for example, if the first cache device, cache_2, initiates a read request and the cache line at the target address exists in cache_3 and is in the Shared state, this indicates that the cache line data is consistent with the memory and can be safely accessed. At this point, the cache coherence control module, without requiring additional synchronization, directly retrieves the data from the third cache device or memory expansion device and feeds it back to the first cache device via the switch's internal channel. For example, if the cache line status is Shared, the data has been synchronized to the memory. The module can choose to read from the memory or directly from cache_3, ultimately returning the data to cache_2 to complete the request response.

[0114] If the cache line exists in the third cache device but the state does not meet the consistency requirement of the target request, a snoop request is generated and sent to the third cache device, and a target mode is selected according to the scenario to process data transfer.

[0115] When a cache line's status doesn't meet requirements, for example, cache_2 initiates a read request, but the target cache line exists in cache_3 and is in the Modified state. This means the data has been modified and not synchronized with the memory. Directly reading the memory will result in outdated data, so a snoop request (such as Read Snoop) is triggered. The specific process is as follows: Generate Snoop Request: The cache coherence control module generates a request based on the CXL.cache protocol, requesting the third cache device to process its cache line. Subsequently, the target mode is selected based on the current scenario to handle the data transfer.

[0116] The cache line states of the first cache device and the third cache device in the coherence directory are updated.

[0117] In this example, no matter which mode is adopted for data transfer, the cache coherence control module needs to update the cache line status of the first cache device and the third cache device in the directory to reflect the latest coherence status.

[0118] Furthermore, the failure to find the third cache device means that the cache line status of the target address in the cache coherence directory is "Invalid," meaning that the data is not cached by any physical cache device and exists only in memory. Therefore, the target data is retrieved from memory and processed, without updating the cache line status of the first and third cache devices in the directory.

[0119] In one example, selecting a target mode for processing data transmission according to a scenario includes:

[0120] In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device.

[0121] In this example, the write-back mode refers to a mode in which, during data transfer, the third cache device first synchronizes the modified data to the memory expansion device, and then the memory expansion device returns the data.

[0122] When write-back mode is selected for data transfer, for example, if the target data requires persistent storage or may be accessed later by other devices, the third cache device writes the modified data in its cache back to the memory expansion device (the mem portion of mem_cache_1). This operation synchronizes the data between the memory and cache, ensuring that the latest version of the data is stored in memory. After the write-back is complete, the third cache device updates its own cache line status from Modified to Shared. The memory expansion device then returns the latest data to the first cache device that initiated the request, completing data feedback. For example, if cache_2 requests to read data at address 0xC000, and that data has been modified in cache_3 (status is Modified), cache_3 must first write the data back to the mem portion of mem_cache_1. mem_cache_1 then returns the data to cache_2, ensuring that cache_2 has the latest data.

[0123] In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

[0124] In this example, direct transmission mode refers to the peer-to-peer (P2P) feature based on the CXL 3.0 protocol. It allows cache devices to communicate directly through the forwarding channel inside the switch without the need for coordination between the memory expansion device or the host.

[0125] When direct transmission mode is selected for data transfer, for example, in latency-sensitive high-performance computing scenarios, the third cache device uses the P2P feature of CXL 3.0 to send data directly to the first cache device through the fully connected crossbar switch (Crossbar architecture) inside the switch. The core of this mode is to bypass the transit of the memory expansion device and the cache consistency control module to shorten the data path. For example, when the cache line status of cache_3 is Modified and cache_2 initiates a read request, the inter-cache forwarding channel is enabled inside the switch, allowing cache_3 to send data directly to cache_2 without having to write it back to memory first. At this time, the cache line status of the third cache device is updated to Shared, and the cache line status of the first cache device is updated from Invalid to Shared.

[0126] The present disclosure also provides a distributed cache consistency method, which is applied to a computing fast link CXL switch, wherein the CXL switch is connected to multiple physical devices, wherein the physical device includes a first cache device and a memory expansion device based on the CXL memory protocol, wherein the memory expansion device includes a second cache device and a memory device, such as Figure 2 As shown, the method includes:

[0127] Step 201: Aggregate the multiple physical devices into multiple logical devices through a virtualization aggregation layer;

[0128] Step 202: intercept the operation request sent by the first cache device to the target address through the cache consistency control module, and perform protocol conversion on the operation request; query the cache consistency directory to determine the cache line status of the target address and the physical device where the cache line is located; determine the target data based on the cache line status and the physical device where the cache line is located and provide feedback.

[0129] In one example, aggregating the multiple physical devices into multiple logical devices through a virtualization aggregation layer includes:

[0130] aggregating cache devices in the physical device into at least one logical cache device, wherein the logical cache device is presented to the host as a CXL cache endpoint;

[0131] Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

[0132] In one example, a CXL switch connects physical devices via downstream ports, including:

[0133] Connecting the logical memory device and the logical cache device to corresponding memory expansion devices through a first downstream port;

[0134] The logical cache device is connected to the first cache device through the second downstream port.

[0135] In one example, the method includes: intercepting an operation request sent by a first cache device to a target address using a cache coherence control module, performing protocol conversion on the operation request; querying a cache coherence directory to determine a cache line status of the target address and a physical device where the cache line is located; and determining target data and providing feedback based on the cache line status and the physical device where the cache line is located, including:

[0136] intercepting an operation request initiated by the first cache device to the logical device, and converting the operation request into a target request for the corresponding memory expansion device;

[0137] Querying a locally maintained cache coherence directory to determine a third cache device and a cache line status where a cache line corresponding to a target address indicated by the target request is located;

[0138] According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

[0139] In one example, determining target data and providing feedback based on the cache line state and the third cache device where the cache line is located specifically includes:

[0140] If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, directly feeding back the target data to the first cache device;

[0141] If the cache line exists in the third cache device but its status does not meet the consistency requirement of the target request, generate and send a snoop request to the third cache device, and select a target mode to process data transfer according to the scenario;

[0142] The cache line states of the first cache device and the third cache device in the coherence directory are updated.

[0143] In one example, selecting a target mode for processing data transmission according to a scenario specifically includes:

[0144] In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device;

[0145] In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

[0146] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0147] Figure 3 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0148] like Figure 3As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0149] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the distributed cache consistency method. For example, in some embodiments, the distributed cache consistency method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the distributed cache consistency method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the distributed cache consistency method by any other appropriate means (e.g., by means of firmware).

[0151] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0157] The present disclosure also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the distributed cache consistency method described above in the present disclosure.

[0158] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0159] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0160] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A distributed cache consistency system, characterized in that: Applied to a computing fast link CXL switch, the CXL switch connects multiple physical devices, the physical devices include a first cache device and a memory expansion device based on the CXL memory protocol, the memory expansion device includes a second cache device and a memory device; the system includes: a virtualization aggregation layer and a cache consistency control module; A virtualization aggregation layer, configured to aggregate the multiple physical devices into multiple logical devices; A cache consistency control module is used to intercept operation requests sent by the first cache device to the target address and perform protocol conversion; query the cache consistency directory and determine the cache line status of the target address and the physical device where the cache line is located; determine the target data and provide feedback based on the cache line status and the physical device where the cache line is located.

2. The system according to claim 1, wherein: The virtualization aggregation layer is specifically used for: aggregating cache devices in the physical device into at least one logical cache device, wherein the logical cache device is presented to the host as a CXL cache endpoint; Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

3. The system according to claim 2, characterized in that The CXL switch connects to physical devices through downstream ports, including: The logical memory device and the logical cache device are connected to corresponding memory expansion devices via a first downstream port; The logical cache device is connected to the first cache device through a second downstream port.

4. The system according to claim 1, wherein: The cache consistency control module is specifically used to: intercepting an operation request initiated by the first cache device to the logical device, and converting the operation request into a target request for the corresponding memory expansion device; Querying a locally maintained cache coherence directory to determine a third cache device and a cache line status where a cache line corresponding to a target address indicated by the target request is located; According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

5. The system according to claim 4, characterized in that The determining target data and providing feedback according to the cache line state and the third cache device where the cache line is located includes: If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, directly feeding back the target data to the first cache device; If the cache line exists in the third cache device but its status does not meet the consistency requirement of the target request, generate and send a snoop request to the third cache device, and select a target mode to process data transfer according to the scenario; The cache line states of the first cache device and the third cache device in the coherence directory are updated.

6. The system according to claim 5, characterized in that The step of selecting a target mode according to a scenario to process data transmission includes: In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device; In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

7. A distributed cache consistency method, characterized in that: The method is applied to a computing express link (CXL) switch, wherein the CXL switch is connected to a plurality of physical devices, wherein the physical devices include a first cache device and a memory expansion device based on the CXL memory protocol, wherein the memory expansion device includes a second cache device and a memory device. The method includes: Aggregating the multiple physical devices into multiple logical devices through a virtualization aggregation layer; The cache consistency control module intercepts an operation request sent by the first cache device to the target address and performs protocol conversion on the operation request; queries the cache consistency directory to determine the cache line status of the target address and the physical device where the cache line is located; and determines the target data based on the cache line status and the physical device where the cache line is located and provides feedback.

8. The method according to claim 7, characterized in that The aggregating the multiple physical devices into multiple logical devices through a virtualization aggregation layer includes: aggregating cache devices in the physical device into at least one logical cache device, wherein the logical cache device is presented to the host as a CXL cache endpoint; Memory devices in the physical device are independently mapped into logical memory devices, and the logical memory devices are presented to the host as CXL memory devices.

9. The method according to claim 8, characterized in that CXL switches connect to physical devices through downstream ports, including: Connecting the logical memory device and the logical cache device to corresponding memory expansion devices through a first downstream port; The logical cache device is connected to the first cache device through the second downstream port.

10. The method according to claim 1, characterized in that The cache coherence control module intercepts an operation request sent by the first cache device to the target address and performs protocol conversion on the operation request; queries the cache coherence directory to determine the cache line status of the target address and the physical device where the cache line is located; Determining target data and providing feedback based on the cache line state and the physical device where the cache line is located, including: intercepting an operation request initiated by the first cache device to the logical device, and converting the operation request into a target request for the corresponding memory expansion device; Querying a locally maintained cache coherence directory to determine a third cache device and a cache line status where a cache line corresponding to a target address indicated by the target request is located; According to the cache line state and the third cache device where the cache line is located, target data is determined and fed back.

11. The method according to claim 10, characterized in that The determining target data and feeding back the target data according to the cache line state and the third cache device where the cache line is located specifically includes: If the cache line exists in the third cache device and the state meets the consistency requirement of the target request, directly feeding back the target data to the first cache device; If the cache line exists in the third cache device but its status does not meet the consistency requirement of the target request, generate and send a snoop request to the third cache device, and select a target mode to process data transfer according to the scenario; The cache line states of the first cache device and the third cache device in the coherence directory are updated.

12. The method according to claim 11, characterized in that The process of selecting a target mode for processing data transmission according to a scenario specifically includes: In the write-back mode, after the third cache device writes the target data back to the corresponding memory expansion device, the memory expansion device returns the target data to the first cache device; In the direct transmission mode, the third cache device directly sends the target data to the first cache device through the direct transmission feature of the CXL protocol.

13. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 7 to 12.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 7 to 12.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 7 to 12 is implemented.

Images