Data exchange architecture, system and method enabling cache coherency

Abstract

The application discloses a data exchange architecture, system and method capable of realizing cache consistency, and the data exchange architecture comprises a data exchange device and a plurality of devices electrically connected with the data exchange device. The data exchange device comprises a shared storage space comprising memory units corresponding to the device IDs, a cache state array comprising the device IDs and the cache state array, and a management module. The cache state array comprises cache states corresponding to each device ID. The memory units with the cache state of exclusive writing cannot be written or read by other devices except the writing device. The memory units with the cache state of exclusive writing can be written or read by any device. Other devices cannot cache the data of the memory units of the corresponding device through data invalidation / uncaching. Other devices can cache the data of the memory units of the device with the cache state of shared reading. The application can realize the consistency of the cache among the plurality of devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer and communication technology, and in particular to a data exchange architecture, system and method capable of realizing cache consistency. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), deep learning (DL), high-performance computing (HPC) and big data processing technologies, the demand for data transmission is growing explosively, especially in high-bandwidth, low-latency data exchange scenarios. The traditional PCIe architecture faces many serious challenges such as insufficient bandwidth, delay and long data transmission path. To address these challenges, a technical means to solve these problems through shared memory is not lost.

[0003] However, the complexity and iteration frequency of data transmission are also growing explosively. Especially in multi-device data exchange scenarios, such as deep learning training, graphics rendering, high-performance scientific computing and other large-scale parallel systems, the cache consistency problem caused by data modification and concurrent access is particularly prominent in the data transmission mode based on shared memory.

[0004] For example, the memory of a GPU is frequently accessed in multi-device parallel computing, which can cause cache inconsistency problems. That is, when the data in the memory of a GPU is modified, the previous cache of other GPUs may not be updated in time, resulting in inconsistent or incorrect data. This is a typical synchronization problem in parallel computing environments, and the cache consistency problem becomes more prominent when multiple devices (such as multiple GPUs) need to read and write shared data simultaneously.

[0005] The disclosure of the above background art is only used to assist in understanding the inventive concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present application, nor does it necessarily provide technical teaching. In the absence of explicit evidence that the above content has been disclosed before the filing date of the present application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0006] The purpose of the present application is to provide a data exchange architecture, system and method capable of realizing cache consistency, which can realize cache consistency between multiple devices.

[0007] To achieve the above purpose, the technical solutions adopted by the present application are as follows:

[0008] A data exchange architecture capable of realizing cache consistency, comprising a data exchange device and a plurality of devices to be communicated electrically connected with the data exchange device, each of the devices being configured with a unique device ID;

[0009] The data exchange device comprises a shared storage space, a cache state array list and a management module, the shared storage space comprises a memory unit corresponding to each of the device IDs, the cache state array list comprises a cache state array corresponding to each of the device IDs, the cache state array comprises a cache state corresponding to each of the device IDs, and the cache state comprises exclusive writing, shared reading, data invalidation and uncaching; the exclusive writing means that the memory unit is being written by and can only be written by one device, the shared reading means that the device only reads data from the memory unit without modifying the data, the data invalidation means that the data read from the memory unit is invalid, and the uncaching means that the device does not read data from the memory unit;

[0010] For each memory unit, the management module is configured to determine the access right of each of the devices to the memory unit according to the cache state array corresponding to the memory unit, comprising:

[0011] When there is a cache state of exclusive writing in the cache state array, the memory unit cannot be written / read by devices other than the current writing device, and the device ID corresponding to the current writing device is consistent with the device ID corresponding to the exclusive writing;

[0012] When there is no cache state of exclusive writing in the cache state array, the memory unit can be written / read by one device;

[0013] The management module is further configured to determine the access right of other devices to the data of the memory unit cached by one device according to the cache state array, comprising:

[0014] For the cache state array corresponding to a memory unit, other devices cannot cache the data in the memory unit through the device corresponding to the cache state of data invalidation / uncaching;

[0015] For the cache state array corresponding to a memory unit, other devices can cache the data in the memory unit through the device corresponding to the cache state of shared reading.

[0016] Further, any of the technical solutions or a combination of the technical solutions described above, the management module is further configured to update the cache state array according to the current access state of the memory unit and the current cache state of the cache state array corresponding thereto;

[0017] The access state comprises reading data or writing data of an access object, and the access object is one or more of the devices.

[0018] Further, any one of the above technical solutions or a combination of the above technical solutions, for each of the memory units, the management module updates the cache state array by the following way:

[0019] If the memory unit is requested to be written data by a device, the management module modifies the cache state corresponding to the device ID in the cache state array to exclusive write, and the device is one of the devices;

[0020] Update the cache state corresponding to the other device ID to shared read to invalid state;

[0021] Maintain the cache state corresponding to the other device ID as uncached / data invalid.

[0022] Further, any one of the above technical solutions or a combination of the above technical solutions, for each of the memory units, the management module updates the cache state array by the following way:

[0023] If the request device completes writing data to the memory unit, the management module modifies the cache state corresponding to the request device ID in the cache state array from exclusive write to shared read.

[0024] Further, any one of the above technical solutions or a combination of the above technical solutions, for each of the memory units, the management module updates the cache state array by the following way:

[0025] If the request device completes writing data to the memory unit, the management module broadcasts a data update notification to other devices;

[0026] The other devices can optionally cache the data in the memory unit in response to receiving the data update notification;

[0027] If the device caches the data in the memory unit, the management module updates the cache state corresponding to the device ID to data sharing;

[0028] If the device does not cache the data in the memory unit, the management module maintains the cache state corresponding to the device ID unchanged.

[0029] Further, any one of the above technical solutions or a combination of the above technical solutions, for each of the memory units, the management module responds to the request device completing writing data to the memory unit, and for the cache state array corresponding to the memory unit, determines the device ID corresponding to the cache state of data sharing as the target device ID, and the management module broadcasts a data update notification to the device corresponding to the target device ID.

[0030] Further, any one of the above technical solutions or a combination of the above technical solutions, for each of the memory units, the management module updates the cache state array by the following way:

[0031] If the memory unit is requested by a device to read data, the management module modifies the cache state corresponding to the request device ID in the cache state array to shared reading, and the request device is one of the devices;

[0032] The cache states corresponding to other device IDs are maintained unchanged as data sharing / uncached / data invalid.

[0033] Further, any one of the above technical solutions or a combination of the above technical solutions, the shared memory space is a global address space based on DMA, and the memory unit is a DMA address, and the DMA address corresponding to the same device ID directly communicates with the physical memory address of the device;

[0034] The management module is also configured to transfer data in one DMA address to another DMA address.

[0035] Further, any one of the above technical solutions or a combination of the above technical solutions, further comprising an upstream switch, the upstream switch is respectively electrically connected with each of the devices, and the upstream switch is electrically connected with the CPU;

[0036] When two of the devices need to communicate, the two devices do not communicate with the CPU through the upstream switch, but communicate through the data transmission unit.

[0037] Further, any one of the above technical solutions or a combination of the above technical solutions, comprising a first device group and a second device group, the first device group and the second device group each comprising a plurality of devices;

[0038] The plurality of devices in the first device group are respectively electrically connected with a first upstream switch, and the plurality of devices in the second device group are respectively electrically connected with a second upstream switch, and the first upstream switch and the second upstream switch are respectively electrically connected with a CPU;

[0039] Each of the devices in the first device group and the second device group is respectively electrically connected with the data exchange device, and any two of the devices do not communicate through the first upstream switch, the second upstream switch and the CPU, but communicate through the data transmission unit.

[0040] Further, any one of the above technical solutions or a combination of the above technical solutions, each of the DMA addresses does not overlap; and / or,

[0041] the DMA address corresponding to different device IDs and the physical memory address of the device do not communicate; and / or,

[0042] The device is a PCIe device.

[0043] According to another aspect of the present application, a communication system is provided, which comprises the data exchange architecture capable of realizing cache coherence according to any one or combination of the technical solutions above.

[0044] According to another aspect of the present application, a data exchange method capable of realizing cache coherence is provided, which comprises the following steps:

[0045] A data exchange device is electrically connected with a plurality of devices to be communicated, each of the devices is configured with a unique device ID, the data exchange device comprises a shared storage space, a cache state array list and a management module, the shared storage space comprises a memory unit corresponding to each device ID, the cache state array list comprises a cache state array corresponding to each device ID, the cache state array comprises a cache state corresponding to each device ID, and the cache state comprises exclusive writing, shared reading, data invalidation and uncaching.

[0046] For each memory unit, the management module is configured to determine the access right of each device to the memory unit according to the cache state array corresponding to the memory unit, which comprises:

[0047] When there is a cache state of exclusive writing in the cache state array, the memory unit cannot be written / read by devices other than the current writing device, and the device ID corresponding to the current writing device is consistent with the device ID corresponding to exclusive writing;

[0048] When there is no cache state of exclusive writing in the cache state array, the memory unit can be written / read by any device;

[0049] The management module is further configured to determine the access right of other devices to the data of the memory unit cached by one device according to the cache state array, which comprises:

[0050] For the cache state array corresponding to a memory unit, other devices cannot cache the data in the memory unit through the device with the cache state of data invalidation / uncaching;

[0051] For the cache state array corresponding to a memory unit, other devices can cache the data in the memory unit through the device with the cache state of shared reading.

[0052] Further, any one of the technical solutions or the combination of multiple technical solutions mentioned above, the management module updates the cache state array according to the data exchange process, including:

[0053] For the cache state array corresponding to each memory unit, the management module updates the cache state array according to the current access state of the memory unit and the current cache state in the cache state array.

[0054] Further, any one of the technical solutions or the combination of multiple technical solutions mentioned above, for each memory unit, the management module updates the cache state array by the following way:

[0055] If the memory unit is requested by the device to write data, the management module modifies the cache state corresponding to the request device ID in the cache state array to exclusive write, and the request device is one of the devices;

[0056] Update the cache state corresponding to other device IDs to shared read to invalid state;

[0057] Maintain the cache state corresponding to other device IDs as uncached / data invalid.

[0058] Further, any one of the technical solutions or the combination of multiple technical solutions mentioned above, for each memory unit, the management module updates the cache state array by the following way:

[0059] If the request device completes writing data to the memory unit, the management module sends a data update notification to other devices;

[0060] Other devices can optionally cache data in the memory unit in response to receiving the data update notification;

[0061] If the device caches data in the memory unit, the management module updates the cache state corresponding to the device ID to data sharing;

[0062] If the device does not cache data in the memory unit, the management module maintains the cache state corresponding to the device ID unchanged.

[0063] The technical solutions provided by the application have the following beneficial effects:

[0064] a. The application can avoid the risk of data invalidation being accessed, multiple devices competing for the same memory unit, and other errors by configuring a shared memory space including memory units corresponding to each device in a data exchange device, a cache state array including a cache state array corresponding to each memory unit, and a cache state array including a cache state corresponding to each device, configuring a cache state including multiple states of exclusive write, shared read, data invalidation, and uncached, and using a management module to control the access rights of each device to each memory unit and the access rights of other devices to the previous cache of one device according to the cache state array, which can assist in implementing write synchronization, read redirection, and other consistency operations;

[0065] b. The application can reasonably adjust the cache state array corresponding to each memory unit according to the access state of each memory unit, and then effectively adjust the access rights of other devices to the memory unit, effectively avoiding the risk of parallel write and read errors caused by caching old values, by updating the cache state of the memory unit according to the access state of the memory unit and its corresponding cache state array through the management module;

[0066] c. The application not only adjusts the cache state array corresponding to the memory unit in time through the management module during the access of the memory unit, but also updates the cache state array corresponding to the memory unit again according to whether other devices synchronize the data of the memory unit after the access of the memory unit is completed, and broadcasts the update information of the memory unit to other devices through the management module, which can completely solve the cache consistency conflict problem caused by data modification and concurrent access, and is especially suitable for large-scale parallel systems such as deep learning training, graphics rendering, and high-performance scientific computing. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0068] Figure 1 The module schematic diagram of the data exchange architecture provided for an exemplary embodiment of the present application is shown in the figure.

[0069] Figure 2 The configuration principle schematic diagram of the cache state array provided for an exemplary embodiment of the present application is shown in the figure.

[0070] Figure 3Fig. 1 shows a schematic diagram of the principle of updating the cache state array 1 when the memory unit 1 is written data by the device 1 according to an exemplary embodiment of the present application;

[0071] Figure 4 Fig. 2 shows a schematic diagram of the principle of updating the cache state array 1 when the memory unit 1 is read data by the device 3 according to an exemplary embodiment of the present application;

[0072] Figure 5 Fig. 3 shows a schematic diagram of the principle of the first shared memory based data transmission between devices according to an exemplary embodiment of the present application;

[0073] Figure 6 Fig. 4 shows a schematic diagram of the principle of the second shared memory based data transmission between devices according to an exemplary embodiment of the present application;

[0074] Figure 7 Fig. 5 shows a schematic diagram of the working flow of the data exchange architecture when the device 1 needs to write data to the memory unit 1 according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the personnel in the field without creative labor should belong to the scope of protection of the present application.

[0076] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or apparatus including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or apparatuses.

[0077] In the shared memory based multi-device data exchange architecture, the shared memory will be frequently accessed and modified, and each GPU will read data from the shared memory to its own cache, which will cause the occurrence of cache consistency.

[0078] Specifically, if multiple GPUs share the same memory region, and each GPU caches its own data copy, cache coherence problems will occur, at least including the following aspects.

[0079] Data modification asynchronization: Suppose GPU1 modifies some data in the shared memory (for example, updates the weight of a model parameter), but this modification only exists in the just-updated shared memory. If GPU2 still uses the old data in the local cache instead of obtaining the updated data from the shared memory, data inconsistency will occur, which may cause incorrect calculation results.

[0080] Cache invalidation: After GPU1 modifies the data, GPU2 needs to know that the data in its cache has been invalidated so as to reload the latest data from the shared memory. If GPU2 is not timely notified to update the cache, GPU2 will continue to use outdated data, causing calculation problems.

[0081] Based on the above problems, in one embodiment of the present application, a data exchange architecture capable of realizing cache coherence is provided, as shown in Figure 1 、 Figure 2 、 Figure 7 and Table 1, which includes a data exchange device and a plurality of devices to be communicated electrically connected with the data exchange device, each of the devices is configured with a unique device ID, and the devices are optionally GPUs, ASICs, and other PCIe devices;

[0082] The data exchange device includes a shared storage space, a cache status array table (Cache Status Array, CSA), and a management module (Cache Coherence Monitor, CCM module). The shared storage space includes a memory unit corresponding to each device ID. The cache status array table includes a cache status array corresponding to each device ID. The cache status array includes a cache status corresponding to each device ID. The cache status includes Write-Owner (W), Read-Shared (R), Expired (X), and Not-Cached (N). Write-Owner means that the memory unit is being written by one device and can only be written by that device. Read-Shared means that the device only reads data from the memory unit without modifying the data. Expired means that the data read from the memory unit is invalid. Not-Cached means that the device does not read data from the memory unit.

[0083] For each memory unit, the management module is configured to determine the access rights of each device to the memory unit according to the cache status array corresponding to the memory unit, including:

[0084] When there is a cache state of exclusive write in the cache state array, the memory unit cannot be written / read by other devices except the current writing device, and the device ID corresponding to the current writing device is consistent with the device ID corresponding to the exclusive write;

[0085] When there is no cache state of exclusive write in the cache state array, the memory unit can be written / read by a device;

[0086] The management module is further configured to determine, according to the cache state array, the access right of other devices to the data of the memory unit cached by a device, comprising:

[0087] For the cache state array corresponding to a memory unit, other devices cannot cache the data in the memory unit through the device corresponding to the cache state of data invalidation / uncached;

[0088] For the cache state array corresponding to a memory unit, other devices can cache the data in the memory unit through the device corresponding to the cache state of shared reading.

[0089] Referring to Figure 3 , for the current cache state array of memory unit 1 corresponding to device 1 ID, the cache state corresponding to device 2 ID is R, which means that device 2 has shared reading data from memory unit 1 in advance, and the data of memory unit 1 cached by device 2 can be read by other devices, that is, device 2 can share the data in memory unit 1 read by it to other devices; the cache state corresponding to device 3 ID is X, which means that the data read from memory unit 1 in advance by device 3 has been invalidated (if there is a use requirement, the data should be updated in advance, and the invalidated data cannot be used), and the data of memory unit 1 cached by device 3 in advance cannot be read by other devices, that is, device 3 cannot share the data in memory unit 1 read by it to other devices; the cache state corresponding to device 4 ID is N, which means that device 4 has not cached the data read from memory unit 1, and other devices cannot read the data of memory unit 1 cached by device 4, or in other words, device 4 cannot share the data of memory unit 1 to other devices. Among them, the data corresponding to shared reading can be used or shared; but the data corresponding to data invalidation cannot be used or shared, and there is no data of the memory unit in the uncached state, so it is inevitable that it cannot be used / shared.

[0090] In the embodiment, first, in the process of configuring the data exchange device, an empty CSA table (indexed by device ID, i.e. block address) is created by initialization. After the start of each device is completed, the cache controller of each device actively registers with the CCM and establishes a state reporting channel. Each row structure in the cache state array in the CSA table is: block address (device ID), device 1 to device n (GPU0~GPUn) state bit field.

[0091] When the cache controller of each device accesses the memory unit, the cache block state of the current operation, i.e. the access request, is uploaded to the management module through the corresponding transaction layer, such as the state reporting transaction layer, during reading / writing. The management module queries the cache state array table for the current access request, which is reading / writing, and judges whether the current access request operation is allowed, including triggering invalidation / rejection notification or allowing reading / writing or read forwarding, etc.

[0092] Specifically, when any GPU (such as GPU1) proposes an access request to write data to the memory unit 1 of the shared memory, the data exchange device first sends the access request of GPU1 to the management module; the management module queries each cache state in the cache state array 1 corresponding to the GPU1 ID in the cache state array table. If there is an exclusive write cache state corresponding to the other GPU ID in each cache state, the management module returns a write rejection signal to GPU1; if there is no exclusive write cache state corresponding to the other GPU ID, the management module returns a write permission signal to GPU1.

[0093] Specifically, when any GPU (such as GPU1) proposes an access request to read data from the memory unit 1 of the shared memory, the data exchange device first sends the access request of GPU1 to the management module in the management module; the management module queries each cache state in the cache state array 1 corresponding to the GPU1 ID in the cache state array table. If there is an exclusive write cache state corresponding to the other GPU ID in each cache state, the management module returns a read rejection signal to GPU1; if there is no exclusive write cache state corresponding to the other GPU ID, the management module returns a read permission signal to GPU1.

[0094]

[0095] In the embodiment, the management module is further configured to update the cache state array according to the current access state of the memory unit and the current cache state of the cache state array corresponding thereto; the access state includes reading data or writing data of the access object, and the access object is one or more of the plurality of devices.

[0096] Specifically, for each of the memory units, the management module updates the cache status array in the following manner.

[0097] If the memory cell is being written to by the requesting device, the management module modifies the cache status corresponding to the requesting device ID in the cache status array to exclusive write, where the requesting device is one of the devices. It should be noted that if the memory cell is currently being written to by another device, then according to Table 1, the memory cell cannot be written to by the requesting device. That is, for the same memory cell, there can only be one cache status in the cache status array that is exclusive write. This prevents data errors caused by two devices modifying the same memory cell at the same time.

[0098] like Figure 3 As shown, in the cache status array 1 corresponding to memory unit 1, cache status 1 to cache status n are the access statuses of devices 1 to n to memory unit 1, respectively. The current access status of memory unit 1 is data being written by device 1. It should be noted that, for cache status array 1 corresponding to memory unit 1, when the current cache status corresponding to device 1 ID is shared read / uncached / data invalid (R / N / X) and the cache status corresponding to other device IDs does not have exclusive write (W), then device 1 can write data to memory unit 1. Based on the current access status of memory unit 1, the management module sets cache status 1 in the cache status array 1 corresponding to memory unit 1 to exclusive write (W) and updates the cache status corresponding to other devices in cache status array 1.

[0099] exist Figure 3 In the example, if cache status 2 is shared read (R), it means that device 2 has previously read data from memory unit 1 and the data is shared. Since new data is currently being written into memory unit 1 by device 1, in order to ensure cache data consistency between multiple devices, including that the data read by device 2 from memory unit 1 is not used by itself and is not further read by other devices, cache status 2 needs to be changed from shared read (R) status to data invalidation (X) status.

[0100] After device 1 completes writing to memory unit 1, the management module changes the cache state 1 corresponding to the requesting device ID in the cache state array from exclusive write to shared read, and the management module sends a data update notification to other devices. In response to receiving the data update notification, the other devices may selectively cache the data in the memory unit.

[0101] For example, if device 2 receives the data update notification and caches the data in memory unit 1, the management module updates the cache state 2 corresponding to device 2 to data sharing. In this way, when the data in memory unit 1 is updated, other devices that previously cached the data in memory unit 1 can also update the data synchronously.

[0102] Alternatively, device 2 responds to receiving the data update notification but does not cache the data in memory unit 1. For example, if device 2 no longer needs to use the data in memory unit 1, it may choose not to cache it. In this case, cache status 2 corresponding to device 2 maintains the data invalid (X). This ensures data cache consistency without adding additional system overhead.

[0103] exist Figure 3 In the example, cache status 3 in cache status array 1 is "Data Stale" (X), indicating that device 3 previously cached data in memory unit 1. However, after the data in memory unit 1 was updated, device 3 did not synchronize its cached data from memory unit 1 (as described above for device 2). While in this "Data Stale" state, the previously cached data in memory unit 1 cannot be read by other devices. Furthermore, because memory unit 1 is currently being written to by device 1, meaning the previously cached data is being updated, the management module maintains cache status 3 in cache status array 1 as "Data Stale" (X) in either case to prevent other devices from caching outdated, erroneous data.

[0104] Similarly, cache state 4 in cache state array 1 is not cached (N), indicating that device 4 has not previously cached the data in memory unit 1. Since device 4 has not previously cached the data in memory unit 1, the management module maintains cache state 4 in cache state array 1 as not cached (N) to prevent other devices from caching delayed or erroneous data.

[0105] For each of the memory units, the management module also updates the cache status array in the following manner: if the memory unit is read by a requesting device, the management module modifies the cache status corresponding to the requesting device ID in the cache status array to shared read, and the requesting device is one of the devices; and maintains the cache status corresponding to other device IDs as data sharing / uncached / data invalid.

[0106] like Figure 4As shown, in the cache state array 1 corresponding to the memory unit 1, the cache state 1 to the cache state n are the access states of the devices 1 to the devices n to the memory unit 1. The current access state of the memory unit 1 is that the data is read by the device 3, and it needs to be noted that, for the cache state array 1 corresponding to the memory unit 1, when the cache state corresponding to each device ID is not exclusively written (W), then the device 3 can read data from the memory unit 1. The management module modifies the cache state 1 in the cache state array from shared read / data invalidation / uncached (R / X / N) to shared read (R).

[0107] For the cache states corresponding to other device IDs, only three states of shared read, data invalidation, and uncached are possible, such as Figure 4 As shown, the management module maintains the cache states corresponding to other device IDs as data sharing / uncached / data invalidation unchanged.

[0108] As above, in the present application, the management module serves as the control core of the data exchange device, responsible for coordinating the cache states of each GPU, and the cache state array table, namely the CSA table, is the database of the real-time cache states of each device to each memory unit. In each data reading / writing step, the management module performs table lookup operation, update operation, broadcast and state transition operation on the cache state array table, thereby efficiently realizing the data cache consistency of multiple devices.

[0109] Of course, in another embodiment of the present application, for each memory unit, the management module determines, in response to the request device completing the write operation on the corresponding memory unit, that the cache state corresponding to the cache state array of the memory unit is the target device ID corresponding to the device ID of data sharing, and broadcasts a data update notification to the device corresponding to the target device ID. In response to receiving the data update notification, the device reads the latest data in the memory unit, that is, the corresponding previous cache data is received after the broadcast data update notification. The difference between this embodiment and the above-mentioned embodiment is that, in this embodiment, after the data in a memory unit is updated, the management module does not broadcast a data update notification to all other devices, but only broadcasts to the devices that have cached the memory unit, and the devices receiving the broadcast data update notification will definitely update the corresponding previous cache to ensure the data cache consistency.

[0110] For the data exchange architecture capable of realizing cache consistency according to any of the above-mentioned embodiments, the shared storage space is preferably a global address space based on DMA, the memory unit is a DMA address, and the DMA address corresponding to the same device ID directly communicates with the physical memory address of the device. Preferably, each DMA address does not overlap.

[0111] In the embodiment, the data exchange device further comprises a global address mapping table, which comprises one-to-one device ID and DMA address, and the DMA addresses corresponding to the same device ID are directly communicated with the physical memory addresses.

[0112] When a request device sends an access request to the data exchange device, the access request comprises a request device ID, a target device ID and an access request type. The management module determines the DMA address corresponding to the target device ID in the global address mapping table as a target DMA address, and realizes the data transmission operation from the request device to the target device through the target DMA address.

[0113] In the embodiment, the data exchange architecture further comprises an uplink switch, which is respectively electrically connected with each of the devices, and the uplink switch is electrically connected with the CPU; when two of the devices need to communicate, the two devices do not communicate with the CPU through the uplink switch, but communicate through the data transmission unit.

[0114] The data transmission operation from the request device to the target device through the target DMA address includes two implementation manners of reading and writing. One memory sharing implementation manner is shown in Figure 5 Each of the DMA addresses is a memory unit, the DMA addresses corresponding to the same device ID are directly communicated with the physical memory addresses, and the physical memory addresses and the DMA addresses corresponding to different device IDs cannot be directly communicated, and need to be forwarded through the management module.

[0115] For example, when device 1 needs to send data to device 4, device 1 sends an access request to the management module, that is, the access request type is writing, the management module determines the target DMA address as DMA address 4 based on the target device ID in the access request, and finds that each cache state in the cache state array 4 corresponding to the DMA address 4 in the CAS table has no exclusive writing (it needs to be noted that because DMA address 1 can only be written by device 1, it is not necessary to query each cache state of the cache state array 1 corresponding to the DMA address 1 when determining whether to agree to the access request), then the management module returns an access agreement signal to device 1 and sends the access request to device 4. Otherwise, the management module returns a refusal access signal to device 1.

[0116] The device 1 responds to the receiving of the access permission signal, and then the data in the physical memory address of the device 1 is directly moved to the DMA address 1 (equivalent to the memory unit 1) by the DMA controller corresponding to the device 1. In this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array according to the access state of the DMA address 1 and the current cache state array. The management module moves the data in the DMA address 1 to the DMA address 4, and in this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array according to the access state of the DMA address 4 and the current cache state array. The data in the DMA address 4 is directly moved to the physical memory address of the device 4 by the DMA controller corresponding to the device 4, and in this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array according to the access state of the DMA address 4 and the current cache state array.

[0117] For example, when the device 1 needs to read data from the device 4, the device 1 sends an access request to the management module, and the access request type is reading. The management module determines the target DMA address as the DMA address 4 based on the target device ID in the access request, and finds that each cache state in the cache state array 4 corresponding to the DMA address 4 in the CAS table does not have exclusive writing (it needs to be noted that because the DMA address 1 can only be written by the device 1, it is not necessary to query each cache state in the cache state array 1 corresponding to the DMA address 1 when determining whether to approve the access request). Then, the management module returns an access permission signal to the device 1 and sends the access request to the device 4. Otherwise, the management module returns a denial access signal to the device 1.

[0118] In response to receiving the access request, the device 4's corresponding DMA controller directly moves the data in the physical memory address of the device 4 to the DMA address 4. During this process, as in the data cache consistency control method described in the above embodiment, the management module updates its cache status array based on the access status of the DMA address 4 and its current cache status array. The management module moves the data in the DMA address 4 to the DMA address 1. During this process, as in the data cache consistency control method described in the above embodiment, the management module updates its cache status array based on the access status of the DMA address 1 and its current cache status array. In response to receiving the access consent signal, the device 1 directly moves the data in the DMA address 1 (equivalent to the memory unit 1) to the physical memory address of the device 1 through the DMA controller corresponding to the device 1. During this process, as in the data cache consistency control method described in the above embodiment, the management module updates its cache status array based on the access status of the DMA address 1 and its current cache status array.

[0119] Another way to implement memory sharing is as follows Figure 6 As shown, each of the DMA addresses serves as a memory unit, and each of the devices can perform read / write operations on the DMA address corresponding to any device ID. In this manner, data forwarding in different DMA addresses does not need to be performed through a management module.

[0120] For example, when device 1 needs to send data to device 4, device 1 sends an access request to the management module, i.e., the access request type is write. The management module determines that the target DMA address is DMA address 4 based on the target device ID in the access request, and searches the cache status array 4 corresponding to DMA address 4 in the CAS table to find out whether each cache status does not have exclusive write (it should be noted that because data in the physical memory address of device 1 can be written directly to DMA address 4 without being transferred through DMA address 1, it is not necessary to query the cache status array 1 corresponding to DMA address 1 when determining whether the access request can be approved). The management module then returns an access approval signal to device 1 and sends the access request to device 4. Otherwise, the management module returns an access rejection signal to device 1.

[0121] The device 1 responds to the receiving of the access permission signal, and then the corresponding DMA controller of the device 1 directly carries the data in the physical memory address of the device 1 to the DMA address 4 (equivalent to the memory unit 4). In this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array of the DMA address 4 according to the access state of the DMA address 4 and the current cache state array of the DMA address 4. The device 4 responds to the receiving of the access request, and then the corresponding DMA controller of the device 4 directly carries the data in the DMA address 4 to the physical memory address of the device 4. In this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array 4 according to the access state of the DMA address 4 and the current cache state array of the DMA address 4.

[0122] For example, when the device 1 needs to read data from the device 4, the device 1 sends an access request to the management module, and the access request type is reading. The management module determines the target DMA address as the DMA address 4 (equivalent to the memory unit 4) based on the target device ID in the access request, and finds that each cache state in the cache state array 4 corresponding to the DMA address 4 in the CAS table is not exclusive writing (it should be noted that, because the physical memory address of the device 1 can directly read the data in the DMA address 4, it does not need to be transferred through the DMA address 1, and therefore, when determining whether to approve the access request, it does not need to query each cache state in the cache state array 1 corresponding to the DMA address 1). Then, the management module returns an access permission signal to the device 1 and sends the access request to the device 4. Otherwise, the management module returns a denial access signal to the device 1.

[0123] The device 4 responds to the receiving of the access request, and then the corresponding DMA controller of the device 4 directly carries the data in the physical memory address of the device 4 to the DMA address 4. In this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array of the DMA address 4 according to the access state of the DMA address 4 and the current cache state array of the DMA address 4. The device 1 responds to the receiving of the access permission signal, and then the corresponding DMA controller of the device 1 directly carries the data in the DMA address 4 (equivalent to the memory unit 1) to the physical memory address of the device 1. In this process, the data cache consistency control method as described in the above embodiment is used, and the management module updates the cache state array 4 according to the access state of the DMA address 1 and the current cache state array of the DMA address 1.

[0124] It should be noted that in the above two implementation examples, when the device 1 needs to read data from the device 4, if the cache state corresponding to the device 1 ID in the cache state array 4 corresponding to the DMA address 4 is data invalid / uncached, the management module can also directly return a denial of access signal to the device 1.

[0125] In another embodiment of the present application, the data exchange architecture comprises a first device group and a second device group, and each of the first device group and the second device group comprises a plurality of devices. The plurality of devices in the first device group are respectively electrically connected with a first uplink switch, the plurality of devices in the second device group are respectively electrically connected with a second uplink switch, and the first uplink switch and the second uplink switch are respectively electrically connected with a CPU. Each of the devices in the first device group and the second device group is respectively electrically connected with the data exchange device, and any two devices do not communicate through the first uplink switch, the second uplink switch and the CPU, but communicate through the data transmission unit. The first uplink switch and the second uplink switch can be electrically connected with the same CPU or different CPUs.

[0126] In one embodiment of the present application, a communication system is also provided, and the communication system comprises the data exchange architecture capable of realizing cache consistency according to any one of the above embodiments or a combination of multiple embodiments.

[0127] In one embodiment of the present application, a data exchange method capable of realizing cache consistency is also provided, and as shown in the method comprises the following steps: Figure 7

[0128] The data exchange device is configured to be respectively electrically connected with a plurality of devices to be communicated, each of the devices is configured with a unique device ID, the data exchange device comprises a shared storage space, a cache state array list and a management module, the shared storage space comprises a memory unit corresponding to the device ID, the cache state array list comprises a cache state array corresponding to the device ID, the cache state array comprises a cache state corresponding to each of the device IDs, and the cache state comprises exclusive writing, shared reading, data invalidation and uncaching.

[0129] For each memory unit, the management module is configured to determine the access authority of each of the devices to the memory unit according to the cache state array corresponding to the memory unit, including:

[0130] ​When there is a cache state of exclusive write in the cache state array, the memory unit cannot be written / read by other devices except the current write device, and the device ID corresponding to the current write device is consistent with the device ID corresponding to the exclusive write;

[0131] When there is no cache state of exclusive write in the cache state array, the memory unit can be written / read by any device;

[0132] The management module is further configured to determine, according to the cache state array, access rights of other devices to data of a memory unit cached by a device, comprising:

[0133] For a cache state array corresponding to a memory unit, other devices cannot cache data in the memory unit through a device corresponding to a cache state of data invalidation / uncached;

[0134] For a cache state array corresponding to a memory unit, other devices can cache data in the memory unit through a device corresponding to a cache state of shared reading.

[0135] In this embodiment, the management module updates the cache state array according to a data exchange process, comprising: for a cache state array corresponding to each memory unit, the management module updates the cache state array according to the current access state of the memory unit and the current cache state in the cache state array.

[0136] Specifically, for each memory unit, the management module updates the cache state array in the following manner: if the memory unit is requested to be written by a device, the management module modifies the cache state corresponding to the device ID of the requesting device in the cache state array to exclusive write, the requesting device being one of the devices; the management module updates the cache state corresponding to the device ID of other devices to shared reading to invalid state; and the management module maintains the cache state corresponding to the device ID of other devices to uncached / data invalid.

[0137] For each memory unit, the management module updates the cache state array in the following manner: if the requesting device completes writing to the memory unit, the management module sends a data update notification to other devices; other devices, in response to receiving the data update notification, can selectively cache data in the memory unit; if the device caches data in the memory unit, the management module updates the cache state corresponding to the device ID to data sharing; and if the device does not cache data in the memory unit, the management module maintains the cache state corresponding to the device ID unchanged.

[0138] In one embodiment of the present application, a data exchange device capable of realizing cache coherence is also provided, which comprises the shared memory space, the cache state array table and the management module as described in the above embodiment. The shared memory space comprises memory units corresponding to the device IDs one by one, the data exchange device comprises a plurality of ports, the ports are high-speed communication ports and support high-bandwidth parallel communication, the management module is electrically connected with each of the ports respectively, and the ports are configured to be electrically connected with devices to be communicated, such as PCIe devices.

[0139] The management module controls the access right of each of the devices to the data in each of the memory units based on the access request of each of the devices and the cache state array table. The working principle and working mode of the data exchange device are the same as those of the above-mentioned data exchange architecture capable of realizing cache coherence, and will not be described again.

[0140] It should be noted that the communication system, the data exchange method capable of realizing cache coherence and the data exchange device capable of realizing cache coherence provided by the present application have the same inventive concept as the above-mentioned data exchange architecture capable of realizing cache coherence, and the entire content of the data exchange architecture capable of realizing cache coherence is incorporated into the communication system, the data exchange method capable of realizing cache coherence and the data exchange device capable of realizing cache coherence by introduction.

[0141] It should be noted that in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, and do not necessarily require or imply that these entities or actions exist in any such actual relationship or order. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or other elements inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0142] The above description is only a specific embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A data exchange system capable of cache coherency, characterized in that, The data exchange device and a plurality of devices to be communicated electrically connected with the data exchange device, each of the devices is configured with a unique device ID; The data exchange device includes a shared memory space, a cache state array list and a management module, the shared memory space includes a memory unit corresponding to the device ID, the cache state array list includes a cache state array corresponding to the device ID; The cache state array includes a cache state corresponding to each of the device IDs, and the cache state includes exclusive writing, shared reading, data invalidation and uncached; For each memory unit, the management module is configured to determine the access authority of each device to the memory unit according to the cache state array corresponding to the memory unit, including: when there is an exclusive writing cache state in the cache state array, the memory unit cannot be written / read by other devices except the current writing device, and the device ID corresponding to the current writing device is consistent with the device ID corresponding to the exclusive writing; when there is no exclusive writing cache state in the cache state array, the memory unit can be written / read by any device; The management module is further configured to determine the access authority of other devices to the data of a device cached memory unit according to the cache state array, including: for a cache state array corresponding to a memory unit, other devices cannot cache the data in the memory unit through the device corresponding to the cache state of data invalidation / uncached; for a cache state array corresponding to a memory unit, other devices can cache the data in the memory unit through the device corresponding to the cache state of shared reading; The management module is further configured to update the cache state array according to the current access state of the memory unit and the current cache state of the cache state array corresponding thereto; the access state includes reading data or writing data of an access object, and the access object is one or more of the plurality of devices; For each of the memory units, the management module updates the cache state array by: if the requesting device completes writing data to the memory unit, the management module broadcasts a data update notification to other devices; other devices can selectively cache the data in the memory unit in response to receiving the data update notification; if the device caches the data in the memory unit, the management module updates the cache state corresponding to the device ID to data sharing; if the device does not cache the data in the memory unit, the management module maintains the cache state corresponding to the device ID unchanged.

2. The data exchange system capable of cache coherency as claimed in claim 1 wherein, For each of the memory units, the management module updates the cache state array by: If the memory unit is written by the requesting device, the management module modifies the cache state corresponding to the requesting device ID in the cache state array to exclusive writing, and the requesting device is one of the devices; Update the cache state corresponding to the other device ID to invalid state; The cache state corresponding to other device IDs remains unchanged.

3. The data exchange system capable of cache coherency as claimed in claim 2 wherein, For each memory unit, the management module updates the cache state array in the following manner: If the requesting device completes writing data to the memory unit, the management module modifies the cache state corresponding to the requesting device ID in the cache state array from exclusive writing to shared reading.

4. The data exchange system capable of cache coherency as claimed in claim 2 wherein, For each memory unit, the management module determines, in response to the requesting device completing writing data to the memory unit, the target device ID corresponding to the cache state as the device ID corresponding to the data sharing, and broadcasts a data update notification to the device corresponding to the target device ID.

5. The data exchange system capable of cache coherency implementation of claim 1, wherein, For each memory unit, the management module updates the cache state array in the following manner: If the memory unit is read by the requesting device, the management module modifies the cache state corresponding to the requesting device ID in the cache state array to shared reading, the requesting device being one of the devices; The cache state corresponding to other device IDs remains unchanged.

6. The data exchange system capable of cache coherency implementation of claim 1, wherein, The shared memory space is a DMA-based global address space, the memory unit is a DMA address, and the DMA address corresponding to the same device ID directly communicates with the physical memory address of the device. The management module is further configured to transfer data in one DMA address to another DMA address.

7. The data exchange system capable of cache coherency as claimed in claim 6 wherein, The communication system further comprises an uplink switch, the uplink switch being electrically connected to each of the devices, and the uplink switch being electrically connected to the CPU. When two devices need to communicate, the two devices do not communicate with the CPU through the uplink switch, but communicate through the data transmission unit.

8. The data exchange system capable of cache coherency implementation of claim 7, wherein, The communication system comprises a first device group and a second device group, each of the first device group and the second device group comprising a plurality of devices. The plurality of devices in the first device group are electrically connected to a first uplink switch, and the plurality of devices in the second device group are electrically connected to a second uplink switch, the first uplink switch and the second uplink switch being electrically connected to the CPU. Each of the devices in the first device group and the second device group is electrically connected to the data exchange device, and any two devices do not communicate through the first uplink switch, the second uplink switch, and the CPU, but communicate through the data transmission unit.

9. The data exchange system capable of cache coherency implementation of claim 6, wherein, Each of the DMA addresses does not overlap; and / or, The DMA addresses corresponding to different device IDs do not communicate with the physical memory address of the device; and / or, The devices are PCIe devices.

10. A communication system, characterized by The communication system comprises the data exchange system capable of realizing cache consistency according to any one of claims 1 to 9.

11. A data exchange method capable of realizing cache coherency, characterized by, The data exchange method based on the data exchange system capable of realizing cache consistency according to claim 1 comprises the following steps: A data exchange device is configured to be electrically connected with a plurality of devices to be communicated, each of the devices is configured with a unique device ID, the data exchange device comprises a shared storage space, a cache state array list and a management module, the shared storage space comprises a memory unit corresponding to each of the device IDs, the cache state array list comprises a cache state array corresponding to each of the device IDs, the cache state array comprises a cache state corresponding to each of the device IDs, and the cache state comprises exclusive writing, shared reading, data invalidation and uncaching; For each memory unit, the management module is configured to determine the access right of each of the devices to the memory unit according to the cache state array corresponding to the memory unit, including: When there is an exclusive writing cache state in the cache state array, the memory unit cannot be written / read by devices other than the current writing device, and the device ID corresponding to the current writing device is consistent with the device ID corresponding to the exclusive writing; When there is no exclusive writing cache state in the cache state array, the memory unit can be written / read by any device; The management module is further configured to determine the access right of other devices to the data of the memory unit cached by one device according to the cache state array, including: For a cache state array corresponding to a memory unit, other devices cannot cache the data in the memory unit through the device corresponding to the cache state of data invalidation / uncaching; For a cache state array corresponding to a memory unit, other devices can cache the data in the memory unit through the device corresponding to the cache state of shared reading.

12. The data exchange method capable of realizing cache coherency according to claim 11, characterized in that, The management module updates the cache state array list according to the data exchange process, including: For each cache state array corresponding to each memory unit, the management module updates the cache state array according to the current access state of the memory unit and the current cache state in the cache state array.

13. The data exchange method capable of realizing cache coherency according to claim 12, characterized in that, For each memory unit, the management module updates the cache state array in the following way: If the memory unit is requested to be written by a device, the management module modifies the cache state corresponding to the request device ID in the cache state array to exclusive writing, and the request device is one of the devices; Update the cache state corresponding to other device IDs to shared reading to invalid state; Maintain the cache state corresponding to other device IDs as uncaching / data invalidation.

14. The data exchange method capable of realizing cache coherency according to claim 12, characterized in that, For each memory unit, the management module updates the cache state array in the following way: If the request device completes writing data to the memory unit, the management module sends a data update notification to other devices; Other devices can optionally cache the data in the memory unit in response to receiving the data update notification; If the device caches the data in the memory unit, the management module updates the cache state corresponding to the device ID to data sharing; If the device does not cache data in the memory unit, the management module maintains the cache state corresponding to the device ID unchanged.

Images