Distributed data processing method, computing device and computing system

Abstract

The invention relates to the field of computing, and discloses a distributed data processing method, which is applied to a target computing device in a computing system, and the computing system comprises a plurality of computing devices and a plurality of CXL memory devices. Each CXL memory device is connected with at least two computing devices; the method comprises the following steps: receiving a data writing request sent by first computing equipment; the data writing request comprises the number of copies of the shared data and the size of the shared data; based on the number of copies of the shared data and the size of the shared data, determining a plurality of second CXL memory devices of which the capacities of free memory areas are matched with the size of the shared data from a plurality of first CXL memory devices connected with the first computing device; the number of the second CXL memory devices is equal to the number of copies of the shared data; and sending the plurality of first memory addresses to the first computing device, so that the first computing device writes the shared data into the plurality of first memory regions based on the plurality of first memory addresses. The storage cost of distributed data can be reduced.

Description

Technical Field

[0001] This application relates to the field of computing, and in particular, to a method for processing distributed data, a computing device, and a computing system. Background Art

[0002] Distributed data storage refers to storing data in multiple servers. This storage method can meet the needs of large-scale data storage and processing, providing higher reliability, scalability, and performance. In a distributed system, a data replication strategy is usually adopted to replicate each shared data to multiple server nodes. For example, each shared data can be replicated to all server nodes, so that each server node can directly obtain the shared data from its own storage area when reading or writing the shared data; or, each shared data can be replicated to some server nodes, so that each server node can read the shared data in other server nodes through network transmission. Summary of the Invention

[0003] Embodiments of this application provide a method for processing distributed data, a computing device, and a computing system, which can reduce the storage cost of distributed data.

[0004] In a first aspect, embodiments of this application provide a method for processing distributed data, which is applied to a target computing device. The target computing device is any computing device in a computing system. The computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices. Among them, each CXL memory device in the multiple CXL memory devices is connected to at least two computing devices. The method includes: receiving a data write request sent by a first computing device. The first computing device is any computing device in the computing system. The data write request is used to request to write shared data into a first CXL memory device. The first CXL memory device is a CXL memory device in the multiple CXL memory devices that is connected to the first computing device. The data write request includes the number of copies of the shared data and the size of the shared data. The number of copies is a positive integer greater than 1. Based on the number of copies of the shared data and the size of the shared data, determine multiple second CXL memory devices from the multiple first CXL memory devices. The second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data. The number of second CXL memory devices is equal to the number of copies of the shared data. Send multiple first memory addresses corresponding to multiple first memory areas to the first computing device, so that the first computing device writes the shared data into the multiple first memory areas based on the multiple first memory addresses. The first memory area is a free memory area in the second CXL memory device for writing the shared data.

[0005] Based on this implementation, the target computing device can uniformly manage the memory areas of multiple CXL memory devices, and can allocate memory areas in multiple CXL memory devices for each first computing device to write the shared data of the application program into different CXL memory devices. When each second computing device needs to read the shared data, the target computing device can determine the CXL memory device that can be read and the corresponding area information according to the identifier provided by the second computing device, so that the second computing device can read the required shared data in the corresponding CXL memory device. Thereby, it is possible to reduce the storage space consumed by distributed storage of data, and further reduce the storage cost of distributed data.

[0006] In a possible implementation, before determining multiple second CXL memory devices from multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data, the method further includes: obtaining the connection relationship between each CXL memory device in the computing system and each computing device in the multiple computing devices; and determining multiple first CXL memory devices from the multiple CXL memory devices according to the connection relationship.

[0007] Based on this implementation, the target computing device can obtain the topology of the cluster to determine the connection relationship between each CXL memory device and each computing device.

[0008] In another possible implementation, determining multiple second CXL memory devices from multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data includes: obtaining the number of third CXL memory devices; wherein the third CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; determining whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; and in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, selecting multiple second CXL memory devices from the multiple third CXL memory devices based on the number of copies of the shared data.

[0009] Based on this implementation, before determining the first memory area corresponding to each second CXL memory device, it is possible to first determine the number of each second CXL memory device, the size of the free memory area, and the connection relationship of the cluster, so as to ensure that the number of second CXL memory devices is equal to the number of copies of the shared data, and there is sufficient memory space in each second CXL memory device to store the shared data; and the cluster topology needs to meet the preset connection relationship to ensure that each computing device can read the shared data written by other computing devices in the CXL memory device connected to it.

[0010] In yet another possible implementation, the method further includes: receiving a data locking request; the data locking request is used to indicate setting the attribute of the shared data to read-only; determining whether the data locking request originates from a first computing device; and in the case where the data locking request originates from the first computing device, setting the attribute of the shared data to read-only.

[0011] Based on this implementation, the target computing device can change the read status corresponding to the shared data in the CXL memory device; and, the memory management device can verify whether the computing device is the source computing device corresponding to the shared data, so as to prevent other computing devices other than the source computing device from tampering with the read status of the shared data, thereby ensuring the security of the shared data in the CXL memory device.

[0012] In yet another possible implementation, the method further includes: receiving a data reading request sent by a second computing device; wherein, the second computing device is any computing device in the computing system; the data reading request is used to indicate reading the shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used to identify the shared data; determining a plurality of first memory addresses from the associated data based on the identifier of the shared data; wherein, the associated data includes the identifier of the shared data and a plurality of first memory addresses; determining a second memory address from the plurality of first memory addresses; wherein, the second memory address is the address of a second memory area; the second memory area is the shortest access path for the second computing device to read the shared data; and returning the second memory address to the second computing device, so that the second computing device reads the shared data from the second memory area based on the second memory address.

[0013] Based on this implementation, the second computing device can implement reading of the shared data in the CXL memory device, so that the computing device can quickly read the shared data in the CXL memory device, thereby improving the reading speed of the shared data.

[0014] In yet another possible implementation, before determining the second memory address from the plurality of first memory addresses, the method further includes: determining whether the shared data exists based on the identifier of the shared data; in the case where the shared data exists, determining whether the attribute of the shared data is read-only; determining the second memory address from the plurality of first memory addresses includes: in the case where the attribute of the shared data is read-only, determining the second memory address from the plurality of first memory addresses.

[0015] Based on this implementation, after receiving the data reading request of the second computing device and the data identifier corresponding to the shared data to be read, the target computing device can first verify the data identifier and the read status of the shared data, so as to prevent the computing device from reading incomplete shared data.

[0016] In yet another possible implementation, the method further includes: receiving a data reading failure response sent by a second computing device; wherein the data reading failure response is used to indicate that the second computing device fails to read shared data; determining a third memory area from a second CXL memory device corresponding to a second memory address; wherein the third memory area is used to write the shared data; sending a data recovery command to a first computing device; wherein the data recovery command is used to indicate the recovery of the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address for reading the shared data to be recovered; the third memory address is used to indicate the memory address of the third memory area for writing the shared data.

[0017] Based on this implementation, in the case where the second computing device fails to read the shared data, the second computing device can request the target computing device to recover the shared data, and the memory management device can interact with the first computing device that writes the shared data to implement the recovery of the shared data in the CXL memory device, so that the second computing device can successfully read the complete shared data.

[0018] In yet another possible implementation, the method further includes: receiving a data recovery success response sent by a first computing device; wherein the data recovery success response is used to indicate that the shared data is successfully recovered; sending the third memory address to the second computing device, so that the second computing device reads the shared data based on the third memory address.

[0019] Based on this implementation, after the first computing device writes the shared data to the third memory area corresponding to the second CXL memory device, it can send a data recovery success response to the target computing device to notify that the shared data has been recovered.

[0020] In yet another possible implementation, the method further includes: updating the second memory address in the associated data based on the third memory address.

[0021] Based on this implementation, after recovering the shared data, the target computing device can update the memory address storing the shared data.

[0022] In yet another possible implementation, the method further includes: receiving a data reading success response sent by a second computing device; wherein the data reading success response is used to indicate that the shared data is successfully read; increasing the number of dependency relationships of the shared data in accordance with a preset step.

[0023] Based on this implementation manner, when the second computing device reads the shared data in the target CXL memory device, it can send a data reading success response to the target computing device to notify the memory management module that the second computing device has successfully read the shared data. The target computing device can receive the data reading success response and increase the number of dependency relationships (i.e., occupancy number) of the shared data in accordance with a preset step, so as to indicate that there is a computing device in the cluster that is reading the shared data.

[0024] In yet another possible implementation manner, the method further includes: receiving a data release request sent by the second computing device; wherein, the data release request is used to indicate the release of the occupancy of the shared data by the second computing device; reducing the number of dependency relationships of the shared data in accordance with a preset step.

[0025] Based on this implementation manner, after receiving the data release request sent by the second computing device, the target computing device can reduce the number of dependency relationships of the shared data in accordance with a preset step to release the occupancy of the shared data by the second computing device.

[0026] In yet another possible implementation manner, the method further includes: receiving a data deletion request; wherein, the data deletion request is used to indicate deleting the shared data from the first memory area; the data deletion request includes the identifier of the shared data; determining whether the identifier of the shared data exists, whether the data deletion request comes from the first computing device, and whether the number of dependency relationships of the shared data is zero; when the identifier of the shared data exists, the data deletion request comes from the first computing device, and the number of dependency relationships of the shared data is not zero, sending the identifier of the shared data, a plurality of first memory addresses, and a deletable flag to the first computing device, so that the first computing device deletes the shared data based on the plurality of first memory addresses; wherein, the deletable flag is used to indicate that the shared data is deletable.

[0027] Based on this implementation manner, the first computing device can implement the deletion of the shared data in the CXL memory device through interaction with the target computing device; and the target computing device can verify whether the computing device is the source computing device corresponding to the shared data and the occupancy number corresponding to the shared data, so as to prevent other computing devices other than the source computing device from tampering with the reading status of the shared data, thereby ensuring the security of the shared data in the CXL memory device; and preventing the computing device that is reading the shared data from being unable to successfully read the shared data.

[0028] Second aspect, an embodiment of the present application provides a method for processing distributed data, which is applied to a first computing device. The first computing device is any computing device in a computing system, and the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; the first computing device is connected to the multiple CXL memory devices; the method includes: sending a data write request to a target computing device; wherein, the target computing device is any computing device in the computing system; the data write request is used to request to write shared data into a first CXL memory device; the data write request includes the size of the shared data and the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to the first computing device; receiving multiple first memory addresses corresponding to multiple first memory regions sent by the target computing device; wherein, the first memory region is a memory region in a second CXL memory device for writing shared memory; the second CXL memory device is a first CXL memory device whose capacity of the free memory region matches the size of the shared data; the number of second CXL memory devices is the number of copies of the shared data; writing the shared data into the multiple first memory regions based on the multiple first memory addresses.

[0029] Based on this implementation manner, each computing device in the cluster can store the shared data therein in multiple CXL memory devices to form multiple data copies, so as to reduce the storage space consumed for distributed storage of data. The computing devices in the cluster can also efficiently read the data copies through the CXL memory devices and improve the channel utilization rate. Moreover, the memory regions in the CXL memory devices can be managed and allocated based on the cluster topology in the target computing device, so that the computing device can efficiently write the shared data into the corresponding CXL memory device, thereby enabling efficient management of distributed data.

[0030] In another possible implementation manner, the method further includes: sending a data lock request to the target computing device; wherein, the data lock request is used to indicate setting the attribute of the shared data to read-only.

[0031] Based on this implementation manner, the target computing device can change the read status corresponding to the shared data in the CXL memory device; moreover, the memory management device can verify whether the computing device is the source computing device corresponding to the shared data to prevent other computing devices other than the source computing device from tampering with the read status of the shared data, thereby ensuring the security of the shared data in the CXL memory device.

[0032] In yet another possible implementation, the method further includes: receiving a data recovery command sent by a target computing device; wherein the data recovery command is used to indicate the recovery of shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address where the shared data is stored in the first computing device; the third memory address is used to indicate the memory address of a third memory area; the third memory area is located in the second CXL memory device corresponding to the second memory address; the second memory address is the memory address from which the second computing device reads the shared data; reading the shared data based on the source address; and writing the shared data into the third memory area based on the third memory address.

[0033] Based on this implementation, in the case where the second computing device fails to read the shared data in the CXL memory device, the first computing device can obtain the original shared data from the source address and write the shared data into the third memory area re-allocated by the memory management module again, so that the second computing device can successfully read the shared data in the third memory area.

[0034] In yet another possible implementation, the method further includes: sending a data recovery success response to the target computing device; wherein the data recovery success response is used to indicate the completion of writing the shared data into the third memory area.

[0035] Based on this implementation, after the first computing device writes the shared data into the third memory area, it can send a data recovery success response to the target computing device to notify that the shared data has been recovered.

[0036] In yet another possible implementation, the method further includes: sending a data deletion request to the target computing device; wherein the data deletion request is used to indicate deleting the shared data from the first CXL memory device; receiving the identifier of the shared data, the storage address in the first CXL memory device, and a deletable flag sent by the target computing device; wherein the deletable flag is used to indicate that the shared data can be deleted; the identifier of the shared data is used to identify the shared data; and deleting the shared data based on the identifier of the shared data and the storage address in the multiple first CXL memory devices.

[0037] Based on this implementation, the computing device can implement the deletion of the shared data in the CXL memory device through interaction with the memory management module; and the memory management device can verify whether the computing device is the source computing device corresponding to the shared data and the occupancy quantity corresponding to the shared data, so as to avoid other computing devices outside the source computing device from tampering with the reading status of the shared data, thereby ensuring the security of the shared data in the CXL memory device; and avoiding the situation where the computing device that is reading the shared data fails to successfully read the shared data.

[0038] In a third aspect, an embodiment of the present application provides a method for processing distributed data, which is applied to a second computing device. The second computing device is any computing device in a computing system, and the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; the second computing device is connected to at least one CXL memory device; the method includes: sending a data reading request to a target computing device; wherein, the target computing device is any computing device in the computing system; the data reading request is used to obtain the storage address of shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used for the shared data; receiving a second memory address corresponding to a second memory area returned by the target computing device; wherein, the second memory area is the shortest access path for the second computing device to read the shared data in a first memory area; the first memory area is a memory area in a second CXL memory device for writing shared memory; the second CXL memory device is the first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to a first computing device; the first computing device is the computing device that writes the shared data; reading the shared data from the second memory area based on the second memory address.

[0039] Based on this implementation, the second computing device can read the shared data in the CXL memory device through interaction with the target computing device, so that the computing device can quickly read the shared data in the CXL memory device to improve the reading speed of the shared data.

[0040] In another possible implementation, the method further includes: sending a data reading failure response to the target computing device; wherein, the reading failure response is used to indicate that the second computing device fails to read the shared data.

[0041] Based on this implementation, when the second computing device fails to read the shared data, it can send a data reading failure response to the target computing device to enable the memory management module to perform recovery processing on the shared data.

[0042] In another possible implementation, the method further includes: receiving a third memory address sent by the target computing device; wherein, the third memory address is used to indicate the memory address of a third memory area; the third memory area is used to write the recovered shared data; the third memory area is located in the second CXL memory device corresponding to the second memory address; reading the shared data from the third memory area based on the third memory address.

[0043] Based on this implementation, after the first computing device rewrites the shared data into the third memory area, the second computing device can read the shared data based on the third memory address corresponding to the third memory area.

[0044] In another possible implementation, the method further includes: sending a data reading success response to the target computing device; wherein, the shared data reading success response is used to indicate that the shared data has been successfully read.

[0045] Based on this implementation, when the second computing device reads the shared data in the target CXL memory device, it can send a data reading success response to the target computing device to notify the second computing device that the shared data has been successfully read.

[0046] In another possible implementation, the method further includes: sending a data release request to the target computing device; wherein, the data release request is used to indicate releasing the second computing device's occupancy of the shared data.

[0047] Based on this implementation, after the second computing device finishes reading the shared data, it can send a data release request to the target computing device to release the second computing device's occupancy of the shared data.

[0048] In a fourth aspect, an embodiment of the present application further provides a processing device for distributed data, configured in a target computing device, where the target computing device is any computing device in a computing system; the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; each CXL memory device among the multiple CXL memory devices is connected to at least two computing devices; the device includes: a first receiving module, configured to receive a data writing request sent by a first computing device; wherein, the first computing device is any computing device in the computing system; the data writing request is used to request writing shared data into a first CXL memory device; the first CXL memory device is a CXL memory device among the multiple CXL memory devices that is connected to the first computing device; the data writing request includes the number of copies of the shared data and the size of the shared data; the number of copies is a positive integer greater than 1; a determining module, configured to determine multiple second CXL memory devices from the multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data; wherein, the second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; a first sending module, configured to send multiple first memory addresses corresponding to the multiple first memory areas to the first computing device, so that the first computing device writes the shared data into the multiple first memory areas based on the multiple first memory addresses; wherein, the first memory area is a free memory area in the second CXL memory device for writing the shared data.

[0049] In a possible implementation, the apparatus further includes: an acquisition module; the acquisition module is configured to: acquire the connection relationship between each CXL memory device in the computing system and each computing device among the multiple computing devices; the determination module is further configured to determine multiple first CXL memory devices from the multiple CXL memory devices according to the connection relationship.

[0050] In another possible implementation, the determination module is specifically configured to: acquire the number of third CXL memory devices; wherein, the third CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; determine whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, select multiple second CXL memory devices from the multiple third CXL memory devices based on the number of copies of the shared data.

[0051] In yet another possible implementation, the apparatus further includes: a fourth receiving module and a setting module; the fourth receiving module is configured to: receive a data locking request; the data locking request is used to indicate setting the attribute of the shared data to read-only; the determination module is further configured to: determine whether the data locking request originates from the first computing device; the setting module is configured to: in the case where the data locking request originates from the first computing device, set the attribute of the shared data to read-only.

[0052] In yet another possible implementation, the fourth receiving module is further configured to: receive a data reading request sent by a second computing device; wherein, the second computing device is any computing device in the computing system; the data reading request is used to indicate reading the shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used to identify the shared data; the determination module is further configured to: determine multiple first memory addresses from the associated data based on the identifier of the shared data; wherein, the associated data includes the identifier of the shared data and multiple first memory addresses; determine a second memory address from the multiple first memory addresses; wherein, the second memory address is the address of a second memory area; the second memory area is the shortest access path for the second computing device to read the shared data; the first sending module is further configured to: return the second memory address to the second computing device, so that the second computing device reads the shared data from the second memory area based on the second memory address.

[0053] In yet another possible implementation, the determination module is further configured to: determine whether the shared data exists based on the identifier of the shared data; in the case where the shared data exists, determine whether the attribute of the shared data is read-only; the determination module is specifically configured to: in the case where the attribute of the shared data is read-only, determine a second memory address from the multiple first memory addresses.

[0054] In yet another possible implementation, the fourth receiving module is further configured to: receive a data reading failure response sent by the second computing device; wherein, the data reading failure response is used to indicate that the second computing device fails to read the shared data; the determining module is further configured to: determine a third memory area from the second CXL memory device corresponding to the second memory address; wherein, the third memory area is used to write the shared data; the first sending module is further configured to: send a data recovery command to the first computing device; wherein, the data recovery command is used to indicate the recovery of the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address for reading the shared data to be recovered; the third memory address is used to indicate the memory address of the third memory area for writing the shared data.

[0055] In yet another possible implementation, the fourth receiving module is further configured to: receive a data recovery success response sent by the first computing device; wherein, the data recovery success response is used to indicate that the shared data is successfully recovered; the first sending module is further configured to: send the third memory address to the second computing device, so that the second computing reads the shared data based on the third memory address.

[0056] In yet another possible implementation, the apparatus further includes: an updating module; the updating module is configured to: update the second memory address in the associated data based on the third memory address.

[0057] In yet another possible implementation, the fourth receiving module is further configured to: receive a data reading success response sent by the second computing device; wherein, the data reading success response is used to indicate that the shared data is successfully read; the updating module is further configured to: increase the number of dependency relationships of the shared data in accordance with a preset step.

[0058] In yet another possible implementation, the fourth receiving module is further configured to: receive a data release request sent by the second computing device; wherein, the data release request is used to indicate the release of the occupancy of the shared data by the second computing device; the updating module is further configured to: decrease the number of dependency relationships of the shared data in accordance with a preset step.

[0059] In yet another possible implementation, the fourth receiving module is further configured to: receive a data deletion request; wherein the data deletion request is used to indicate deleting shared data from the first memory area; the data deletion request includes an identifier of the shared data; the determination module is further configured to: determine whether the identifier of the shared data exists, whether the data deletion request originates from the first computing device, and whether the number of dependency relationships of the shared data is zero; the first sending module is further configured to: when the identifier of the shared data exists, the data deletion request originates from the first computing device, and the number of dependency relationships of the shared data is not zero, send the identifier of the shared data, multiple first memory addresses, and a deletable flag to the first computing device, so that the first computing device deletes the shared data based on the multiple first memory addresses; wherein the deletable flag is used to indicate that the shared data can be deleted.

[0060] In a fifth aspect, an embodiment of the present application further provides a processing device for distributed data, configured in a first computing device, where the first computing device is any computing device in a computing system, and the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; the first computing device is connected to the multiple CXL memory devices; the device includes: a second sending module, configured to send a data write request to a target computing device; wherein the target computing device is any computing device in the computing system; the data write request is used to request writing shared data into a first CXL memory device; the data write request includes the size of the shared data and the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to the first computing device; a second receiving module, configured to receive multiple first memory addresses corresponding to multiple first memory areas sent by the target computing device; wherein the first memory area is a memory area in a second CXL memory device for writing shared memory; the second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is the number of copies of the shared data; a writing module, configured to write the shared data into the multiple first memory areas based on the multiple first memory addresses.

[0061] In a possible implementation, the device further includes: a fourth sending module; the fourth sending module is configured to: send a data locking request to the target computing device; wherein the data locking request is used to indicate setting the attribute of the shared data to read-only.

[0062] In another possible implementation, the device further includes: a second reading module; the second receiving module is further configured to: receive a data recovery command sent by a target computing device; wherein, the data recovery command is used to indicate the recovery of shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address in the first computing device where the shared data is stored; the third memory address is used to indicate the memory address of a third memory area; the third memory area is located in the second CXL memory device corresponding to the second memory address; the second memory address is the memory address where the second computing device reads the shared data; the second reading module is configured to: read the shared data based on the source address; the writing module is further configured to: write the shared data into the third memory area based on the third memory address.

[0063] In yet another possible implementation, the fourth sending module is further configured to: send a data recovery success response to the target computing device; wherein, the data recovery success response is used to indicate the completion of writing the shared data into the third memory area.

[0064] In yet another possible implementation, the device further includes: a deletion module; the fourth sending module is further configured to: send a data deletion request to the target computing device; wherein, the data deletion request is used to indicate the deletion of the shared data from the first CXL memory device; the second receiving module is further configured to: receive the identifier of the shared data, the storage address in the first CXL memory device, and a deletable flag sent by the target computing device; wherein, the deletable flag is used to indicate that the shared data is deletable; the identifier of the shared data is used to identify the shared data; the deletion module is configured to: delete the shared data based on the identifier of the shared data and the storage address in the plurality of first CXL memory devices.

[0065] Sixth aspect, an embodiment of the present application further provides a processing device for distributed data, configured in a second computing device. The second computing device is any computing device in a computing system, and the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; the second computing device is connected to at least one CXL memory device; the device includes: a third sending module, configured to send a data reading request to a target computing device; wherein, the target computing device is any computing device in the computing system; the data reading request is used to obtain the storage address of the shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used for the shared data; a third receiving module, configured to receive a second memory address corresponding to a second memory area returned by the target computing device; wherein, the second memory area is the shortest access path for the second computing device to read the shared data in the first memory area; the first memory area is a memory area in the second CXL memory device for writing the shared memory; the second CXL memory device is the first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to the first computing device; the first computing device is the computing device that writes the shared data; a first reading module, configured to read the shared data from the second memory area based on the second memory address.

[0066] In a possible implementation, the third sending module is further configured to: send a data reading failure response to the target computing device; wherein, the reading failure response is used to indicate that the second computing device fails to read the shared data.

[0067] In another possible implementation, the third receiving module is further configured to: receive a third memory address sent by the target computing device; wherein, the third memory address is used to indicate the memory address of a third memory area; the third memory area is used to write the restored shared data; the third memory area is located in the second CXL memory device corresponding to the second memory address; the first reading module is further configured to: read the shared data from the third memory area based on the third memory address.

[0068] In yet another possible implementation, the third sending module is further configured to: send a data reading success response to the target computing device; wherein, the shared data reading success response is used to indicate that the shared data is successfully read.

[0069] In yet another possible implementation, the third sending module is further configured to: send a data release request to the target computing device; wherein, the data release request is used to indicate releasing the occupancy of the shared data by the second computing device.

[0070] Seventh aspect, an embodiment of the present application further provides a computing device. The computing device includes: a memory and a processor; the memory and the processor are connected; wherein, the memory is used to store program instructions; the processor is used to execute the program instructions so that the computing device executes: receiving a data writing request sent by a first computing device; wherein, the first computing device is any computing device in the computing system; the data writing request is used to request to write shared data into a first CXL memory device; the first CXL memory device is a CXL memory device connected to the first computing device among multiple CXL memory devices; the data writing request includes the number of copies of the shared data and the size of the shared data; the number of copies is a positive integer greater than 1; based on the number of copies of the shared data and the size of the shared data, determining multiple second CXL memory devices from multiple first CXL memory devices; wherein, the second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; sending multiple first memory addresses corresponding to multiple first memory areas to the first computing device, so that the first computing device writes the shared data into the multiple first memory areas based on the multiple first memory addresses; wherein, the first memory area is a free memory area in the second CXL memory device for writing the shared data.

[0071] In a possible implementation manner, before the computing device executes determining multiple second CXL memory devices from multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data, the computing device further executes: obtaining the connection relationship between each CXL memory device in the computing system and each computing device among multiple computing devices; according to the connection relationship, determining multiple first CXL memory devices from multiple CXL memory devices.

[0072] In another possible implementation manner, the computing device further executes: obtaining the number of third CXL memory devices; wherein, the third CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; determining whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, selecting multiple second CXL memory devices from multiple third CXL memory devices based on the number of copies of the shared data.

[0073] In yet another possible implementation manner, the computing device further executes: receiving a data locking request; the data locking request is used to indicate setting the attribute of the shared data to read-only; determining whether the data locking request comes from the first computing device; in the case where the data locking request comes from the first computing device, setting the attribute of the shared data to read-only.

[0074] In yet another possible implementation, the computing device further performs: receiving a data reading request sent by a second computing device; wherein the second computing device is any computing device in the computing system; the data reading request is used to indicate reading shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used to identify the shared data; determining a plurality of first memory addresses from the associated data based on the identifier of the shared data; wherein the associated data includes the identifier of the shared data and a plurality of first memory addresses; determining a second memory address from the plurality of first memory addresses; wherein the second memory address is the address of a second memory area; the second memory area is the shortest access path for the second computing device to read the shared data; returning the second memory address to the second computing device so that the second computing device reads the shared data from the second memory area based on the second memory address.

[0075] In yet another possible implementation, before determining the second memory address from the plurality of first memory addresses, the computing device further performs: determining whether the shared data exists based on the identifier of the shared data; in the case where the shared data exists, determining whether the attribute of the shared data is read-only; determining the second memory address from the plurality of first memory addresses includes: in the case where the attribute of the shared data is read-only, determining the second memory address from the plurality of first memory addresses.

[0076] In yet another possible implementation, the computing device further performs: receiving a data reading failure response sent by the second computing device; wherein the data reading failure response is used to indicate that the second computing device fails to read the shared data; determining a third memory area from the second CXL memory device corresponding to the second memory address; wherein the third memory area is used to write the shared data; sending a data recovery command to the first computing device; wherein the data recovery command is used to indicate recovering the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address for reading the shared data to be recovered; the third memory address is used to indicate the memory address of the third memory area for writing the shared data.

[0077] In yet another possible execution mode, the computing device further performs: receiving a data recovery success response sent by the first computing device; wherein the data recovery success response is used to indicate that the shared data is successfully recovered; sending the third memory address to the second computing device so that the second computing reads the shared data based on the third memory address.

[0078] In yet another possible execution mode, the computing device further performs: updating the second memory address in the associated data based on the third memory address.

[0079] In yet another possible execution mode, the computing device further executes: receiving a data reading success response sent by a second computing device; wherein the data reading success response is used to indicate that the shared data reading is successful; increasing the number of dependency relationships of the shared data in accordance with a preset step.

[0080] In yet another possible execution mode, the computing device further executes: receiving a data release request sent by a second computing device; wherein the data release request is used to indicate releasing the occupancy of the shared data by the second computing device; decreasing the number of dependency relationships of the shared data in accordance with a preset step.

[0081] In yet another possible execution mode, the computing device further executes: receiving a data deletion request; wherein the data deletion request is used to indicate deleting the shared data from a first memory area; the data deletion request includes an identifier of the shared data; determining whether the identifier of the shared data exists, whether the data deletion request originates from a first computing device, and whether the number of dependency relationships of the shared data is zero; in the case where the identifier of the shared data exists, the data deletion request originates from a first computing device, and the number of dependency relationships of the shared data is not zero, sending the identifier of the shared data, a plurality of first memory addresses, and a deletable flag to the first computing device, so that the first computing device deletes the shared data based on the plurality of first memory addresses; wherein the deletable flag is used to indicate that the shared data is deletable.

[0082] In an eighth aspect, an embodiment of the present application further provides a computing device. The computing device includes: a memory and a processor; the memory and the processor are connected; wherein, the memory is used to store program instructions; the processor is used to execute the program instructions so that the computing device executes: sending a data writing request to a target computing device; wherein the target computing device is any computing device in a computing system; the data writing request is used to request writing the shared data into a first CXL memory device; the data writing request includes the size of the shared data and the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to a first computing device; receiving a plurality of first memory addresses corresponding to a plurality of first memory areas sent by the target computing device; wherein the first memory area is a memory area in a second CXL memory device for writing shared memory; the second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of the second CXL memory devices is the number of copies of the shared data; writing the shared data into the plurality of first memory areas based on the plurality of first memory addresses.

[0083] In another possible execution mode, the computing device further executes: sending a data locking request to a target computing device; wherein the data locking request is used to indicate setting the attribute of the shared data to read-only.

[0084] In yet another possible execution manner, the computing device further executes: receiving a data recovery command sent by a target computing device; wherein the data recovery command is used to indicate the recovery of shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address in the first computing device where the shared data is stored; the third memory address is used to indicate the memory address of a third memory area; the third memory area is located in a second CXL memory device corresponding to a second memory address; the second memory address is the memory address where the second computing device reads the shared data; reading the shared data based on the source address; and writing the shared data into the third memory area based on the third memory address.

[0085] In yet another possible execution manner, the computing device further executes: sending a data recovery success response to the target computing device; wherein the data recovery success response is used to indicate the completion of writing the shared data into the third memory area.

[0086] In yet another possible execution manner, the computing device further executes: sending a data deletion request to the target computing device; wherein the data deletion request is used to indicate deleting the shared data from the first CXL memory device; receiving the identifier of the shared data, the storage address in the first CXL memory device, and a deletable flag sent by the target computing device; wherein the deletable flag is used to indicate that the shared data is deletable; the identifier of the shared data is used to identify the shared data; and deleting the shared data based on the identifier of the shared data and the storage address in the plurality of first CXL memory devices.

[0087] In a ninth aspect, an embodiment of the present application further provides a computing device. The computing device includes: a memory and a processor; the memory and the processor are connected; wherein the memory is used to store program instructions; the processor is used to execute the program instructions so that the computing device executes: sending a data reading request to a target computing device; wherein the target computing device is any computing device in the computing system; the data reading request is used to obtain the storage address of the shared data; the data reading request includes the identifier of the shared data; the identifier of the shared data is used for the shared data; receiving the second memory address corresponding to the second memory area returned by the target computing device; wherein the second memory area is the shortest access path for the second computing device to read the shared data in the first memory area; the first memory area is the memory area in the second CXL memory device used for writing the shared memory; the second CXL memory device is the first CXL memory device whose free memory area capacity matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is the CXL memory device connected to the first computing device; the first computing device is the computing device that writes the shared data; and reading the shared data from the second memory area based on the second memory address.

[0088] In another possible execution manner, the computing device further executes: sending a data read failure response to the target computing device; wherein, the read failure response is used to indicate that the second computing device fails to read the shared data.

[0089] In another possible execution manner, the computing device further executes: receiving a third memory address sent by the target computing device; wherein, the third memory address is used to indicate the memory address of a third memory area; the third memory area is used to write the restored shared data; the third memory area is located in the second CXL memory device corresponding to the second memory address; reading the shared data from the third memory area based on the third memory address.

[0090] In another possible execution manner, the computing device further executes: sending a data read success response to the target computing device; wherein, the shared data read success response is used to indicate that the shared data is successfully read.

[0091] In another possible execution manner, the computing device further executes: sending a data release request to the target computing device; wherein, the data release request is used to indicate releasing the occupancy of the shared data by the second computing device.

[0092] In a tenth aspect, an embodiment of the present application provides a computing system, which includes a plurality of computing devices and a plurality of Compute Express Link (CXL) memory devices; wherein, each CXL memory device among the plurality of CXL memory devices is connected to at least two computing devices; the plurality of computing devices include a first computing device, a second computing device, and a target computing device; the target computing device is any one of the plurality of computing devices; wherein, the target computing device is the computing device described in the seventh aspect, the first computing device is the computing device described in the eighth aspect, and the second computing device is the computing device described in the ninth aspect.

[0093] In an eleventh aspect, an embodiment of the present application provides a chip, which is used to execute the method according to any one of the first aspect, or the method according to any one of the second aspect, or the method according to any one of the third aspect.

[0094] In a twelfth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a computer, the method according to any one of the first aspect, or the method according to any one of the second aspect, or the method according to any one of the third aspect is implemented.

[0095] In a thirteenth aspect, an embodiment of the present application provides a program product, including a computer program, which when executed by a processor, implements the method according to any one of the first aspect, or the method according to any one of the second aspect, or the method according to any one of the third aspect. Description of the Drawings

[0096] Figure 1 It is a scenario diagram of a method for processing distributed data provided by an embodiment of the present application;

[0097] Figure 2A It is a schematic diagram of direct connection between a CXL memory device and a computing device provided by an embodiment of the present application;

[0098] Figure 2B It is a schematic diagram of connection between a CXL memory device and a computing device through a switching device provided by an embodiment of the present application;

[0099] Figure 3 It is an interaction diagram of a method for processing distributed data provided by an embodiment of the present application;

[0100] Figure 4 It is a schematic diagram of a first memory area provided by an embodiment of the present application;

[0101] Figure 5 It is a schematic diagram of distributed storage of data provided by an embodiment of the present application;

[0102] Figure 6 It is an interaction diagram of read-only processing of data provided by an embodiment of the present application;

[0103] Figure 7 It is an interaction diagram of data deletion processing provided by an embodiment of the present application;

[0104] Figure 8 It is an interaction diagram of data reading processing provided by an embodiment of the present application;

[0105] Figure 9 It is an interaction diagram of determining the data reading status provided by an embodiment of the present application;

[0106] Figure 10 It is an interaction diagram of data release processing provided by an embodiment of the present application;

[0107] Figure 11 It is an interaction diagram of data recovery processing provided by an embodiment of the present application;

[0108] Figure 12 It is another interaction diagram of data reading processing provided by an embodiment of the present application;

[0109] Figure 13 It is a schematic diagram of a device for processing distributed data provided by an embodiment of the present application;

[0110] Figure 14 It is another schematic diagram of a device for processing distributed data provided by an embodiment of the present application;

[0111] Figure 15Schematic diagram of another processing device for distributed data provided by an embodiment of the present application;

[0112] Figure 16 Schematic diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0113] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application. For the convenience of clearly describing the technical solutions in the embodiments of the present application, the first, second, etc. descriptions that appear in the embodiments of the present application are only for schematic and distinguishing the described objects, without order, and do not represent special limitations on the number of devices in the embodiments of the present application, and cannot constitute any limitation to the embodiments of the present application.

[0114] The distributed data processing method provided by the embodiments of the present application can be applied to a computing device, and the computing device can be a server or a terminal device. Multiple Compute Express Link (CXL) memory devices are used as a memory pool and connected to multiple computing devices through the CXL.mem protocol. The memory management module can uniformly manage the memory areas of multiple CXL memory devices and can allocate memory areas in multiple CXL memory devices for the computing device to write the shared data of the application program into different CXL memory devices. The computing device can also read the required shared data in the corresponding CXL memory device. Thereby, the storage space consumed for distributed storage of data can be reduced, and the reading efficiency of distributed data can be improved.

[0115] Figure 1 Scene diagram of a distributed data processing method provided by an embodiment of the present application. As Figure 1 shown, a cluster (or called a computing system) includes multiple computing devices 110, and the multiple computing devices 110 are physically connected to multiple CXL memory devices 120 through high-speed cables. Among them, the transmission interface of the high-speed cable can be a Peripheral Component Interconnect Express (PCIe) interface, so that data can be transmitted between the CXL memory device 120 and the computing device 110 through the PCIe interface, thereby improving the data transmission speed.

[0116] The CXL memory device 120 can be externally connected with one or more memory modules, such as a solid-state disk (SSD), a dynamic random-access memory (DRAM), etc. The memory areas provided by the memory modules can be used to store the data of the application program (hereinafter referred to as shared data) transmitted by the computing device.

[0117] The CXL memory device 120 can be a multi-head device. When the CXL memory device 120 is a multi-head device, each CXL memory device 120 can be directly connected to multiple computing devices 110 to form a network. Refer to Figure 2A the schematic diagram showing the direct connection between a CXL memory device and a computing device as shown in Figure 2A As shown, CXL memory device 1, CXL memory device 2, CXL memory device 3, CXL memory device 4, CXL memory device 5, and CXL memory device 6 are all multi-head devices. Each CXL memory device is connected to two computing devices in a direct connection manner. For example, CXL memory device 1 is directly connected to computing device A and computing device B, and CXL memory device 2 is directly connected to computing device A and computing device C.

[0118] Alternatively, each computing device can also form a switch network with multiple CXL memory devices 120 through a switch. Refer to Figure 2B the schematic diagram showing the connection between a CXL memory device and a computing device through a switch as shown in Figure 2B As shown, each computing device is respectively connected to multiple CXL memory devices through a switch. For example, computing device A is connected to CXL memory device 1 and CXL memory device 2 through switch A, and computing device B is connected to CXL memory device 3 and CXL memory device 4 through switch B.

[0119] The memory management module 10 can be configured in any computing device 110 within the cluster or in other computing devices outside the cluster. When the memory management module 10 is configured in a computing device outside the cluster, this computing device is also physically connected to the CXL memory device. Among them, the memory management module 10 can be a fabric manager for managing the memory area of the CXL memory device. A network connection is established between the computing device where the memory management module 10 is located and other computing devices in the cluster for data transmission through the network.

[0120] Each computing device 110 is configured with a preset module, including a local memory management daemon (LMD) and a software development kit (SDK). Among them, the application program in each computing device 110 can call the interface provided by the SDK module. The SDK module can, in response to the call of the application program, send a corresponding request to the LMD module. The LMD module can, in response to the request sent by the SDK module, interact with the memory management module 10.

[0121] Exemplarily, the LMD module and the SDK module can be used to write the shared data corresponding to the application program in the computing device 110 (such as the data of the application program in the disk or memory) into the CXL memory device 120 connected to the computing device 110, and to read the shared data corresponding to the application program in other computing devices 110 from the CXL memory device 120 connected to the computing device 110.

[0122] In some embodiments, the application program in any computing device 110 (such as the first computing device) can determine the size of the shared data to be written, and call the data writing interface of the SDK module to send a data writing request to the LMD module through the SDK module, where the data writing request includes the number of copies to be written corresponding to the shared data and the size of the shared data. The LMD module sends the data writing request to the memory management module 10, and the memory management module 10, in response to the data writing request, determines a plurality of first CXL memory devices 120 connected to the first computing device 110 among the plurality of CXL memory devices 120, and determines a plurality of second CXL memory devices 120 in the plurality of first CXL memory devices 120 based on the number of copies of the shared data and the size of the shared data, where the number of the second CXL memory devices 120 is equal to the number of copies of the shared data. Then, the memory management module 10 determines the first memory area for storing the shared data and the area information (such as the memory address) of the first memory area in each second CXL memory device 120, and sends the area information of the first memory area corresponding to each second CXL memory device 120 to the LMD module of the first computing device 110. After receiving the area information of the first memory area corresponding to each second CXL memory device 120, the LMD module sends the area information of each first memory area to the SDK module, so that the SDK module can write the shared data into the first memory area corresponding to each second CXL memory device 120 based on the area information of each first memory area.

[0123] In some embodiments, after the first computing device 110 finishes writing the shared data, an application on other computing devices (such as the second computing device) among the multiple computing devices 110 can call the data reading interface of the SDK module to send a data reading request to the LMD module through the SDK module. The data reading request includes the identifier of the shared data to be read (i.e., the data identifier). After the LMD module sends the data reading request to the memory management module 10, the memory management module 10, in response to the data reading request, determines, among the multiple CXL memory devices 120, a first CXL memory device 120 that is connected to the second computing device 110 and stores the shared data corresponding to the data identifier, and obtains the region information of the first memory region corresponding to the first CXL memory device 120. Then, the memory management module 10 sends the region information of the first memory region corresponding to the first CXL memory device 120 to the LMD module of the second computing device 110. After receiving the region information of the first memory region corresponding to the first CXL memory device 120, the LMD module sends the region information to the SDK module, so that the SDK module can read the corresponding shared data in the first memory region corresponding to the first CXL memory device 120 based on the region information. The number of the first CXL memory devices 120 can be one or more.

[0124] Through the above solution, the memory management module can uniformly manage the memory regions of multiple CXL memory devices, and can allocate memory regions among the multiple CXL memory devices for each first computing device to write the shared data of the application into different CXL memory devices. When each second computing device needs to read the shared data, the memory management module can determine the CXL memory device that can be read and the corresponding region information according to the data identifier of the shared data provided by the second computing device, so that the second computing device can read the required shared data in the corresponding CXL memory device, thereby reducing the storage space consumed by distributed storage of data and improving the reading efficiency of distributed data.

[0125] Figure 3 An interaction diagram of a method for processing distributed data provided by an embodiment of the present application. This method can be applied to Figure 1 the memory management module and the first computing device shown in the figure. As Figure 3 shown, this method includes steps 310 to 340.

[0126] Step 310, the first computing device sends a data writing request to the target computing device.

[0127] In some embodiments, the application program in the first computing device determines the size of the shared data to be written corresponding to the application program, the number of copies of the shared data to be written, and the storage location (or storage path) of the shared data in the first computing device, and calls the data writing interface of the SDK module in the first computing device to enable the SDK module to obtain the size of the shared data and the storage location of the shared data. Then, the SDK module sends a data writing request to the memory management module in the target computing device through the LMD module. The data writing request includes the number of copies of the shared data and the size of the shared data, and the number of copies is a positive integer greater than 1.

[0128] Exemplarily, when the application program in the first computing device sends a data writing request through the SDK module, it can generate a verification parameter (such as a topic parameter) corresponding to the shared data to be written and write it into the data writing request for subsequent verification of the data writing request by the memory management module. The verification parameter is used to distinguish different shared data to be written, that is, the verification parameters corresponding to each shared data are unique.

[0129] Step 320, in response to the data writing request, the target computing device determines a plurality of second CXL memory devices from a plurality of first CXL memory devices based on the number of copies of the shared data and the size of the shared data.

[0130] In some embodiments, the memory management module in the target computing device can scan a plurality of CXL memory devices in the computing system to obtain a cluster topology. The cluster topology includes the connection relationship between each CXL memory device and each computing device in the cluster. As Figure 2A shown, the memory management module can determine that CXL memory device 1 is respectively connected to computing device A and computing device B, and CXL memory device 2 is respectively connected to computing device A and computing device C, etc.

[0131] In some embodiments, after receiving the data writing request sent by the LMD module, the memory management module, in response to the data writing request, determines a plurality of first CXL memory devices connected to the first computing device from a plurality of CXL memory devices according to the obtained connection relationship. The device identifier corresponding to the first computing device may also be included in the data writing request.

[0132] In some embodiments, before determining the above-mentioned first CXL memory device, the memory management module may also verify the verification parameters in the data write request to determine whether the shared data corresponding to the verification parameters has been written to the CXL memory device before. If the shared data has been written to the CXL memory device, there is no need to write the shared data again. At this time, the memory management module determines that the data write request is invalid and may send a data write failure message to the SDK module through the LMD module; if the shared data has not been written to the CXL memory device, the memory management module determines that the data write request is valid and continues to process the data write request.

[0133] Exemplarily, the memory management module may save the verification parameters corresponding to the shared data written to the CXL memory device. Thus, when the memory management module receives a new data write request, it obtains the verification parameters in the data write request and determines whether the verification parameters are among the saved verification parameters. If the verification parameters are among the saved verification parameters, the memory management module determines that the shared data has been written to the CXL memory device; if the verification parameters are not among the saved verification parameters, the memory management module determines that the shared data has not been written to the CXL memory device.

[0134] In some embodiments, after determining multiple first CXL memory devices, the memory management module may, based on the number of copies of the shared data, the size of the shared data, and the memory size of each first CXL memory device among the multiple first CXL memory devices, determine a second CXL memory device in which the capacity of the free memory area matches the size of the shared data, and determine a first memory area for storing the shared data in the free memory area of the second CXL memory device. Wherein, the area size of the first memory area is equal to the size of the shared data. Moreover, the memory management module may determine the area information of the first memory area corresponding to each target CXL memory device and generate a unique data identifier corresponding to the shared data.

[0135] Wherein, the area information of the first memory area may include the device information (such as device identifier) of the CXL memory device where it is located, the first memory address corresponding to the first memory area (including the start address and offset), and the area size. The first memory address corresponding to the first memory area is determined according to the sum of the start address and the offset corresponding to the first memory area.

[0136] Reference Figure 4 As shown in a schematic diagram of a first memory area, such as Figure 4As shown in the figure, taking the first memory area allocated by the memory management module for application A as the first memory area A in the second CXL memory device and the first memory area allocated for application B as the first memory area B in the second CXL memory device as an example, the starting addresses corresponding to the first memory area A and the first memory area B are both the starting address of this section of the memory area in the second CXL memory device. The offset A corresponding to the first memory area A is 0, and the offset B corresponding to the first memory area B is equal to the area size of the first memory area A.

[0137] Among them, the data identifier of the shared data may include a main identifier and a sub-identifier. The main identifier is used to distinguish different shared data, and the sub-identifier is used to distinguish the same shared data in different CXL memory devices. Refer to Figure 5 A schematic diagram of a data distributed storage as shown in Figure 5 As shown in the figure, taking the shared data A in the first computing device A as an example, after the memory management module determines that the second CXL memory devices corresponding to the shared data A are the second CXL memory device 1, the second CXL memory device 2, and the second CXL memory device 3, it can generate the main identifier of the shared data A as obj_A, and determine the sub-identifiers of the shared data A corresponding to each second CXL memory device as obj_1, obj_2, and obj_3 respectively. Thus, it is determined that the data identifier of the shared data A corresponding to the second CXL memory device 1 is obj_A_1, the identifier of the shared data A corresponding to the second CXL memory device 2 is obj_A_2, and the data identifier of the shared data A corresponding to the second CXL memory device 3 is obj_A_3. Another example is that it can be determined that the data identifier of the shared data B corresponding to the second CXL memory device 1 is obj_B_1, the data identifier of the shared data B corresponding to the second CXL memory device 4 is obj_B_4, and the data identifier of the shared data B corresponding to the second CXL memory device 5 is obj_B_5.

[0138] It should be noted that the specific form of the data identifier of the above-mentioned shared data is only an example, and this is not limited in this embodiment.

[0139] Step 330, the target computing device sends multiple first memory addresses corresponding to multiple first memory areas to the first computing device.

[0140] In some embodiments, after the memory management module determines the area information of the first memory area corresponding to each second CXL memory device, it sends the area information of each first memory area and the data identifier of the shared data to the LMD module in the first computing device, so that the LMD module returns the area information of the first memory area and the data identifier of the shared data to the SDK module.

[0141] Step 340, the first computing device receives multiple first memory addresses corresponding to multiple first memory regions sent by the target computing device, and writes the shared data into the multiple first memory regions based on the multiple first memory addresses.

[0142] In some embodiments, after the SDK module in the first computing device receives the region information of the first memory regions corresponding to the above-mentioned second CXL memory devices and the data identifier of the shared data, it determines the shared data corresponding to the data identifier among the multiple shared data to be written according to the data identifier of the shared data, and determines the second CXL memory devices to be accessed according to each device information. Then, the SDK module writes the shared data corresponding to the data identifier into the first memory regions corresponding to the second CXL memory devices respectively according to the first memory addresses of the first memory regions corresponding to the second CXL memory devices. After the writing is completed, the first memory regions corresponding to the second CXL memory devices store the shared data corresponding to the data identifier, and the shared data is the data copy corresponding to the shared data in the first computing device.

[0143] Exemplarily, the SDK module can read the corresponding shared data according to the storage location of the shared data in the first computing device, and write the read shared data into the first memory regions corresponding to the second CXL memory devices.

[0144] Exemplarily, the SDK module can write the shared data into the first memory regions corresponding to the second CXL memory devices in sequence, or write the shared data into the first memory regions corresponding to the second CXL memory devices simultaneously in parallel. This embodiment does not limit this.

[0145] Through the above solution, each computing device in the cluster can store the shared data therein in multiple CXL memory devices to form multiple data copies, thereby reducing the storage space consumed by distributed storage of data. The computing devices in the cluster can also efficiently read the data copies through the CXL memory devices and improve the channel utilization rate. And, the memory management module can manage and allocate the memory regions in the CXL memory devices based on the cluster topology, so that the computing device can efficiently write the shared data into the corresponding CXL memory device, thereby realizing the efficient management of distributed data.

[0146] In some embodiments, the above method further includes: the target computing device, in response to a data write request sent by the first preset module, obtains the connection relationship between each CXL memory device in the computing system and each computing device among the multiple computing devices; and determines multiple first CXL memory devices from the multiple CXL memory devices according to the connection relationship.

[0147] In some embodiments, after receiving a data write request sent by a first computing device, the memory management module in the target computing device may scan multiple CXL memory devices in the computing system to obtain the cluster topology. The cluster topology includes the connection relationships between the CXL memory devices and the computing devices in the cluster. Exemplarily, the memory management module may scan multiple CXL memory devices to obtain the device identifiers of the computing devices connected to each CXL memory device, and determine the connection relationships between the CXL memory devices and the computing devices based on the device identifiers of the computing devices connected to each CXL memory device and the device identifiers corresponding to the computing devices.

[0148] As Figure 2A shown, the memory management module may obtain the device identifiers of the computing devices connected to CXL memory device 1, CXL memory device 2, CXL memory device 3, CXL memory device 4, CXL memory device 5, and CXL memory device 6 respectively, and obtain the device identifiers corresponding to computing device A, computing device B, computing device C, and computing device D respectively, and determine the connection relationships between the CXL memory devices and the computing devices based on the device identifiers of the computing devices connected to each CXL memory device and the device identifiers corresponding to the computing devices. For example, the device identifiers of the computing devices connected to CXL memory device 1 are node_A and node_B respectively, and the device identifier of computing device A is node_A, and the device identifier of computing device B is node_B, then it can be determined that the computing devices connected to CXL memory device 1 are computing device A and computing device B. It should be noted that the form of the above device identifier is only an example, and this is not limited in this embodiment.

[0149] In some examples, as shown in Table 1, the memory management module may also save the obtained Figure 2A corresponding cluster topology in the form of a table (hereinafter referred to as the cluster topology table):

[0150] Table 1

[0151] Computing device CXL memory device connected to the computing device node_A CXL_1, CXL_2, CXL_3 node_B CXL_1, CXL_4, CXL_5 node_C CXL_2, CXL_4, CXL_6 node_D CXL_3, CXL_5, CXL_6

[0152] Among them, node_A, node_B, node_C, and node_D are the device identifiers corresponding to computing device A, computing device B, computing device C, and computing device D respectively, and CXL_1, CXL_2, CXL_3, CXL_4, CXL_5, and CXL_6 are the device identifiers corresponding to CXL memory device 1, CXL memory device 2, CXL memory device 3, CXL memory device 4, CXL memory device 5, and CXL memory device 6 respectively.

[0153] Through the above solution, the memory management module can obtain the topology of the cluster to determine the connection relationship between each CXL memory device and each computing device.

[0154] In some embodiments, step 320 above includes: the target computing device obtains the number of third CXL memory devices; wherein, the third CXL memory device is the first CXL memory device whose capacity of the free memory area matches the size of the shared data; determines whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; and in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, selects a plurality of second CXL memory devices from the plurality of third CXL memory devices based on the number of copies of the shared data.

[0155] In some embodiments, after receiving a data write request sent by the first computing device, the memory management module in the target computing device may first determine the size of the memory areas of each first CXL memory device connected to the first computing device. Exemplarily, the memory management module may determine the size of the free memory areas in each first CXL memory device, where the free memory area is the memory area that does not store shared data; and determine the third CXL memory devices whose capacity of the free memory area is greater than or equal to the size of the shared data and the number of third CXL memory devices among the plurality of first CXL memory devices. Then, the memory management module may determine whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, and in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, select a plurality of second CXL memory devices equal in number to the number of copies of the shared data from the plurality of third CXL memory devices.

[0156] Moreover, the memory management module can also determine whether the connection relationship between each CXL memory device and each computing device in the cluster meets a preset connection relationship. Among them, in the case where the connection relationship meets the preset connection relationship, the first computing device can read other shared data written by computing devices other than the first computing device among the plurality of computing devices in each second CXL memory device. As Figure 5 shown, the connection relationship of this cluster meets the preset connection relationship, that is, each computing device can read the shared data written by other computing devices in the CXL memory device connected to it. For example, computing device A can read all other shared data written by other computing devices, including shared data B, shared data C, and shared data D, in the CXL memory device 1, CXL memory device 2, and CXL memory device 3 connected to it.

[0157] When the number of third CXL memory devices is greater than or equal to the number of copies of the shared data (i.e., the number of second CXL memory devices is equal to the number of copies of the shared data), the memory area sizes of the second CXL memory devices meet the requirements, and the connection relationships between the CXL memory devices and the computing devices meet the preset connection relationships, the memory management module may perform area allocation for the second CXL memory devices, that is, determine the first memory areas corresponding to the second CXL memory devices.

[0158] In some embodiments, if the number of third CXL memory devices is less than the number of copies of the shared data (i.e., the number of second CXL memory devices is less than the number of copies of the shared data), or the memory area sizes of the second CXL memory devices do not meet the requirements, or the connection relationships between the CXL memory devices and the computing devices do not meet the preset connection relationships, the memory management module may return an information of memory allocation failure to the SDK module through the LMD module to notify the SDK module that the shared data cannot be written into the CXL memory devices currently.

[0159] Through the above solution, before determining the first memory areas corresponding to the second CXL memory devices, the number, free memory area size, and connection relationship of the second CXL memory devices may be determined first to ensure that the number of second CXL memory devices is equal to the number of copies of the shared data, and there is sufficient memory space in each second CXL memory device to store the shared data; and the cluster topology needs to meet the preset connection relationship to ensure that each computing device can read the shared data written by other computing devices in the CXL memory device connected to it.

[0160] In some embodiments, the above step 340 includes: the first computing device determines the number of devices of a plurality of second CXL memory devices, and creates a plurality of data writing threads according to the number of devices, the area information of the first memory areas corresponding to the second CXL memory devices, and the data identifier of the shared data; the first computing device executes the plurality of data writing threads to write the shared data into the first memory areas corresponding to the second CXL memory devices in parallel.

[0161] In some embodiments, after the SDK module in the first computing device receives the area information of the first memory areas corresponding to the second CXL memory devices and the data identifier of the shared data sent by the memory management module, the SDK module may write the shared data corresponding to the data identifier of the shared data into the first memory areas corresponding to the second CXL memory devices in parallel based on the area information of the first memory areas corresponding to the second CXL memory devices.

[0162] Exemplarily, the memory management module may first determine the number of devices of multiple second CXL memory devices, and create multiple data writing threads based on the number of devices, the region information of the first memory regions corresponding to the respective second CXL memory devices, and the data identifier of the shared data. Among them, the number of the multiple data writing threads is equal to the number of devices of the second CXL memory devices. Then, the SDK module may execute the multiple data writing threads to write the shared data corresponding to the data identifier into the first memory regions corresponding to the respective second CXL memory devices in parallel.

[0163] Through the above solution, writing the shared data into the first memory regions corresponding to the respective second CXL memory devices in parallel can improve the writing efficiency of the shared data.

[0164] Figure 6 An interaction diagram for data read-only processing provided by an embodiment of this application is as Figure 6 shown, and the above method further includes steps 610 to 630.

[0165] Step 610, the first computing device sends a data locking request to the target computing device.

[0166] In some embodiments, after the first computing device finishes writing the shared data, it may send a data locking request to the memory management module in the target computing device. Among them, the data locking request includes the data identifier of the shared data, so that the memory management module determines that the first computing device has finished writing the shared data, and other computing devices in the cluster can read the shared data.

[0167] Exemplarily, after the SDK module in the first computing device finishes writing the shared data, the application program may call the data read-only interface of the SDK module, so that the SDK module sends a data locking request to the memory management module through the LMD module.

[0168] Step 620, the target computing device determines whether the data locking request comes from the first computing device in response to the data locking request.

[0169] In some embodiments, the memory management module determines the source computing device where the shared data corresponding to the data identifier is located according to the identifier in the data locking request in response to the data locking request sent by the LMD module. Among them, the source computing device is the computing device that writes the shared data into each CXL memory device; the data locking request may further include the device identifier corresponding to the first computing device; the data locking request is used to instruct the memory management module to set the attribute of the shared data to read-only. For example, as Figure 5 shown, the source computing device corresponding to the shared data A in the CXL memory device 1 is the computing device A, and the source computing device corresponding to the shared data D in the CXL memory device 3 is the computing device D.

[0170] In some embodiments, the memory management module may establish a shared data information table, which may include associated data such as the data identifier of each shared data, the device identifier corresponding to the CXL memory device where each shared data is located, the region information corresponding to the first memory region where each shared data is located, the device identifier of the source computing device corresponding to each shared data, the read status corresponding to each shared data, and the occupancy quantity corresponding to each shared data, so as to store and update the relevant information corresponding to each shared data.

[0171] Among them, the read status corresponding to each shared data may include a read-only status (i.e., a readable status) and an unreadable status. The shared data being in the read-only status indicates that the shared data has been written and can be read; the shared data being in the unreadable status indicates that the shared data is in the writing process and cannot be read currently. The occupancy quantity corresponding to the shared data represents the number of computing devices that are currently reading the shared data. For example, if the occupancy quantity corresponding to shared data A is 2, it means that there are currently 2 computing devices in the cluster that are reading shared data A.

[0172] In some examples, the shared data information table may be as shown in Table 2:

[0173] Table 2

[0174]

[0175] Exemplarily, the memory management module may, in the above-mentioned shared data information table, find the device identifier of the corresponding source computing device according to the data identifier sent by the LMD module. For example, as shown in Table 2, taking the data identifiers received by the memory management module as obj_B_1, obj_B_4, and obj_B_5 (or the data identifier sent by the SDK module may also be obj_B, and the memory management module may find all the data identifiers obj_B_1, obj_B_4, and obj_B_5 with the same main identifier in the shared data information table according to obj_B), the memory management module may determine that the device identifier of the source computing device corresponding to the above data identifier in the shared data information table is node_B.

[0176] Step 630, when the data lock request comes from the first computing device, set the attribute of the shared data to read-only.

[0177] In some embodiments, after the memory management module determines the device identifier of the source computing device corresponding to the data identifier, it may match the device identifier corresponding to the source computing device with the device identifier corresponding to the first computing device to determine whether the first computing device is the source computing device corresponding to the shared data.

[0178] Exemplarily, when the device identifier corresponding to the source computing device is the same as the device identifier corresponding to the first computing device, the first computing device is the source computing device corresponding to the shared data. Then, the memory management module can determine the device identifiers of the second CXL memory devices corresponding to the data identifier and the address information corresponding to each first memory area in the shared data information table, and determine the read status of the shared data in the first memory area corresponding to each second CXL memory device as the readable state according to each device identifier and each address information. Among them, before the source computing device corresponding to each shared data sends a data lock request to the memory management module, the memory management module defaults the read status corresponding to each shared data to the unreadable state.

[0179] For example, as shown in Table 2, taking the data identifiers received by the memory management module as obj_B_1, obj_B_4, and obj_B_5 as an example, the memory management module can determine that the device identifiers of the second CXL memory devices corresponding to the above data identifiers are CXL_1, CXL_4, and CXL_5 in the shared data information table, as well as the address information corresponding to each first memory area. Thus, the memory management module updates the read statuses corresponding to the shared data obj_B_1, obj_B_4, and obj_B_5 in the first memory areas of CXL memory device 1, CXL memory device 3, and CXL memory device 4 to the read-only state respectively according to the above device identifiers and address information.

[0180] Exemplarily, when the device identifier corresponding to the source computing device is different from the device identifier corresponding to the first computing device, the first computing device is not the source computing device corresponding to the shared data. Then, the memory management module can return the information that the data lock request fails to the SDK module through the LMD module to notify the SDK module that the first computing device cannot change the read status of the shared data.

[0181] Through the above solution, the computing device can change the read status of the shared data in the CXL memory device through the memory management module; and the memory management device can verify whether the computing device is the source computing device corresponding to the shared data to avoid other computing devices outside the source computing device from tampering with the read status of the shared data, thereby ensuring the security of the shared data in the CXL memory device.

[0182] Figure 7 An interaction diagram for data deletion processing provided by an embodiment of the present application is as follows Figure 7 As shown, the method further includes steps 710 to 740.

[0183] Step 710, the first computing device sends a data deletion request to the target computing device.

[0184] In some embodiments, the application program in the first computing device can also delete the shared data written into the CXL memory device through the memory management module. Exemplarily, the application program in the first computing device calls the data deletion interface in the SDK module to enable the SDK module to obtain the data identifier corresponding to the shared data to be deleted, and sends a data deletion request to the memory management module in the target computing device through the LMD module. Wherein, the data deletion request includes the data identifier of the shared data.

[0185] Step 720, in response to the data deletion request, the target computing device determines whether the identifier of the shared data exists, whether the data deletion request comes from the first computing device, and whether the number of dependency relationships of the shared data is zero.

[0186] In some embodiments, after receiving the data deletion request sent by the LMD module, the memory management module can first determine whether the data identifier corresponding to the shared data exists in the shared data information table. If the data identifier does not exist in the shared data information table, the memory management module can send a data deletion failure message to the SDK module through the LMD module in the first computing device to notify the SDK module that the shared data corresponding to the data identifier does not exist in the CXL memory device. If the data identifier exists in the shared data information table, the memory management module can search in the shared data information table for the source computing device of the shared data corresponding to the data identifier, and the occupancy quantity of the shared data corresponding to the data identifier in each second CXL memory device.

[0187] For example, as shown in Table 2, taking the data identifiers received by the memory management module as obj_A_1, obj_A_2, and obj_A_3 (or the identifier sent by the SDK module can also be obj_A, and the memory management module can find all the data identifiers obj_A_1, obj_A_2, and obj_A_3 with the same main identifier in the shared data information table according to obj_A), the memory management module can determine that the device identifier of the source computing device of the shared data corresponding to the above data identifier in the shared data information table is node_A, and the occupancy quantities of the shared data in CXL memory device 1, CXL memory device 2, and CXL memory device 3 are 0, 1, and 2 respectively.

[0188] In some embodiments, after determining the device identifier of the source computing device corresponding to the data identifier, the memory management module may match the device identifier corresponding to the source computing device with the device identifier corresponding to the first computing device to determine whether the first computing device is the source computing device corresponding to the shared data. If the device identifier corresponding to the source computing device is the same as the device identifier corresponding to the first computing device, the first computing device is the source computing device corresponding to the shared data. If the device identifier corresponding to the source computing device is different from the device identifier corresponding to the first computing device, the first computing device is not the source computing device corresponding to the shared data. Moreover, the memory management module may determine whether the occupancy quantity (or called the dependency quantity) corresponding to the shared data in each second CXL memory device is 0 to determine whether there is a computing device in the cluster that is reading the shared data.

[0189] Step 730, when the identifier of the shared data exists, the data deletion request comes from the first computing device, and the dependency quantity of the shared data is not zero, send the identifier of the shared data, multiple first memory addresses, and a deletable flag to the first computing device.

[0190] In some embodiments, if the data identifier of the shared data exists, the first computing device is the source computing device corresponding to the shared data, and the occupancy quantity corresponding to the shared data in each second CXL memory device is 0, the memory management module may send the permission to delete information (such as a deletable flag), the data identifier of the shared data, and the region information of the first memory region corresponding to each second CXL memory device to the SDK module through the LMD module.

[0191] In some embodiments, if the data identifier of the shared data does not exist, or the first computing device is not the source computing device corresponding to the shared data, or the occupancy quantity corresponding to the shared data in any second CXL memory device is not 0, the memory management module may return an information indicating data deletion failure to the SDK module through the LMD module to notify the SDK module that the first computing device cannot delete the shared data in each second CXL memory device.

[0192] Step 740, the first computing device deletes the shared data in each first memory region based on the region information of the first memory region corresponding to each second CXL memory device.

[0193] In some embodiments, after the SDK module receives the permission deletion information sent by the memory management module and the region information of the first memory regions corresponding to each second CXL memory device, the SDK module may determine the starting deletion position according to the first memory address, and determine the ending deletion position according to the region size and the starting deletion position; thus, the SDK module deletes the shared data in the first memory regions corresponding to each second CXL memory device according to the starting deletion position and the ending deletion position.

[0194] Through the above solution, the computing device can implement the deletion of the shared data in the CXL memory device through interaction with the memory management module; moreover, the memory management device can verify whether the computing device is the source computing device corresponding to the shared data and the occupied quantity corresponding to the shared data, so as to prevent other computing devices other than the source computing device from tampering with the reading status of the shared data, thereby ensuring the security of the shared data in the CXL memory device; and preventing the computing device that is reading the shared data from failing to successfully read the shared data.

[0195] Figure 8 The following is an interaction diagram of data reading processing provided by an embodiment of the present application. As Figure 8 shown, the above method further includes steps 810 to 850.

[0196] Step 810, the second computing device sends a data reading request to the target computing device.

[0197] In some embodiments, any one of the multiple computing devices (such as the second computing device) may read the shared data in the CXL memory device connected thereto. The second computing device includes an LMD module and an SDK module. The application program in the second computing device may call the data reading interface of the SDK module in the second computing device, so that the SDK module obtains the data identifier corresponding to the shared data to be read, and sends a data reading request to the LMD module, where the data reading request includes the data identifier corresponding to the shared data. Among them, the data identifier corresponding to the shared data may be input to the SDK module when the user calls the data reading interface of the SDK module.

[0198] Step 820, the target computing device responds to the data reading request and determines multiple first memory addresses from the associated data based on the identifier of the shared data.

[0199] In some embodiments, after the memory management module in the target computing device receives the data reading request sent by the LMD module, in response to the data reading request, it determines multiple second CXL memory devices in which the shared data corresponding to the data identifier is stored among the multiple CXL memory devices.

[0200] Exemplarily, the memory management module may look up the device identifiers of multiple second CXL memory devices corresponding to the data identifier in the shared data information table. For example, as shown in Table 2, taking the data identifiers received by the memory management module as obj_A_1, obj_A_2, and obj_A_3 (or the data identifier sent by the SDK module may also be obj_A, and the memory management module may look up all data identifiers obj_A_1, obj_A_2, and obj_A_3 with the same primary identifier in the shared data information table according to obj_A), the memory management module may determine that the device identifiers of the target CXL memory devices of the shared data corresponding to the above data identifiers in the shared data information table are CXL_1, CXL_2, and CXL_3, that is, the target CXL memory devices corresponding to the above data identifiers are CXL memory device 1, CXL memory device 2, and CXL memory device 3.

[0201] Step 830, the target computing device determines a second memory address from multiple first memory addresses.

[0202] In some embodiments, after determining the multiple second CXL memory devices, the memory management module may further determine, according to the cluster topology, the CXL memory device connected to the second computing device among the multiple second CXL memory devices, and obtain the region information of the second memory region storing the shared data corresponding to the data identifier in the CXL memory device, so that the second computing device can read the shared data through the shortest access path.

[0203] Exemplarily, the memory management module may look up the target CXL memory device connected to the second computing device among the multiple second CXL memory devices in the above cluster topology table. For example, as shown in Table 1, taking the multiple second CXL memory devices as the second CXL memory device 1, the second CXL memory device 2, and the second CXL memory device 3, and the second computing device as the second computing device B as an example, then in the cluster topology table, according to the device identifiers of the multiple second CXL memory devices and the device identifier of the second computing device B, it can be determined that the device identifier of the target CXL memory device connected to node_B is CXL_1, that is, the target CXL memory device connected to the second computing device B is the target CXL memory device 1.

[0204] Exemplarily, after determining the target CXL memory device, the memory management module may look up the second memory address and the region size of the corresponding second memory region in the above-mentioned shared data information table according to the device identifier and the data identifier corresponding to the target CXL memory device. For example, as shown in Table 2, taking the target CXL memory device as CXL memory device 1 and the data identifier as obj_A, in the shared data information table, according to the device identifier CXL_1 of the target CXL memory device 1 and the data identifier obj_A, it can be determined that the second memory address of the second memory region storing the corresponding shared data is 0x10000000 + 0x00000000, and the region size of the second memory region is 8GB.

[0205] Step 840, the target computing device returns the second memory address to the second computing device.

[0206] In some embodiments, after the memory management module determines the region information of the second memory region corresponding to the above-mentioned target CXL memory device, it returns the region information of the second memory region corresponding to the target CXL memory device to the SDK module through the LMD module in the second computing device.

[0207] In some embodiments, after the memory management module returns the region information of the second memory region corresponding to the target CXL memory device to the SDK, it can increment the occupancy count of the corresponding shared data by 1.

[0208] Step 850, the second computing device reads the shared data from the second memory region based on the second memory address.

[0209] In some embodiments, after the SDK module in the second computing device receives the region information of the second memory region corresponding to the target CXL memory device, it reads the corresponding shared data in the second memory regions corresponding to the target CXL memory devices according to the second memory addresses and the region sizes of the second memory regions corresponding to the target CXL memory devices. Among them, the size of the second memory region is equal to the size of the shared data.

[0210] In some embodiments, the SDK module may determine the starting read position according to the address information of the second memory region, and determine the ending read position according to the region size and the starting read position; the SDK module reads the shared data in the second memory regions corresponding to the target CXL memory devices according to the starting read position and the ending read position (i.e., reads the data copy corresponding to the shared data).

[0211] Continuing with the above example, the SDK module uses the address information 0x10000000 + 0x00000000 in the second memory area as the starting read position, and determines the ending read position based on the area size of 8GB and the starting read position; then, according to the starting read position and the ending read position, it reads 8GB of shared data in the second memory area corresponding to each target CXL memory device.

[0212] Through the above solution, the computing device can realize the reading of shared data in the CXL memory device through the interaction with the memory management module, so that the computing device can quickly read the shared data in the CXL memory device to improve the reading speed of the shared data.

[0213] Figure 9 An interaction diagram for determining the data reading status provided by an embodiment of this application, as Figure 9 shown, the above method further includes steps 910 to 930.

[0214] Step 910, in response to a data reading request, the target computing device determines whether the shared data exists based on the identifier of the shared data.

[0215] In some embodiments, when the memory management module in the target computing device responds to a data reading request, it can first determine whether there is a data identifier in the data reading request in the above shared data information table. If the data identifier does not exist in the shared data information table, the memory management module can send a message indicating that the shared data does not exist to the SDK module through the LMD module in the second computing device to notify the SDK module that the shared data corresponding to the data identifier does not exist in the CXL memory device.

[0216] Step 920, in the case where the shared data exists, the target computing device determines whether the attribute of the shared data is read-only.

[0217] In some embodiments, if the data identifier exists in the shared data information table, the memory management module can look up the reading status of the shared data corresponding to the data identifier in the shared data information table. If the data identifier does not exist in the shared data information table, the memory management module can send a message indicating that the shared data does not exist to the second computing device to notify the SDK module that the shared data corresponding to the data identifier is currently not readable.

[0218] For example, as shown in Table 2, taking the data identifier received by the memory management module as obj_A_1, the memory management module can determine that the reading status of the shared data corresponding to the data identifier obj_A_1 in the shared data information table is the read-only status.

[0219] For another example, as shown in Table 2, taking the data identifier received by the memory management module as obj_B_1 as an example, the memory management module can determine that the reading status of the shared data corresponding to the data identifier obj_B_1 in the shared data information table is the unreadable status.

[0220] Step 930, when the reading status of the shared data is the unreadable status, the target computing device sends information indicating that the data is unreadable to the second computing device.

[0221] In some embodiments, when the reading status of the shared data is the unreadable status, the memory management module sends information indicating that the data is unreadable to the second computing device to notify the SDK module that the shared data corresponding to the data identifier is currently unreadable.

[0222] In some embodiments, when the reading status of the shared data is the read-only status, the memory management module can continue to process the data reading request of the second computing device and determine the second memory address from multiple first memory addresses.

[0223] Through the above solution, after receiving the data reading request of the second computing device and the data identifier corresponding to the shared data to be read, the memory management module can first verify the data identifier and the reading status of the shared data to avoid the computing device reading incomplete shared data.

[0224] In some embodiments, the above method further includes: the second computing device sends a data reading success response to the target computing device; wherein, the shared data reading success response is used to indicate that the shared data is read successfully; the target computing device receives the data reading success response sent by the second computing device; the target computing device increases the number of dependency relationships of the shared data according to a preset step.

[0225] In some embodiments, when the second computing device reads the shared data in the target CXL memory device, it can send a data reading success response to the memory management module in the target computing device through the SDK module and the LMD module to notify the memory management module that the second computing device has successfully read the shared data. Then, the memory management module can receive the data reading success response and increase the number of dependency relationships (i.e., the occupancy number) of the shared data according to a preset step, such as increasing the occupancy number of the shared data by 1 to indicate that there is a computing device in the cluster that is reading the shared data.

[0226] Figure 10 An interaction diagram for data release processing provided by an embodiment of the present application, as Figure 10 shown, the above method further includes steps 1010 to 1030.

[0227] Step 1010, the second computing device sends a data release request to the target computing device.

[0228] In some embodiments, after the SDK module in the second computing device finishes reading the shared data, it can send a data release request to the memory management module in the target computing device through the LMD module, so that the memory management module releases the occupancy of the shared data by the second computing device. Among them, the data release request includes the data identifier corresponding to the shared data to be released.

[0229] Step 1020, in response to the data release request, the target computing device reduces the number of dependency relationships of the shared data according to a preset step.

[0230] In some embodiments, after the memory management module receives the data release request sent by the SDK module, in response to the data release request, it determines the occupancy quantity of the corresponding shared data in its second CXL memory device according to the data identifier in the data release request. Exemplarily, the memory management module can determine the occupancy quantity of the shared data corresponding to the data identifier in the above shared data information table.

[0231] For example, as shown in Table 2, taking the data identifier received by the memory management module as obj_A_1, the memory management module can determine that the occupancy quantity of the shared data corresponding to the data identifier obj_A_1 in the CXL memory device 2 is 0, that is, there is no computing device in the cluster currently reading the shared data with the data identifier obj_A_2 in the CXL memory device 1.

[0232] For another example, as shown in Table 2, taking the data identifier received by the memory management module as obj_A_2, the memory management module can determine that the occupancy quantity of the shared data corresponding to the data identifier obj_A_2 in the CXL memory device 2 is 1, that is, there is 1 computing device in the cluster currently reading the shared data with the data identifier obj_A_2 in the CXL memory device 2.

[0233] In some embodiments, when the occupancy quantity of the shared data corresponding to the data identifier is not 0, the memory management module reduces the number of dependency relationships of the shared data according to a preset step, such as reducing the occupancy quantity of the shared data by 1, so as to release the occupancy of the shared data by the second computing device.

[0234] Step 1030, when the occupancy quantity of the shared data is 0, the target computing device sends a data release failure message to the second computing device.

[0235] In some embodiments, when the occupancy count of the data block corresponding to the data identifier for sharing data is 0, it is not possible to further reduce the occupancy count. Thus, the memory management module can send a data release failure message to the SDK module through the LMD module of the second computing device.

[0236] Through the above solution, the computing device can release the occupancy of the shared data in the CXL memory device through the memory management module; moreover, the memory management device can determine the occupancy count of the shared data to release the occupancy of the shared data by the computing device only when the occupancy count is greater than 0.

[0237] Figure 11 An interaction diagram for data recovery processing provided by an embodiment of this application is as Figure 11 shown, and the above method further includes steps 1110 to 1150.

[0238] Step 1110, when the second computing device fails to read the shared data, it sends a data read failure response to the target computing device.

[0239] In some embodiments, when the SDK module in the second computing device fails to read the shared data, the SDK module in the second computing device can send a data read failure response to the memory management module in the target computing device through the LMD module in the second computing device. Among them, the data read failure response includes the data identifier corresponding to the shared data for which the read fails.

[0240] Exemplarily, the reason for the second computing device to fail to read the shared data may be that the second memory area in the second CXL memory device is damaged, faulty, etc., or it may be because the shared data fails the integrity check. After the second computing device reads the shared data, it can perform an integrity check (such as a hash check) on the read shared data to determine whether the shared data has been tampered with.

[0241] Step 1120, the target computing device receives the data read failure response and determines a third memory area from the second CXL memory device corresponding to the second memory address.

[0242] In some embodiments, after the memory management module receives the data read failure response sent by the LMD module in the second computing device, it can reallocate a third memory area in the second CXL device where the second computing device reads the shared data based on the size of the shared data to store the recovered data copy, and determine the area information corresponding to the third memory area. The area size of the third memory area is equal to the size of the shared data, and the third memory area is a memory area in the second CXL memory device different from the second memory area.

[0243] In some embodiments, the memory management module may determine the source computing device (such as the first computing device) for writing the shared data according to the data identifier. For example, as shown in Table 2, taking the data identifier received by the memory management module as obj_A_1, the memory management module may determine that the device identifier of the source computing device corresponding to the data identifier obj_A_1 in the shared data information table is node_A, that is, the source computing device corresponding to this data identifier is the first computing device A.

[0244] Step 1130, the target computing device sends a data recovery command to the first computing device.

[0245] In some embodiments, after the memory management module determines the area information corresponding to the third memory area in the second CXL memory device, it may update the area information of the second memory area corresponding to the above data identifier in the shared data information table to the area information corresponding to this third memory area; and send a data recovery command to the LMD module in the first computing device to instruct the first computing device to recover the shared data. Among them, the data recovery command includes the area information corresponding to the third memory area and the data identifier corresponding to the shared data to be recovered, and the area information corresponding to the third memory area includes the third memory address and the area size corresponding to the third memory area.

[0246] Step 1140, the first computing device receives the data recovery command sent by the target computing device and reads the shared data based on the source address.

[0247] In some embodiments, the LMD module in the first computing device may determine the storage location (i.e., the source address) of the corresponding shared data in the first computing device according to the data identifier to obtain the shared data.

[0248] In some embodiments, if the first computing device does not retain the local shared data after writing the shared data into the second CXL memory device (that is, the shared data has been deleted from the first computing device), the LMD module in the first computing device may obtain a data copy of the shared data from other CXL memory devices (such as the fourth CXL memory device) connected to the first computing device and storing the shared data.

[0249] Exemplarily, the memory management module may determine the fourth CXL memory device in the above cluster topology table and determine the area information of the second memory area corresponding to this fourth CXL memory device in the above shared data information table.

[0250] For example, as shown in Table 1, taking the data identifier received by the memory management module as obj_A_1, the first computing device as the first computing device A, and the second CXL memory device whose stored shared data cannot be read as the second CXL memory device 1 as an example, the memory management module can determine, in the cluster topology table, the device identifiers corresponding to the multiple first CXL memory devices connected to the first computing device A as CXL_1, CXL_2, and CXL_3 according to the device identifier node_A corresponding to the first computing device A. Thus, the first CXL memory device with the device identifier CXL_2 or CXL_3 can be determined as the fourth CXL memory device. That is, the fourth CXL memory device can be the fourth CXL memory device 2 or the fourth CXL memory device 3. Then, as shown in Table 2, the memory management module determines, in the shared data information table, the region information of the second memory region storing the shared data obj_A_1 corresponding to the fourth CXL memory device 2 or the fourth CXL memory device 3 according to the data identifier obj_A_1 and the device identifier corresponding to the fourth CXL memory device 2 or the fourth CXL memory device 3.

[0251] In some embodiments, after determining the fourth CXL memory device, the memory management module sends the region information of the second memory region corresponding to the fourth CXL memory device to the LMD module in the first computing device, so that the LMD module in the first computing device reads the corresponding shared data in the second memory region corresponding to the fourth CXL memory device.

[0252] Continuing with the above example, the LMD module in the first computing device can read the shared data obj_A_2 in the second memory region corresponding to the fourth CXL memory device 2, or read the shared data obj_A_3 in the second memory region corresponding to the fourth CXL memory device 3.

[0253] It should be noted that the specific method of reading the shared data in the second memory region corresponding to the fourth CXL memory device is the same as the specific method of reading the shared data in the second memory region corresponding to the second CXL memory device, and will not be elaborated here.

[0254] Step 1150, the first computing device writes the shared data into the third memory region based on the third memory address.

[0255] In some embodiments, after receiving the region information of the third memory region corresponding to the second CXL memory device and the data identifier corresponding to the shared data to be restored, the LMD module in the first computing device can obtain the shared data to be restored according to the data identifier, and write the shared data into the third memory region corresponding to the second CXL memory device based on the region information of the third memory region to complete the restoration of the shared data.

[0256] With the above solution, in the case where the second computing device fails to read the shared data, the second computing device can request the memory management device to restore the shared data, and the memory management device can interact with the first computing device that wrote the shared data to achieve the restoration of the shared data in the CXL memory device, so that the second computing device can successfully read the complete shared data.

[0257] Figure 12 Another interaction diagram of data reading processing provided by the embodiments of this application is as Figure 12 shown, and the above method further includes steps 1210 to 1230.

[0258] Step 1210, the first computing device sends a data restoration success response to the target computing device.

[0259] In some embodiments, after the LMD module in the first computing device writes the shared data into the third memory area corresponding to the second CXL memory device, it can send a data restoration success response to the memory management module to notify the memory management module that the shared data has been restored. Among them, the data restoration success response may include the data identifier of the restored shared data.

[0260] Step 1220, the target computing device receives the data restoration success response sent by the first computing device and sends the area information of the third memory area to the second computing device.

[0261] In some embodiments, after the memory management module receives the data restoration success response sent by the LMD module in the first computing device, it can send the area information of the third memory area corresponding to the second CXL memory device to the SDK module in the second computing device through the LMD module in the second computing device.

[0262] Step 1230, the second computing device reads the shared data based on the area information of the third memory area.

[0263] In some embodiments, after the SDK module in the second computing device receives the area information of the third memory area, it rereads the shared data in the third memory area corresponding to the second CXL memory device based on the area information of the third memory area.

[0264] It should be noted that the method of reading the shared data in the third memory area corresponding to the second CXL memory device is the same as the specific method of reading the shared data in the second memory area corresponding to the second CXL memory device, and will not be elaborated here.

[0265] After the first computing device writes the shared data to the third memory area corresponding to the second CXL memory device, it can send a data recovery success response to the memory management module to notify the memory management module that the shared data has been recovered, so that the second computing device can read the recovered shared data again.

[0266] Applying the technical solution of the present application, each computing device in the cluster can store the shared data therein in multiple CXL memory devices to form multiple data copies, so that the computing devices in the cluster can efficiently read the data copies through the CXL memory devices and improve the channel utilization rate. Moreover, the memory management module can manage and allocate the memory areas in the CXL memory devices based on the cluster topology, and can process requests such as writing, reading, releasing, deleting, and recovering of the shared data by each computing device, so as to realize efficient management of distributed data.

[0267] Figure 13 It is a schematic diagram of a processing device for distributed data provided by an embodiment of the present application. As Figure 13 shown, the distributed data processing device 1300 is configured in Figure 1 the target computing device shown, and the target computing device is any computing device in the computing system; the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; each CXL memory device among the multiple CXL memory devices is connected to at least two computing devices. The distributed data processing device 1300 includes a first receiving module 1310, a determining module 1320, and a first sending module 1330.

[0268] The first receiving module 1310 is configured to receive a data writing request sent by a first computing device.

[0269] Among them, the first computing device is any computing device in the computing system; the data writing request is used to request to write the shared data to the first CXL memory device; the first CXL memory device is the CXL memory device connected to the first computing device among the multiple CXL memory devices; the data writing request includes the number of copies of the shared data and the size of the shared data; the number of copies is a positive integer greater than 1.

[0270] The determining module 1320 is configured to determine multiple second CXL memory devices from the multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data.

[0271] Among them, the second CXL memory device is the first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data.

[0272] The first sending module 1330 is configured to send multiple first memory addresses corresponding to multiple first memory regions to a first computing device, so that the first computing device writes shared data into the multiple first memory regions based on the multiple first memory addresses.

[0273] Wherein, the first memory region is an idle memory region in the second CXL memory device for writing shared data.

[0274] As Figure 13 shown, the distributed data processing device 1300 further includes: an acquisition module 1340.

[0275] In some embodiments, the acquisition module 1340 is configured to: acquire the connection relationships between each CXL memory device in the computing system and each computing device in the multiple computing devices; the determination module 1320 is further configured to determine multiple first CXL memory devices from the multiple CXL memory devices according to the connection relationships.

[0276] In some embodiments, the determination module 1320 is specifically configured to: acquire the number of third CXL memory devices; wherein, the third CXL memory device is a first CXL memory device whose capacity of the idle memory region matches the size of the shared data; determine whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; in the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, select multiple second CXL memory devices from the multiple third CXL memory devices based on the number of copies of the shared data.

[0277] As Figure 13 shown, the distributed data processing device 1300 further includes: a fourth receiving module 1350 and a setting module 1360.

[0278] In some embodiments, the fourth receiving module 1350 is configured to: receive a data locking request; the data locking request is used to indicate setting the attribute of the shared data to read-only; the determination module 1320 is further configured to: determine whether the data locking request comes from the first computing device; the setting module 1360 is configured to: in the case where the data locking request comes from the first computing device, set the attribute of the shared data to read-only.

[0279] In some embodiments, the fourth receiving module 1350 is further configured to: receive a data reading request sent by a second computing device; wherein the second computing device is any computing device in the computing system; the data reading request is used to indicate reading shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used to identify the shared data; the determining module 1320 is further configured to: determine a plurality of first memory addresses from the associated data based on the identifier of the shared data; wherein the associated data includes the identifier of the shared data and a plurality of first memory addresses; determine a second memory address from the plurality of first memory addresses; wherein the second memory address is the address of a second memory area; the second memory area is the shortest access path for the second computing device to read the shared data; the first sending module 1330 is further configured to: return the second memory address to the second computing device, so that the second computing device reads the shared data from the second memory area based on the second memory address.

[0280] In some embodiments, the determining module 1320 is further configured to: determine whether the shared data exists based on the identifier of the shared data; in the case where the shared data exists, determine whether the attribute of the shared data is read-only; specifically, the determining module 1320 is configured to: in the case where the attribute of the shared data is read-only, determine the second memory address from the plurality of first memory addresses.

[0281] In some embodiments, the fourth receiving module 1350 is further configured to: receive a data reading failure response sent by the second computing device; wherein the data reading failure response is used to indicate that the second computing device fails to read the shared data; the determining module 1320 is further configured to: determine a third memory area from the second CXL memory device corresponding to the second memory address; wherein the third memory area is used to write the shared data; the first sending module 1330 is further configured to: send a data recovery command to the first computing device; wherein the data recovery command is used to indicate recovering the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address for reading the shared data to be recovered; the third memory address is used to indicate the memory address of the third memory area for writing the shared data.

[0282] In some embodiments, the fourth receiving module 1350 is further configured to: receive a data recovery success response sent by the first computing device; wherein the data recovery success response is used to indicate that the shared data is successfully recovered; the first sending module 1330 is further configured to: send the third memory address to the second computing device, so that the second computing reads the shared data based on the third memory address.

[0283] As Figure 13 shown, the distributed data processing apparatus 1300 further includes: an update module 1370.

[0284] In some embodiments, the update module 1370 is configured to update the second memory address in the associated data based on the third memory address.

[0285] In some embodiments, the fourth receiving module 1350 is further configured to receive a data read success response sent by the second computing device; wherein, the data read success response is used to indicate that the shared data has been successfully read; the update module 1370 is further configured to increase the number of dependencies of the shared data by a preset step.

[0286] In some embodiments, the fourth receiving module 1350 is further configured to receive a data release request sent by the second computing device; wherein, the data release request is used to indicate the release of the second computing device's occupancy of the shared data; the update module 1370 is further configured to decrease the number of dependencies of the shared data by a preset step.

[0287] In some embodiments, the fourth receiving module 1350 is further configured to receive a data deletion request; wherein, the data deletion request is used to indicate the deletion of the shared data from the first memory area; the data deletion request includes the identifier of the shared data; the determination module 1320 is further configured to determine whether the identifier of the shared data exists, whether the data deletion request originates from the first computing device, and whether the number of dependencies of the shared data is zero; the first sending module 1330 is further configured to, when the identifier of the shared data exists, the data deletion request originates from the first computing device, and the number of dependencies of the shared data is not zero, send the identifier of the shared data, multiple first memory addresses, and a deletable flag to the first computing device, so that the first computing device deletes the shared data based on the multiple first memory addresses; wherein, the deletable flag is used to indicate that the shared data can be deleted.

[0288] Figure 14 This is a schematic diagram of another distributed data processing device provided by the embodiments of the present application. As Figure 14 shown, the distributed data processing device 1400 is configured in Figure 1 the first computing device among the computing devices shown, the first computing device is any computing device in the computing system, the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; the first computing device is connected to multiple CXL memory devices. The distributed data processing device 1400 includes a second sending module 1410, a second receiving module 1420, and a writing module 1430.

[0289] The second sending module 1410 is configured to send a data writing request to the target computing device.

[0290] Among them, the target computing device is any computing device in the computing system; the data writing request is used to request to write shared data into the first CXL memory device; the data writing request includes the size of the shared data and the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to the first computing device.

[0291] A second receiving module 1420, configured to receive multiple first memory addresses corresponding to multiple first memory regions sent by the target computing device.

[0292] Among them, the first memory region is a memory region in the second CXL memory device for writing shared memory; the second CXL memory device is a first CXL memory device whose capacity of the free memory region matches the size of the shared data; the number of second CXL memory devices is the number of copies of the shared data.

[0293] A writing module 1430, configured to write the shared data into multiple first memory regions based on the multiple first memory addresses.

[0294] As Figure 14 shown, the distributed data processing device 1400 further includes: a fourth sending module 1440.

[0295] In some embodiments, the fourth sending module 1440 is configured to: send a data locking request to the target computing device; wherein, the data locking request is used to indicate setting the attribute of the shared data to read-only.

[0296] As Figure 14 shown, the distributed data processing device 1400 further includes: a second reading module 1450.

[0297] In some embodiments, the second receiving module 1420 is further configured to: receive a data recovery command sent by the target computing device; wherein, the data recovery command is used to indicate recovering the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address storing the shared data in the first computing device; the third memory address is used to indicate the memory address of the third memory region; the third memory region is located in the second CXL memory device corresponding to the second memory address; the second memory address is the memory address where the second computing device reads the shared data; the second reading module 1450 is configured to: read the shared data based on the source address; the writing module 1430 is further configured to: write the shared data into the third memory region based on the third memory address.

[0298] In some embodiments, the fourth sending module 1440 is further configured to: send a data recovery success response to the target computing device; wherein, the data recovery success response is used to indicate that the writing of the shared data into the third memory region is completed.

[0299] As Figure 14 shown, the distributed data processing device 1400 further includes: a deletion module 1460.

[0300] In some embodiments, the fourth sending module 1440 is further configured to: send a data deletion request to a target computing device; wherein, the data deletion request is used to instruct to delete the shared data from the first CXL memory device; the second receiving module 1420 is further configured to: receive the identifier of the shared data sent by the target computing device, the storage address in the first CXL memory device, and a deletable flag; wherein, the deletable flag is used to indicate that the shared data is deletable; the identifier of the shared data is used to identify the shared data; the deletion module 1460 is configured to: delete the shared data based on the identifier of the shared data and the storage address in the plurality of first CXL memory devices.

[0301] Figure 15 This is a schematic diagram of another distributed data processing device provided by the embodiments of the present application. As Figure 15 shown, the distributed data processing device 1500 is configured in Figure 1 the second computing device in the computing device shown, the second computing device is any computing device in the computing system, the computing system includes a plurality of computing devices and a plurality of Compute Express Link (CXL) memory devices; the second computing device is connected to at least one CXL memory device. The distributed data processing device 1500 includes a third sending module 1510, a third receiving module 1520, and a first reading module 1530.

[0302] The third sending module 1510 is configured to send a data reading request to a target computing device.

[0303] Wherein, the target computing device is any computing device in the computing system; the data reading request is used to obtain the storage address of the shared data; the data reading request includes the identifier of the shared data; the identifier of the shared data is used for the shared data.

[0304] The third receiving module 1520 is configured to receive the second memory address corresponding to the second memory area returned by the target computing device.

[0305] Wherein, the second memory area is the shortest access path for the second computing device to read the shared data in the first memory area; the first memory area is the memory area in the second CXL memory device for writing the shared memory; the second CXL memory device is the first CXL memory device whose free memory area capacity matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is the CXL memory device connected to the first computing device; the first computing device is the computing device that writes the shared data.

[0306] The first reading module 1530 is configured to read shared data from the second memory area based on the second memory address.

[0307] In some embodiments, the third sending module 1510 is further configured to: send a data reading failure response to the target computing device; wherein, the reading failure response is used to indicate that the second computing device fails to read the shared data.

[0308] In some embodiments, the third receiving module 1520 is further configured to: receive a third memory address sent by the target computing device; wherein, the third memory address is used to indicate the memory address of the third memory area; the third memory area is used to write the recovered shared data; the third memory area is located in the second CXL memory device corresponding to the second memory address; the first reading module 1530 is further configured to: read the shared data from the third memory area based on the third memory address.

[0309] In some embodiments, the third sending module 1510 is further configured to: send a data reading success response to the target computing device; wherein, the shared data reading success response is used to indicate that the shared data is successfully read.

[0310] In some embodiments, the third sending module 1510 is further configured to: send a data release request to the target computing device; wherein, the data release request is used to indicate releasing the occupancy of the shared data by the second computing device.

[0311] Figure 16 A schematic diagram of a computing device provided in some embodiments of the present application. In some embodiments, the computing device may be a server, a terminal device, etc. The computing device includes one or more processors and a memory. The memory is configured to: store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the distributed data processing method in the above embodiments.

[0312] As Figure 16 shown, the computing device 1600 includes: a processor 1601 and a memory 1602. Exemplarily, the computing device 1600 may further include: a communications interface 1603 and a communication bus 1604.

[0313] Wherein, the processor 1601, the memory 1602, and the communications interface 1603 communicate with each other through the communication bus 1604. The communications interface 1603 is used to communicate with network elements of other devices such as clients or other servers.

[0314] In some embodiments, the processor 1601 is configured to execute the program 1605, and specifically may execute the relevant steps in the above-described embodiments of the method for processing distributed data. Specifically, the program 1605 may include program code, and the program code includes computer-executable instructions.

[0315] Exemplarily, the processor 1601 may be a central processing unit (CPU), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement some embodiments of the present application. The computing device 1600 may include one or more processors, which may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0316] In some embodiments, the memory 1602 is used to store the program 1605. The memory 1602 may include high-speed RAM memory, and may also include non-volatile memory (NVM), such as at least one disk memory.

[0317] The program 1605 may specifically be called by the processor 1601 to cause the computing device 1600 to perform the operations of the method for processing distributed data.

[0318] Some embodiments of the present application provide a computer-readable storage medium storing at least one executable instruction, which, when running on the computing device 1600, causes the computing device 1600 to execute the method for processing distributed data in the above embodiments.

[0319] The executable instruction may specifically be used to cause the computing device 1600 to perform the operations of the method for processing distributed data.

[0320] For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0321] The beneficial effects achievable by the readable storage medium provided by some embodiments of the present application may refer to the beneficial effects in the corresponding method for processing distributed data provided above, and will not be elaborated herein.

[0322] It should be noted that in this application, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0323] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the description of the method embodiments.

[0324] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus or device), or in conjunction with these instruction execution systems, apparatus or devices.

[0325] For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus or device.

[0326] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM).

[0327] In addition, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, deciphering or otherwise processing it as appropriate, and then storing it in a computer memory. It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof.

[0328] In the above-described embodiments, the multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0329] The above-described embodiments are only specific embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing distributed data, characterized in that, Applied to a target computing device, where the target computing device is any computing device in a computing system; the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; among them, each of the multiple CXL memory devices is connected to at least two of the computing devices; The method includes: Receiving a data write request sent by a first computing device; where the first computing device is any computing device in the computing system; the data write request is used to request writing shared data into a first CXL memory device; the first CXL memory device is a CXL memory device among the multiple CXL memory devices that is connected to the first computing device; the data write request includes the number of copies of the shared data and the size of the shared data; the number of copies is a positive integer greater than 1; Based on the number of copies of the shared data and the size of the shared data, determining multiple second CXL memory devices from among the multiple first CXL memory devices; where the second CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of second CXL memory devices is equal to the number of copies of the shared data; Sending multiple first memory addresses corresponding to the multiple first memory areas to the first computing device, so that the first computing device writes the shared data into the multiple first memory areas based on the multiple first memory addresses; where the first memory area is the free memory area in the second CXL memory device for writing the shared data.

2. The method according to claim 1, characterized in that Before determining multiple second CXL memory devices from among the multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data, the method further includes: Obtaining the connection relationship between each of the CXL memory devices in the computing system and each of the computing devices in the multiple computing devices; Based on the connection relationship, determining multiple first CXL memory devices from among the multiple CXL memory devices.

3. The method according to claim 1 or 2, characterized in that, The determining multiple second CXL memory devices from among the multiple first CXL memory devices based on the number of copies of the shared data and the size of the shared data includes: Obtaining the number of third CXL memory devices; where the third CXL memory device is a first CXL memory device whose capacity of the free memory area matches the size of the shared data; Determining whether the number of third CXL memory devices is greater than or equal to the number of copies of the shared data; In the case where the number of third CXL memory devices is greater than or equal to the number of copies of the shared data, selecting the multiple second CXL memory devices from among the multiple third CXL memory devices based on the number of copies of the shared data.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Receiving a data lock request; the data lock request is used to indicate setting the attribute of the shared data to read-only; [[ID= In the case where the data locking request originates from the first computing device, set the attribute of the shared data to read-only.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Receiving a data reading request sent by a second computing device; wherein, the second computing device is any computing device in the computing system; the data reading request is used to indicate reading the shared data; the data reading request includes an identifier of the shared data; the identifier of the shared data is used to identify the shared data; Determining the multiple first memory addresses from the associated data based on the identifier of the shared data; wherein, the associated data includes the identifier of the shared data and the multiple first memory addresses; Determining a second memory address from the multiple first memory addresses; wherein, the second memory address is the address of a second memory area; the second memory area is the shortest access path for the second computing device to read the shared data; Returning the second memory address to the second computing device, so that the second computing device reads the shared data from the second memory area based on the second memory address.

6. The method according to claim 5, before determining the second memory address from the multiple first memory addresses, the method further includes: Determining whether the shared data exists based on the identifier of the shared data; In the case where the shared data exists, determining whether the attribute of the shared data is read-only; The determining the second memory address from the multiple first memory addresses includes: In the case where the attribute of the shared data is read-only, determining the second memory address from the multiple first memory addresses.

7. The method according to claim 5 or 6, the method further includes: Receiving a data reading failure response sent by the second computing device; wherein, the data reading failure response is used to indicate that the second computing device fails to read the shared data; Determining a third memory area from the second CXL memory device corresponding to the second memory address; wherein, the third memory area is used to write the shared data; Sending a data recovery command to the first computing device; wherein, the data recovery command is used to indicate recovering the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address for reading the shared data to be recovered; the third memory address is used to indicate the memory address of the third memory area for writing the shared data.

8. The method according to claim 7, the method further includes: Receiving a data recovery success response sent by the first computing device; wherein, the data recovery success response is used to indicate that the shared data is recovered successfully; Sending the third memory address to the second computing device, so that the second computing reads the shared data based on the third memory address.

9. The method according to claim 8, characterized in that The method further includes: Updating the second memory address in the associated data based on the third memory address.

10. The method according to claim 5, wherein The method further includes: Receiving a data reading success response sent by the second computing device; wherein, the data reading success response is used to indicate that the shared data is read successfully; Increase the number of dependency relationships of the shared data according to a preset step.

11. The method according to any one of claims 5-10, characterized in that, The method further includes: Receiving a data release request sent by the second computing device; wherein, the data release request is used to indicate releasing the occupancy of the shared data by the second computing device; Decrease the number of dependency relationships of the shared data according to the preset step.

12. The method according to any one of claims 1-11, characterized in that, The method further includes: Receiving a data deletion request; wherein, the data deletion request is used to indicate deleting the shared data from the first memory area; the data deletion request includes an identifier of the shared data; Determining whether the identifier of the shared data exists, whether the data deletion request originates from the first computing device, and whether the number of dependency relationships of the shared data is zero; When the identifier of the shared data exists, the data deletion request originates from the first computing device, and the number of dependency relationships of the shared data is not zero, sending the identifier of the shared data, the multiple first memory addresses, and a deletable flag to the first computing device, so that the first computing device deletes the shared data based on the multiple first memory addresses; wherein, the deletable flag is used to indicate that the shared data is deletable.

13. A method for processing distributed data, characterized in that, Applied to a first computing device; the first computing device is any computing device in a computing system, and the computing system includes multiple computing devices and multiple Compute Express Link (CXL) memory devices; The first computing device is connected to multiple of the CXL memory devices; The method includes: Sending a data write request to a target computing device; Wherein, the target computing device is any computing device in the computing system; the data write request is used to request writing shared data into a first CXL memory device; the data write request includes the size of the shared data and the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is a CXL memory device connected to the first computing device; Receiving multiple first memory addresses corresponding to multiple first memory areas sent by the target computing device; wherein, the first memory area is a memory area in a second CXL memory device for writing the shared memory; the second CXL memory device is the first CXL memory device whose capacity of the free memory area matches the size of the shared data; the number of the second CXL memory devices is the number of copies of the shared data; Writing the shared data into the multiple first memory areas based on the multiple first memory addresses.

14. The method according to claim 13, wherein The method further includes: Sending a data lock request to the target computing device; wherein, the data lock request is used to indicate setting the attribute of the shared data to read-only.

15. The method according to claim 13 or 14, characterized in that, The method further includes: Receive a data recovery command sent by the target computing device; wherein, the data recovery command is used to indicate the recovery of the shared data; the data recovery command includes a source address and a third memory address; the source address is used to indicate the memory address in the first computing device where the shared data is stored; the third memory address is used to indicate the memory address of a third memory area; the third memory area is located in the second CXL memory device corresponding to the second memory address; the second memory address is the memory address where the second computing device reads the shared data. Read the shared data based on the source address. Write the shared data into the third memory area based on the third memory address.

16. The method according to claim 15, the method further includes: Send a data recovery success response to the target computing device; wherein, the data recovery success response is used to indicate the completion of writing the shared data into the third memory area.

17. The method according to any one of claims 13-16, characterized in that, The method further includes: Send a data deletion request to the target computing device; wherein, the data deletion request is used to indicate deleting the shared data from the first CXL memory device. Receive the identifier of the shared data, the storage address in the first CXL memory device, and a deletable flag sent by the target computing device; wherein, the deletable flag is used to indicate that the shared data is deletable; the identifier of the shared data is used to identify the shared data. Delete the shared data based on the identifier of the shared data and the storage address in the plurality of first CXL memory devices.

18. A method for processing distributed data, characterized in that Applied to a second computing device; the second computing device is any computing device in a computing system, the computing system includes a plurality of computing devices and a plurality of Compute Express Link (CXL) memory devices. The second computing device is connected to at least one of the CXL memory devices. The method includes: Send a data read request to a target computing device; wherein, the target computing device is any computing device in the computing system; the data read request is used to obtain the storage address of the shared data; the data read request includes the identifier of the shared data; the identifier of the shared data is used for the shared data. Receive the second memory address corresponding to the second memory area returned by the target computing device; wherein, the second memory area is the shortest access path for the second computing device to read the shared data in the first memory area; the first memory area is the memory area in the second CXL memory device used to write the shared memory; the second CXL memory device is the first CXL memory device whose free memory area capacity matches the size of the shared data; the number of the second CXL memory devices is equal to the number of copies of the shared data; the number of copies is a positive integer greater than 1; the first CXL memory device is the CXL memory device connected to the first computing device; the first computing device is the computing device that writes the shared data. Read the shared data from the second memory area based on the second memory address.

19. The method according to claim 18, the method further includes: Send a data read failure response to the target computing device; wherein, the read failure response is used to indicate that the second computing device fails to read the shared data.

20. The method according to claim 19, the method further includes: Receive a third memory address sent by the target computing device; wherein, the third memory address is used to indicate the memory address of a third memory area; the third memory area is used to write the restored shared data; the third memory area is located in the second CXL memory device corresponding to the second memory address; Read the shared data from the third memory area based on the third memory address.

21. The method according to any one of claims 18 - 20, characterized in that, The method further includes: Send a data read success response to the target computing device; wherein, the shared data read success response is used to indicate that the shared data is read successfully.

22. The method according to any one of claims 18-21, characterized in that The method further includes: Send a data release request to the target computing device; wherein, the data release request is used to indicate releasing the occupancy of the shared data by the second computing device.

23. A computing device, characterized in that, Includes: A memory and a processor; the memory and the processor are connected; Wherein, the memory is used to store program instructions; The processor is used to execute the program instructions so that the computing device executes the distributed data processing method according to any one of claims 1-12, claims 13-17, or claims 18-22.

24. A computing system, characterized in that, The computing system includes a plurality of computing devices and a plurality of Compute Express Link (CXL) memory devices; wherein, each of the plurality of CXL memory devices is connected to at least two computing devices; the plurality of computing devices include a first computing device, a second computing device, and a target computing device; the target computing device is any one of the plurality of computing devices; Wherein, the target computing device is the computing device according to claim 23, and the target computing device executes the distributed data processing method according to any one of claims 1-12; The first computing device is the computing device according to claim 23, and the first computing device executes the distributed data processing method according to any one of claims 13-17; The second computing device is the computing device according to claim 23; the second computing device executes the distributed data processing method according to any one of claims 18-22.

Images