Data transmission method and computing device

Abstract

A data transmission method and a computing device. The method comprises: determining address information of a target memory space corresponding to target data, wherein the target memory space belongs to a first computing device, the target data is data to be sent to a second computing device by the first computing device, and the second computing device is used for receiving the target data; sending attribute information of the target data to the second computing device, wherein the attribute information comprises the address information; and sending first application information to a management device on the basis of the address information, wherein the first application information is used for instructing the management device to modify the attribution of the target memory space from the first computing device to the second computing device, so that the second computing device obtains the target data.

Description

Data transmission method and computing device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 6, 2023, with application number 202311666270.1 and application name “A method and computing device for data transmission”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of server technology, and in particular to a data transmission method and computing device. Background Art

[0003] Data transmission technology is one of the most core technologies in modern communication networks. The most critical issue in data transmission is transmitting data from one entity to another while ensuring its accuracy and integrity. With technological advancements, the volume of data that needs to be transmitted is increasing. Therefore, efficient data transmission has become an increasingly important research topic.

[0004] In the prior art, physical media or network media are usually used to complete the transmission and migration of data from one subject to another, wherein data from one subject is mainly transferred to another subject via network transmission.

[0005] However, the above solutions place high demands on network transmission bandwidth, latency, and performance, and there is also the risk of data loss during network transmission. This can lead to low data transmission efficiency when dealing with large amounts of data.

[0006] Summary of the Invention

[0007] The present invention provides a data transmission method and computing device that can improve the efficiency of data transmission. The technical solution is as follows:

[0008] In a first aspect, a data transmission method is provided, which is executed by a first computing device and includes: determining address information of a target memory space corresponding to target data, the target memory space belonging to the first computing device, the target data being data that the first computing device needs to send to a second computing device, and the second computing device being used to receive the target data. Attribute information of the target data is sent to the second computing device, the attribute information including address information. Based on the address information, a first application message is sent to a management unit, the first application message being used to instruct the management unit to change the ownership of the target memory space from the first computing device to the second computing device, so that the second computing device obtains the target data.

[0009] It can be understood that by determining the address information of the target memory space corresponding to the target data and, based on the address information, sending a first request message to the management unit, the ownership of the target memory space is changed from the first computing device to the second computing device. By changing the ownership of the target memory space, the target memory space and the target data are transferred from the first computing device to the second computing device, reducing the overhead of the data copy process and the overhead of media transmission, greatly improving the efficiency of data transmission, saving network deployment costs, and reducing the risk of data loss and attack during data transmission, thereby improving the security of data transmission.

[0010] In a possible implementation, determining the address information of the target memory space corresponding to the target data includes acquiring the address information of the target memory space in the CXL memory pool through a local CXL space manager LMD.

[0011] It is understandable that obtaining the location information of the target memory space in the CXL memory pool through the local CXL space manager LMD can ensure the accuracy of the address information and reduce errors caused by changes in the ownership of the memory space in the CXL memory pool.

[0012] In one possible implementation, before determining the address information of the target memory space corresponding to the target data, the method further includes: determining the target data; sending second request information to a management unit; the second request information indicating the storage requirement of the target data; the management unit allocating target memory space in the CXL memory pool for the target data based on the storage requirement of the target data; the CXL memory pool providing memory space for storing data to the computing cluster; receiving the address information of the target memory space in the CXL memory pool sent by the management unit; and writing the target data into the target memory space according to the address information.

[0013] It is understood that the management unit determines the target memory space based on the second request information, where the second request information indicates the storage requirements of the target data. Therefore, determining the target memory space in the CXL memory pool based on the storage requirements of the target data ensures that the target data can be successfully written to the target memory space in the CXL memory pool.

[0014] In a possible implementation, sending the attribute information of the target data to the second computing device includes sending the attribute information of the target data to the second computing device through a traditional communication network.

[0015] It can be understood that the attribute information data sent by the first computing device is relatively small in size, and the second computing device not only reduces bandwidth pressure through the traditional communication network, but also improves the efficiency of attribute information transmission and further improves the transmission efficiency of target data.

[0016] In a possible implementation, first modification information sent by a management unit is received, where the first modification information is used to indicate that the target memory space belongs to the second computing device.

[0017] It can be understood that after the management unit completes the modification of the ownership of the target memory space, the first computing device receives the first modification information sent by the management unit. At this time, the ownership of the target memory space is the second computing device. The ownership modification of the target memory space is completed, and the second computing device can access the target memory space.

[0018] In a second aspect, a data transmission method is provided, performed by a second computing device, comprising: receiving attribute information sent by a first computing device, the attribute information including address information of target data; the address information indicating the location of a target memory space corresponding to the target data in a CXL memory pool, the target data being data that the first computing device needs to send to the second computing device, the CXL memory pool providing memory space for storing data to a computing cluster, and the first computing device sending the target data; receiving first modification information indicating that the target memory space belongs to the second computing device; and obtaining the target data based on the attribute information. The first modification information is sent by a management device after switching the ownership of the target memory space from the first computing device to the second computing device based on a request from the first computing device.

[0019] It is understood that the second computing device receives attribute information sent by the first computing device, which includes the address information of the target data. Based on the address information, the second computing device can retrieve the target data sent by the first computing device. This reduces media transmission overhead, saves network deployment costs, and significantly reduces the risk of data loss and attacks during transmission, thereby improving data transmission efficiency.

[0020] In a possible implementation, after receiving the first modification information and before obtaining the target data, the second computing device further manages the target memory space.

[0021] It is understandable that after the management unit completes the modification of the ownership of the target memory space, the ownership of the target memory space is now the second computing device. After the ownership modification of the target memory space is completed, the LMD of the second computing device can access the target memory space.

[0022] In a possible implementation, acquiring target data based on the attribute information includes, if the attribute information includes address information, accessing a target memory space based on the address information; and acquiring the target data from the target memory space.

[0023] It can be understood that the target memory space belongs to the second computing device. The second computing device can access the target memory space based on the address information and obtain the target data from the target memory space. This not only reduces the overhead of media transmission and saves network deployment costs, but also greatly reduces the risk of data loss and attack during transmission, thereby improving the efficiency of data transmission.

[0024] In one possible implementation, obtaining the target data from the target memory space includes, if the attribute information includes format information and size information of the target data, obtaining the target data from the target memory space according to the format information and size information of the target data.

[0025] It can be understood that the second computing device can directly obtain and use the target data according to the format information and size information of the target data, which saves data transmission time and greatly improves data transmission efficiency.

[0026] On the third aspect, a data transmission device is provided. The embodiment of the present application can divide the functional modules of the model training device according to the method provided in the first aspect above. For example, each functional module can be divided corresponding to each function, or two or more functions can be inherited in one processing module. Exemplarily, the embodiment of the present application can divide the model training device into a processing module and a sending module according to the function. The description of the possible technical solutions and beneficial effects executed by the above-mentioned divided functional modules can refer to the technical solutions provided by the first aspect or its corresponding possible implementation method, and will not be repeated here.

[0027] In a fourth aspect, a data transmission device is provided. An embodiment of the present application can divide the model training device into functional modules according to the method provided in the second aspect above. For example, each functional module can be divided corresponding to each function, or two or more functions can be inherited in one processing module. Exemplarily, an embodiment of the present application can divide the model training device into a receiving module and a processing module according to the function. The description of the possible technical solutions and beneficial effects executed by the above-mentioned divided functional modules can refer to the technical solutions provided by the above-mentioned first aspect or its corresponding possible implementation method, and will not be repeated here.

[0028] In a fifth aspect, an embodiment of the present application provides a computing cluster, which includes computing devices that can execute the data transmission method in the above aspects.

[0029] In a sixth aspect, an embodiment of the present application provides a computing device, which includes a processor and a memory, wherein the memory stores computer instructions, and the computer instructions are loaded and executed by the processor to enable the computing device to implement the data transmission method as described above.

[0030] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer program is stored. The computer program is loaded and executed by a processor to implement the data transmission method as described above.

[0031] In an eighth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a terminal reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the terminal to perform the data transmission method provided in various optional implementations of the above aspects.

[0032] For the specific description of the third to eighth aspects and their various implementations in the embodiments of the present application, reference can be made to the detailed description in the first aspect, the second aspect and their various implementations; and for the beneficial effects of the third to eighth aspects and their various implementations, reference can be made to the analysis of the beneficial effects in the first aspect, the second aspect and their various implementations, which will not be repeated here.

[0033] These and other aspects of the embodiments of the present application will be more clearly understood in the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of a computing cluster provided in an embodiment of the present application;

[0035] FIG2 is a schematic diagram of a computing cluster provided in an embodiment of the present application;

[0036] FIG3 is a schematic diagram of a computing cluster provided in an embodiment of the present application;

[0037] FIG4 is a schematic diagram of a computing cluster provided in an embodiment of the present application;

[0038] FIG5 is a schematic diagram of the structure of a data transmission system provided in an embodiment of the present application;

[0039] FIG6 is a flow chart of a data transmission method according to an embodiment of the present application;

[0040] FIG7 is a schematic diagram of a data transmission interaction process according to an embodiment of the present application;

[0041] FIG8 is a schematic structural diagram of a data transmission device provided in an embodiment of the present application;

[0042] FIG9 is a schematic structural diagram of a data transmission device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0044] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0045] First, the application scenarios of the embodiments of the present application are exemplarily introduced.

[0046] In the prior art, nodes transmit data from one node to another node in the system through network transmission, or write data from one node to a mobile hard disk and then copy the data on the mobile hard disk to another node.

[0047] However, the above method, especially for large data transmission, not only has high requirements on network transmission bandwidth, latency, etc., but also has the risk of data loss when relying on physical media or network media transmission and complex operations, resulting in low data transmission efficiency.

[0048] With the rapid development of Computer Express Link (CXL) technology, CXL memory pool technology has become increasingly mature, and data transmission capabilities are gradually improving. Based on the development of CXL memory pool technology, the embodiments of the present application provide a new method for inter-node data transmission.

[0049] In summary, an embodiment of the present application provides a data transmission method for determining the address information of the target memory space corresponding to the target data. The current owner of the target memory space is the first computing device. The first computing device sends a first application message to the management unit based on the address information, and the management unit responds to the first application message and changes the ownership of the target memory space from the first computing device to the second computing device. The first computing device sends attribute information to the second computing device, and the attribute information includes address information. At this time, the second computing device can access the target memory space based on the address information, obtain the target data from the target memory space, and complete the transmission of the target data from the first computing device to the second computing device. By modifying the ownership of the target memory space, the second computing device obtains the target data in the target memory space, and completes the target data transmission method, which not only protects the data from being attacked during the transmission process and ensures the security of the data, but also greatly improves the efficiency of data transmission.

[0050] For example, Figure 1 shows a schematic diagram of a computing cluster provided by an embodiment of the present application. The above-mentioned data transmission method can be applied to the computing cluster 100 shown in Figure 1 .

[0051] The computing cluster 100 shown in FIG1 is a computing cluster formed by interconnecting various computing devices and a CXL memory pool via a switch.

[0052] CXL is a development standard for high-speed, high-capacity CPU-to-device and CPU-to-memory connections. CXL technology not only enables high-speed transmission but also supports memory sharing and virtualization.

[0053] As shown in FIG. 1 , the computing cluster 100 may include a computing device 10 , a CXL switch 14 , a management device 15 , and a CXL memory pool 16 .

[0054] A CXL memory pool includes one or more CXL memory devices. The memory space formed by the one or more CXL memory devices constitutes the CXL memory space of the CXL memory pool. The CXL memory devices include a CXL controller and memory, which may include DDR or DRAM. In other embodiments, the memory may also include an SSD.

[0055] The number of computing devices 10 may be two or more, and each computing device is connected to a CXL switch and is connected to a corresponding CXL memory device in the memory pool through the CXL switch.

[0056] Management device 15, which may be a computing device running a fabric manager (FM), is configured to uniformly manage the CXL memory space in memory pool 16, allocate memory space in CXL memory pool 16 to different computing devices 10, and record the mapping between memory space in CXL memory pool 16 and different computing devices 10. Management device 15 can also modify the ownership of memory space in CXL memory pool 16.

[0057] It should be noted that, in some embodiments, the computing cluster 100 may not include the management device 15 , and the fabric manager (FM) may be deployed in a switch or a computing device.

[0058] The computing devices 10 can interact with each other via a traditional communication network. For example, a traditional communication network can be one based on remote direct memory access (RDMA) technology. RDMA technology can quickly move data from one system to a remote system's memory without impacting the operating system. A traditional communication network can also be one based on the Transmission Control Protocol / Internet Protocol (TCP / IP). The TCP / IP protocol is the most basic protocol that ensures the timeliness and integrity of network data and information.

[0059] The computing device 10 may be a server or a computer. Furthermore, the computing device 10 may install a local CXL space management program (LMD). The LMD may be a user-mode program deployed on the computing device 10, which is used for the computing device 10 where the LMD is located to apply for memory space in the CXL memory pool from the management device 15 and to locally manage the memory space applied for. The LMD may also provide memory access and usage to applications on the computing device 10, or convert the memory space into a non-uniform memory access (NUMA) space on the computing device 10 for management by the operating system kernel of the computing device 10.

[0060] The CXL switch 14 can provide more connection ports to connect more nodes. The fabric manager (FM) can be deployed on the CXL switch 14. The FM can be used to identify and manage the memory space of the CXL memory pool and divide the CXL memory pool 16. The FM can also interact with the LMD on each computing device 10 to allocate, reclaim, and manage the ownership of memory space in the CXL memory pool for each computing device 10 in the cluster. For example, a computing device 10 applies to the FM for memory space in the CXL memory pool through the LMD. The FM can change the ownership of the memory space in the CXL memory pool to the computing device 10. The computing device 10 can then access the memory space in the CXL memory pool belonging to the computing device 10 and store data in the memory space in the CXL memory pool.

[0061] As an independent module, FM can be deployed in a variety of ways. For example, Figure 2 shows a schematic diagram of a computing cluster provided by an embodiment of the present application. In computing cluster 100 shown in Figure 2, FM can be deployed as software in the baseboard management controller (BMC) of any computing device, or as a plug-in in each computing device.

[0062] The CXL memory pool 16 may be a pooled memory space formed in the cluster to meet the computing and storage requirements of each node.

[0063] Figure 3 shows a schematic diagram of a computing cluster provided by an embodiment of the present application. As shown in Figure 3, a computing cluster 100 is formed by a direct connection between a computing device 10 and a CXL memory pool via a port chip.

[0064] The above data transmission method can also be applied to the computing cluster 100 shown in FIG3 . As shown in FIG3 , the computing cluster 100 may include: a computing device 10 , a management device 15 , and a CXL memory pool 16 .

[0065] Memory pool 16 may include a CXL memory device, which includes a CXL controller and memory. The memory may be the same as that of the CXL memory device described in FIG1 , and constitutes the memory space of the CXL memory pool. The CXL controller is a multi-head controller, meaning it can connect to multiple computing devices.

[0066] The computing devices 10 are directly connected to each other. The concepts and functions of the computing devices 10, the management device 15 and the memory pool 16 are described in FIG1 and will not be repeated here.

[0067] FM, as an independent module, can be deployed in a variety of ways. Figure 4 shows a schematic diagram of a computing cluster provided by an embodiment of the present application. In the computing cluster 100 shown in Figure 4, FM can be deployed as software in the out-of-band management controller (BMC) of a computing device, or as a plug-in in each computing device.

[0068] In summary, the data transmission method is applicable to the above-mentioned computing cluster, which includes a CXL memory pool system and a traditional network system. The embodiment of the present application does not limit the cluster composition and protocol.

[0069] Next, the system architecture of the embodiment of the present application is exemplarily introduced.

[0070] For example, Figure 5 shows a schematic diagram of the data transmission system structure provided by an embodiment of the present application. The data transmission method provided by an embodiment of the present application can be applied to a data transmission system. The data transmission system can be a part of a computing cluster or the entire computing cluster.

[0071] As shown in FIG5 , the data transmission system may include a first computing device 30 , a second computing device 31 , and a CXL memory pool 16 . The computing devices 10 in the computing cluster may include the first computing device 30 and the second computing device 31 .

[0072] The first computing device 30 can be a server including a processor and memory. The memory can be used to store target data to be transmitted and related task or instruction information, and the processor can be used to process and execute related tasks or instructions. The LMD in the first computing device 30 is used to request memory space in the CXL memory pool 16 from the FM and locally manage the requested memory space. The first computing device 30 can write target data into a target memory space in the CXL memory pool 16 and obtain the address information of the target memory space in the CXL memory pool 16 through the LMD in the first computing device 30. The first computing device 30 can send a first request message to the FM based on the address information. Based on the first request message, the FM changes the ownership of the target memory space from the first computing device 30 to the second computing device 31. The first computing device 30 can send the address information of the target memory space in the CXL memory pool 16 to the second computing device 31 via a traditional communication network. The address information is used by the second computing device 31 to determine the target memory space in the CXL memory pool 16 and retrieve the target data from the target memory space.

[0073] The second computing device 31 may be a server including a processor and memory. The first computing device 30 and the second computing device 31 are different. The second computing device 31 is configured to receive target data. The LMD of the second computing device 31 is configured to locally manage the memory space in the CXL memory pool. The second computing device 31 receives first modification information sent by the FM, indicating that the target memory space belongs to the second computing device. Based on the address information, the second computing device 31 can access the target memory space in the CXL memory pool 16 and retrieve the target data from the target memory space in the CXL memory pool 16.

[0074] The FM can be a software program on the second computing device 31 that identifies and manages memory space in the CXL memory pool and partitions the CXL memory pool 16. The FM can also allocate, reclaim, and manage memory space ownership within the CXL memory pool for each computing device in the cluster through interaction with the LMDs on each computing device. As a software program, the FM can be deployed on any computing device in the cluster. The FM can change the ownership of target memory space in the CXL memory pool 16 from the first computing device 30 to the second computing device 31.

[0075] The CXL memory pool 16 may be a pooled memory space formed in the cluster to meet the storage requirements of each computing device and is used to store data to be transmitted, such as target data.

[0076] It should be noted that the system architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0077] For ease of understanding, the data transmission method provided in an embodiment of the present application is exemplarily introduced below with reference to the accompanying drawings. The data transmission method is applicable to the system shown in FIG5 .

[0078] FIG6 shows a flow chart of a data transmission method according to an embodiment of the present application. The data transmission method includes the following steps:

[0079] S101, determining address information of a target memory space corresponding to target data.

[0080] The address information may indicate a physical address of the target memory space in a CXL memory pool, where the CXL memory pool is used to provide a memory space for storing data to the computing cluster. The computing cluster includes a first computing device and a second computing device.

[0081] The target memory space is the memory space in the CXL memory pool that stores the target data. The target data is the data that the first computing device needs to send to the second computing device. The target memory space currently belongs to the first computing device. That is, when the target memory space belongs to the first computing device, the first computing device can use the target memory space to write data to the target memory device or read data from the target memory space.

[0082] In a possible implementation, the first computing device may obtain address information of the target memory space in the CXL memory pool through a local CXL space manager LMD.

[0083] It should be noted that the LMD can be used to locally manage the memory space applied for in the first computing device. Therefore, the address information of the target memory space in the CXL memory pool can be directly obtained from the LMD.

[0084] In a possible implementation, the first computing device may obtain address information of the target memory space in the CXL memory pool from the management device through the local CXL space manager LMD.

[0085] For example, since the management device records the mapping relationship between the memory space and address information in the CXL memory pool, if the address information of the target memory space in the CXL memory pool cannot be obtained in the LMD, a request can be sent to the management device through the LMD to obtain the address information of the target memory space in the CXL memory pool from the management device.

[0086] Exemplarily, the target memory space is a memory space in the CXL memory pool belonging to the first computing device, and the target data is stored in the target memory space. In this case, the first computing device sends a query request to the management device via the LMD. The query request may instruct the management device to obtain the address information of the target memory space from the management device. Based on the query request, the management device queries the address information of the target memory space storing the target data and returns it to the first computing device via the LMD.

[0087] In one possible implementation, prior to step S101, the data transmission method further includes: determining target data, determining whether the size of the target data is greater than a first threshold, and if the size of the target data is less than the first threshold, transmitting the target data via a traditional communication network. If the size of the target data is greater than the first threshold, steps S101-S104 may be executed, and the target data and the target memory space where the target data resides may be transmitted together by utilizing the CXL cluster and modifying the memory space of the management device.

[0088] It is understood that if the target data size is less than the first threshold, transmitting the target data via a traditional communication network does not place excessive pressure on the transmission medium and can significantly improve the flexibility and convenience of data transmission. If the target data size is greater than the first threshold, the target data is relatively large, and transmission efficiency via a traditional communication network is very low, with the risk of attack and data loss during transmission. However, switching the target memory for transmission can significantly improve data transmission efficiency and avoid the risk of attack and data loss during transmission.

[0089] In one possible implementation, if the target memory space is memory space in the CXL memory pool allocated by the management device to the first computing device based on the second application information, the address information of the target memory space in the CXL memory pool may be stored in the LMD, and the first computing device may directly query the LMD for the address information of the target memory space in the CXL memory pool.

[0090] In one possible implementation, before S101 , the data transmission method may further include: determining target data; sending second request information to a management device to request memory space for the target data; the management device allocating target memory space in the CXL memory pool for the target data based on storage requirements of the target data; receiving address information of the target memory space in the CXL memory pool sent by the management device, and writing the target data into the target memory space according to the address information.

[0091] The target data is the data to be transmitted by the first computing device; the second application information is used to indicate the storage requirement of the target data; and the CXL memory pool is used to provide the computing cluster with memory space for storing data.

[0092] Exemplarily, the data to be transmitted on the first computing device is determined as target data, and based on the size of the target data, a second request message is sent to the management device. The second request message indicates the storage requirements for the target data, which may include the size of the target data. Based on the second request message received from the first computing device, the management device allocates a target memory space with a storage capacity greater than the target data size to the first computing device, and sends the address information of the target memory space in the CXL memory pool to the first computing device. The first computing device then writes the target data into the target memory space based on the target memory address information.

[0093] For example, data M is data to be transmitted from the first computing device to the second computing device, and data M is determined to be the target data. The size of the target data M is determined to be 105GB. The first computing device sends a second application message to the FM based on the size of the target data, applying for a target memory space A in the CXL memory pool that is larger than the target data size for the first computing device. The storage capacity of the target memory space is 150GB. Based on the second application message, the management device updates the ownership of the target memory space A in the CXL memory pool to the first computing device, and sends the address information of the target memory space A in the CXL memory pool to the first computing device. The first computing device stores the target data M in the target space A based on the address information of the target memory space A in the CXL memory pool.

[0094] In one possible implementation, the target data has been stored in the target memory space before it is determined that it needs to be transmitted to the second computing device. When the first computing device needs to send the target data to the second computing device, the first computing device executes S101.

[0095] In one possible implementation, before S101, the target memory space is the memory space of the idle CXL memory pool in the first computing device. When the first computing device needs to send target data to the second computing device, the first computing device writes the target data into the target memory space and then executes S101.

[0096] Illustratively, before S101, the target memory space A is an idle memory space in the CXL memory pool of the first computing device. By comparing the storage capacity of the target memory space A with the size of the target data M to be transferred, it is determined that the storage capacity of the target memory space A is greater than the size of the target data M to be transferred. The first computing device may obtain address information of the target memory space A in the CXL memory pool from the management device, and the first device writes the target data M into the target memory space A based on the address information of the target memory space A in the CXL memory pool.

[0097] S102: The first computing device sends attribute information of the target data to the second computing device.

[0098] The attribute information includes address information of the target memory space in the CXL memory pool, and the address information is used by the second computing device to determine the target memory space in the CXL memory pool and obtain target data.

[0099] In a possible implementation, the attribute information sent by the first computing device to the second computing device may further include format information and size information of the target data, which may be used by the second computing device to load and use the target data.

[0100] In a possible implementation, the first computing device may send the attribute information of the target data to the second computing device via a network.

[0101] It is understandable that simple information transmission through traditional communication networks can greatly improve the flexibility and efficiency of message transmission. At the same time, it reduces the requirements for network bandwidth and significantly reduces the cost of network deployment.

[0102] S103: Send first application information to the management device based on the address information.

[0103] The first application information is used to instruct the management device to change the ownership of the target memory space from the first computing device to the second computing device.

[0104] Illustratively, based on the address information, the first computing device sends a first request message to the management to request to change the ownership of the target memory space in the CXL memory pool, so that the ownership of the target memory space is switched from the first computing device to the second computing device.

[0105] S104: The management device changes the ownership of the target memory space to the second computing device.

[0106] Illustratively, upon receiving the first application information, the management device changes ownership of the target memory space in the CXL memory pool from the first computing device to the second computing device.

[0107] For example, the management device responds to the first application information and modifies the mapping relationship between the target memory space and the first computing device recorded in the management device to a mapping relationship between the target memory space and the second computing device.

[0108] In a possible implementation, if the current ownership of the target memory space is changed from the first computing device to the second computing device, both the first computing device and the second computing device may receive the first modification information sent by the FM.

[0109] The first modification information is used to indicate that the current ownership of the target memory space has been changed from the first computing device to the second computing device.

[0110] It should be noted that in other embodiments, the first computing device may also send attribute information to the second computing device after receiving the first modification information, that is, execute S103 after executing S101, and execute S102 after executing S104. Of course, the first computing device may also first send attribute information to the second computing device, and then send the first modification information to the second computing device after receiving the first modification information. The order is not limited here.

[0111] S105: The second computing device obtains target data from the target memory space.

[0112] The target memory space currently belongs to the second computing device.

[0113] In a possible implementation, the target memory space is accessed based on the address information, and the target data is obtained from the target memory space.

[0114] Exemplarily, the target memory space belongs to the second computing device, so the second computing device can access the target memory space and obtain the target data in the target memory space.

[0115] In a possible implementation, the attribute information may include format information and size information of the target data, and the target data is acquired from the target memory space according to the format information and size information of the target data.

[0116] Exemplarily, if the current target memory space belongs to the second computing device, the second computing device can directly access the target memory space, and simultaneously obtain and use the target data based on the format information and size information of the target data included in the attribute information of the target data sent by the first computing device.

[0117] This method can transfer the target data by modifying the ownership of the target memory space in the CXL memory pool, migrating the target memory space and the target data within it from the first computing device to the second computing device. This fast and flexible transfer process reduces data copying and media transmission overhead, significantly improving data transfer efficiency. This is especially true for the rapid migration of large amounts of data, saving network deployment costs. It also significantly reduces the risk of data loss and attacks during transmission, increasing data transmission security.

[0118] Figure 7 shows a data transmission interaction flow chart provided by an embodiment of the present application. As shown in Figure 7, the first computing device 30 sends a second request message to the management device 15 (S701). The second request message indicates the storage requirements for the target data. The management device 15 is a computing device running FM. Based on the second request message, the management device 15 determines a target memory space in the CXL memory pool (S702). The storage capacity of the target memory space exceeds the storage capacity requirements for the target data. The target memory space currently belongs to the first computing device 30, and the first computing device 30 writes the target data to the target memory space in the CXL memory pool (S703). The first computing device 30 can send an address information request to the management device 15 via the LMD. After querying, the management device 15 returns the address information of the target memory space in the CXL memory pool to the first computing device 30. The first computing device 30 sends attribute information to the second computing device 31 via a traditional communication network (S704). The attribute information may include address information, format information, and size information of the target data.

[0119] Based on the address information, the first computing device 30 sends a first application message to the management device 15 (S705). The first application message is used to notify the management device 15 to change the ownership of the target memory space from the first computing device 30 to the second computing device 31. Based on the first application message, the management device 15 changes the ownership of the target memory space in the CXL memory pool to the second computing device 31 (S706). After the management device 15 completes the ownership change, it sends a first modification message to the second computing device 31 (S707). The first modification message is used to indicate that the target memory space belongs to the second computing device.

[0120] At this time, the target memory space in the CXL memory pool belongs to the second computing device 31 , and the second computing device 31 can access the target memory space in the CXL memory pool and obtain target data from the target memory space in the CXL memory pool ( S708 ).

[0121] In summary, the embodiments of the present application provide a data transmission method in which a first computing device sends a first application message to a management device, whereby the management device, based on the first application message, changes the ownership of the target memory space from the first computing device to the second computing device. The second computing device can then obtain the target data within the target memory space based on the attribute information, thereby transmitting the target data to the second computing device. The transmission process is fast and flexible, not only reducing media transmission overhead and network deployment costs, but also eliminating the need for target data to be transmitted along a network route, significantly reducing the risk of data loss and attacks during transmission, thereby improving data transmission efficiency.

[0122] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the method. It is understandable that in order to realize the above functions, the data transmission device includes at least one of the hardware structure and software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0123] The embodiment of the present application can divide the model training device into functional units according to the above method example. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0124] For example, Figure 8 shows a schematic diagram of the structure of a data transmission device provided in an embodiment of the present application. As shown in Figure 8, the data transmission device can be applied to a first computing device, and the data transmission device 800 includes:

[0125] Processing module 801 is used to determine the address information of the target memory space corresponding to the target data, the target memory space belongs to the first computing device, the target data is the data that the first computing device needs to send to the second computing device, and the second computing device is used to receive the target data.

[0126] The sending module 802 is used to send the attribute information of the target data to the second computing device, the attribute information includes address information; based on the address information, the first application information is sent to the management device, and the first application information is used to instruct the management device to change the ownership of the target memory space from the first computing device to the second computing device.

[0127] In a possible implementation, the processing module 801 is further configured to obtain address information of the target memory space in the CXL memory pool through the local CXL space manager LMD.

[0128] In one possible implementation, processing module 801 is further configured to determine target data, send second request information to a management device, and allocate target memory space in the CXL memory pool to the target data based on the target data's storage requirements. The CXL memory pool provides memory space for data storage to the computing cluster. The processing module 801 receives address information of the target memory space in the CXL memory pool from the management device and writes the target data into the target memory space according to the address information.

[0129] In a possible implementation, the sending module 802 is further configured to send the attribute information of the target data to the second computing device via a traditional communication network.

[0130] In one possible implementation, the data transmission device also includes a receiving module for receiving first modification information sent by the management device, where the first modification information is used to indicate that the target memory space belongs to the second computing device, so that the second computing device obtains the target data.

[0131] For example, Figure 9 shows a schematic diagram of the structure of a data transmission device provided in an embodiment of the present application. As shown in Figure 9, the data transmission device can be applied to a second computing device, and the data transmission device 900 includes:

[0132] Receiving module 901 is configured to receive attribute information sent by a first computing device, the attribute information including address information of target data; the address information indicates the location of the target memory space corresponding to the target data in the CXL memory pool. The target data is data that the first computing device needs to send to the second computing device. The CXL memory pool provides memory space for storing data to the computing cluster, and the first computing device is used to send the target data. It also receives first modification information, which indicates that the target memory space belongs to the second computing device.

[0133] The processing module 902 is configured to obtain target data based on the attribute information. The first modification information is sent by the management device after the management device switches the ownership of the target memory space from the first computing device to the second computing device based on the request of the first computing device.

[0134] In one possible implementation, the data transmission device 900 also includes a management module for managing the target memory space after the second computing device receives the first modification information and before obtaining the target data, so that the second computing device can access the target data in the target memory space.

[0135] In a possible implementation, the processing module 902 is further configured to obtain target data based on the attribute information, including, if the attribute information includes address information, accessing a target memory space based on the address information; and obtaining the target data from the target memory space.

[0136] In one possible implementation, the processing module 902 is further configured to obtain target data from the target memory space, including, if the attribute information includes format information and size information of the target data, obtaining the target data from the target memory space according to the format information and size information of the target data.

[0137] For a detailed description of the above optional methods, please refer to the aforementioned method embodiments, which will not be repeated here. In addition, the explanation and description of the beneficial effects of any of the above-mentioned data transmission devices 800 and 900 can refer to the corresponding method embodiments, which will not be repeated here.

[0138] The present application also provides a computer-readable storage medium storing at least one computer instruction, which is loaded and executed by a processor to implement the data transmission method described in each of the above embodiments. For explanations of the relevant contents and descriptions of the beneficial effects of any of the above-mentioned computer-readable storage media, please refer to the corresponding embodiments above and will not be repeated here.

[0139] The present application also provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to perform any of the methods described in the above embodiments. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available media may be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., SSD).

[0140] It should be noted that the above-mentioned device for storing computer instructions or computer programs provided in the embodiments of the present application, such as but not limited to, the above-mentioned memory, computer-readable storage medium and communication chip, etc., all have non-volatile (non-transitory). Those skilled in the art should be aware that in the above-mentioned one or more examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0141] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A data transmission method, characterized in that: The method is performed by a first computing device and includes: Determine address information of a target memory space corresponding to target data, the target memory space belongs to the first computing device, the target data is data that the first computing device needs to send to a second computing device, and the second computing device is used to receive the target data; Sending attribute information of the target data to the second computing device, the attribute information including the address information; Based on the address information, first application information is sent to the management device, where the first application information is used to instruct the management device to change the ownership of the target memory space from the first computing device to the second computing device, so that the second computing device obtains the target data.

2. The method according to claim 1, characterized in that The step of determining the address information of the target memory space corresponding to the target data includes: The address information of the target memory space in the CXL memory pool is obtained through the local CXL space manager LMD.

3. The method according to claim 1 or 2, characterized in that: Before determining the address information of the target memory space corresponding to the target data, the method further includes: determining the target data; Sending second application information to the management device; the second application information is used to indicate the storage requirement of the target data; the management device is used to allocate target memory space in the CXL memory pool for the target data according to the storage requirement of the target data; the CXL memory pool is used to provide memory space for storing data to the computing cluster; receiving address information of the target memory space in the CXL memory pool sent by the management device; According to the address information, the target data is written into the target memory space.

4. The method according to claim 1, characterized in that The sending the attribute information of the target data to the second computing device includes: The attribute information of the target data is sent to the second computing device via a traditional communication network.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: First modification information sent by the management device is received, where the first modification information is used to indicate that the target memory space belongs to the second computing device.

6. A data transmission method, characterized in that: The method is performed by a second computing device and includes: receiving attribute information sent by the first computing device, wherein the attribute information includes address information of target data; The address information is used to indicate the location of the target memory space corresponding to the target data in the CXL memory pool, the target data is data that the first computing device needs to send to the second computing device, and the CXL memory pool is used to provide a memory space for storing data to the computing cluster; receiving first modification information, where the first modification information is used to indicate that the target memory space belongs to the second computing device; Based on the attribute information, the target data is acquired, wherein the first modification information is sent by the management device after switching the ownership of the target memory space from the first computing device to the second computing device based on a request of the first computing device.

7. The method according to claim 6, characterized in that After receiving the first modification information and before acquiring the target data, the second computing device also manages the target memory space.

8. The method according to claim 6 or 7, characterized in that: The acquiring the target data based on the attribute information includes: If the attribute information includes the address information, accessing the target memory space based on the address information; The target data is acquired from the target memory space.

9. The method according to claim 8, characterized in that The acquiring the target data from the target memory space includes: If the attribute information includes format information and size information of the target data, the target data is acquired from the target memory space according to the format information and size information of the target data.

10. A computing device, characterized in that The computing device comprises: a processor and a memory for storing instructions executable by the processor; the processor is configured to execute the instructions, so that the computing device performs the data transmission method according to any one of claims 1 to 9.

Images