A data transmission method, apparatus, storage medium, and computer program product.

Abstract

This invention discloses a data transmission method, device, storage medium, and computer program product, relating to the field of computer technology. The method includes: determining a time interval within which a data transmission request needs to wait for processing by a shared memory device; determining the memory copy processing time for directly executing the data transmission request between different hosts; if the memory copy processing time is greater than the maximum value of the time interval and the shared memory device does not exhibit any transmission anomalies, then data transmission is performed between different hosts via the shared memory device; if the memory copy processing time is less than the minimum value of the time interval or the shared memory device exhibits a transmission anomaly, then data transmission is performed between different hosts via a server. This invention adds redundant transmission paths when a shared memory device exhibits a transmission anomaly, ensuring data reachability while maintaining low-latency transmission, thereby improving the reliability and overall efficiency of data transmission between hosts.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a data transmission method, apparatus, storage medium, and computer program product. Background Technology

[0002] In data centers, AI training, and cloud computing scenarios, the demand for high-frequency memory sharing between hosts is increasing. In related technologies, CXL (Compute Express Link) multi-head devices are used to replace traditional switches in small-scale, rack-mounted high-performance clusters to achieve memory sharing. This solution directly connects each host through multiple ports on the multi-head device, eliminating the hop count latency caused by switch routing. However, this solution still has reliability limitations: if a port becomes congested, the link is blocked, or the multi-head device itself fails, the shared memory path is severed, and the system can only passively wait for fault recovery, resulting in prolonged data transmission interruptions.

[0003] Therefore, how to improve the reliability and efficiency of data transmission between hosts is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a data transmission method, device, storage medium, and computer program product that improves the reliability and efficiency of data transmission between hosts.

[0005] To achieve the above objectives, the present invention provides a data transmission method applied to a data transmission system. The data transmission system includes multiple hosts, one or more shared memory devices, and a server. Each shared memory device includes multiple ports, with one port connected to one host. The server connects to the multiple hosts and one or more shared memory devices. The method includes: determining a time interval range within which a data transmission request needs to wait for processing by the shared memory device; determining the memory copy processing time for directly executing the data transmission request between different hosts; if the memory copy processing time is greater than the maximum value of the time interval range and the shared memory device does not exhibit any transmission anomalies, then data transmission is performed between different hosts through the shared memory device; if the memory copy processing time is less than the minimum value of the time interval range or the shared memory device exhibits any transmission anomalies, then data transmission is performed between different hosts through the server.

[0006] To achieve the above objectives, the present invention provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the data transmission method described above.

[0007] To achieve the above objectives, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data transmission method described above.

[0008] To achieve the above objectives, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the data transmission method described above.

[0009] The data transmission method provided by this invention achieves dynamic paths by comparing the time interval during which a data transmission request needs to wait for processing by a shared memory device with the memory copy processing time of the data transmission request, and by monitoring transmission anomalies of the shared memory device in real time. The time interval during which a data transmission request needs to wait for processing by the shared memory device can be understood as the tolerable time interval for the shared memory device to complete the data transmission request. The memory copy processing time of the data transmission request can be understood as the time required for the host to directly execute the data transmission request under ideal conditions. The comparison result reflects the real-time availability of the shared memory device path. When the memory copy processing time exceeds the tolerable time interval or when the shared memory device experiences a transmission anomaly, the shared memory device is bypassed, and data transmission between hosts is coordinated by the server. Therefore, this invention adds redundant transmission paths when the shared memory device experiences transmission anomalies, ensuring data reachability while maintaining low-latency transmission, thereby significantly improving the reliability and overall efficiency of data transmission between hosts. This invention also discloses an electronic device, a computer-readable storage medium, and a computer program product that can achieve the same technical effects.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0011] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a structural diagram of a data transmission system in related technologies.

[0013] Figure 2 This is a structural diagram of a data transmission system in related technologies.

[0014] Figure 3 This is a structural diagram illustrating a data transmission system according to an exemplary embodiment.

[0015] Figure 4 This is a schematic diagram of a transmission path according to an exemplary embodiment.

[0016] Figure 5 This is a flowchart illustrating a data transmission method according to an exemplary embodiment.

[0017] Figure 6 This is a flowchart illustrating data transfer via a shared device memory device according to an exemplary embodiment.

[0018] Figure 7 This is a flowchart illustrating a host copy mode during data transmission via a server according to an exemplary embodiment.

[0019] Figure 8 This is a flowchart illustrating a device copy mode during data transmission via a server according to an exemplary embodiment.

[0020] Figure 9 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0022] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0023] CXL (Compute Express Link) is a high-speed interconnect protocol based on PCIe (Peripheral Component Interconnect Express), supporting memory expansion and pooled architectures with shared memory. For example... Figure 1As shown, a shared memory pool using CXL memory expansion technology connects multiple host nodes with a CXL switch as the center. The switch, as the core interconnection device, supports multi-level expansion and can theoretically expand infinitely depending on the number of switch ports, such as large-scale memory resource pooling in data centers.

[0024] For example, in distributed training, models or data need to be split across multiple nodes. High-speed memory copying can quickly copy parameters from one node to other nodes, reducing waiting time during training. In cloud computing environments, virtual machine migration and cloning require the rapid copying of large amounts of memory data. Using CXL technology, multiple nodes can share memory resources, avoiding memory silos and reducing data copying overhead. Existing architectures can achieve memory sharing in large-scale computing clusters through switch-level expansion. However, for high-frequency memory access, such as low-latency, high-bandwidth scenarios like AI (Artificial Intelligence) training and real-time data processing, the switch hop count becomes a bottleneck for low latency and high bandwidth due to the need for switch routing.

[0025] The CXL multi-head device solution is applied to rack-level server solutions and is ideal for small-scale, high-performance clusters, such as shared-memory workloads in single-rack or inter-rack artificial intelligence and cloud computing, maximizing transfer speeds between servers. A key advantage of this architecture is that pooled devices significantly separate memory devices from processing units and enable a large pool for memory sharing, which not only significantly reduces costs but also avoids potential latency overhead.

[0026] like Figure 2 As shown, a multi-head device replaces the CXL switch for direct interconnection between two host compute nodes and a shared memory device, constructing an efficient and low-cost memory sharing solution. The multi-head solution utilizes the CXL shared memory device as the data transmission path for cross-node data exchange, supporting parallel read and write operations across multiple hosts. However, the reliability of this solution still has shortcomings: if a port becomes congested, the link is blocked, or the multi-head device itself fails, the shared memory path is severed, and the system can only passively wait for fault recovery, resulting in a prolonged data transmission interruption.

[0027] The data transmission method provided by this invention achieves dynamic pathing by comparing the time interval during which a data transmission request needs to wait for processing by the shared memory device with the memory copy processing time of the data transmission request, and by monitoring transmission anomalies of the shared memory device in real time. When the memory copy processing time exceeds the tolerable time interval or a transmission anomaly occurs in the shared memory device, the data transmission between hosts is bypassed, and the server coordinates the data transmission. Therefore, this invention adds redundant transmission paths when a transmission anomaly occurs in the shared memory device, ensuring data reachability while maintaining low-latency transmission, thereby significantly improving the reliability and overall efficiency of data transmission between hosts.

[0028] First, the data transmission system used in the data transmission method provided by this invention will be introduced, such as... Figure 3 As shown, the data transmission system includes multiple hosts, one or more shared memory devices, and a server. The shared memory devices include multiple ports, with one port connecting to one host. The server connects to multiple hosts and one or more shared memory devices.

[0029] Each host connects directly to a dedicated port of the shared memory device via its onboard CXL port in a point-to-point manner. The shared memory device integrates multiple physical ports, on-chip cache, task queues, sequence queues, and status registers. The number of ports can be exactly the same as the number of connected hosts, ensuring multi-head parallel access capability. The server maintains bidirectional communication with all hosts and the shared memory device simultaneously through an out-of-band control network separate from the data path. On the one hand, it continuously collects control information such as shared memory requests, read / write status, request frequency, and host identifiers submitted by each host. On the other hand, when it detects anomalies such as port busyness, link congestion, or device failure, it can immediately send redirection messages to the relevant hosts, triggering direct transmission paths between hosts.

[0030] Under normal operating conditions, when the first host needs to transfer data to the second host, it first calculates the potential waiting time range on the shared memory device based on the size of the data to be transferred and the request frequency in the most recent statistical period. Simultaneously, the first host also assesses the processing time required to directly copy data between hosts by bypassing the shared memory device. If the assessment shows that the inter-host copy time exceeds the upper limit of the aforementioned time range and the shared memory device is functioning correctly, the system maintains the default path: the first host writes the data to be transferred into the task queue of the shared memory device and writes the corresponding sequence identifier into the sequence queue; the shared memory device then retrieves the data from the task queue, obtains the sequence identifier from the sequence queue, loads the data into the second host's local DRAM (Dynamic Random Access Memory), and simultaneously writes the sequence identifier into the second host's sequence queue, completing a zero-hop, low-latency data transfer. During this process, the server only listens to the sequence queue in a bypass mode to maintain a global view.

[0031] When the above evaluation results indicate that the inter-host copying time is less than the lower limit of the time interval, or when any abnormality occurs in the shared memory device such as port busy, link blockage, or device failure, the server immediately intervenes: it sends a first message to the third host as the data requester, which carries the address of the fourth host and the sequence identifier of the requested data; at the same time, the server sends a second message to the fourth host as the data responder, which carries the address of the third host and the same sequence identifier. After receiving the second message, the fourth host can choose one of two sub-paths to perform data transmission: First, the fourth host enters kernel mode through the CPU (Central Processing Unit) control path, creates a memory transfer queue, writes the data into the task queue after confirming that there is enough free space in the queue, and notifies the local sending hardware to copy the data to the local cache and encapsulate it into a data packet, which is then sent to the third host via RDMA (Remote Direct Memory Access) protocol or TCP (Transmission Control Protocol). The third host receives and verifies the data and writes it into the designated memory. Second, the fourth host's local device, such as GPU (Graphics Processing Unit) or FPGA (Field-Programmable Gate Array), independently completes the queue creation, data transfer, data packet encapsulation and transmission. The third host's local device receives and completes the final write.

[0032] like Figure 4As shown, in host copy mode, the connection configuration of the host control section ensures the connection between the host and the device. Subsequently, the sending management module prepares the data to be sent, while the queue management module is responsible for creating and maintaining the memory transfer queue. The data encapsulation and parsing module writes data blocks into the task queue and, with the assistance of the control path, enters kernel mode to prepare for data transmission. The control management module of the sending hardware section receives new task request information, the sending engine module reads the task queue information, and is responsible for copying the data from the host memory to the hardware cache, encapsulating it into data packets, and sending it to the receiving end. The receiving engine module of the receiving end is responsible for verifying and parsing the data packets and writing the data to the designated memory. In device copy mode, after the device control section completes the connection configuration, the sending management module prepares the data, and the queue management module creates and maintains the queue. The data encapsulation and parsing module writes the data to the task queue and notifies the sending hardware of new task request information through the data path. The control management module of the sending hardware section receives the task request, the sending engine module reads the task queue information, copies the data from the memory of the local device (such as a GPU or FPGA) to the hardware cache, encapsulates it, and sends it to the receiving end. The receiving engine module of the receiving end completes the reception, verification, and parsing of the data packets and writes the data to the designated memory.

[0033] As can be seen, through the coordination of unified server scheduling and local host decision-making, the system dynamically switches between shared memory device paths and direct host transmission paths, ensuring extremely low latency under normal conditions and quickly falling back to reliable paths in abnormal or congested scenarios. This provides efficient and stable data sharing capabilities for high-concurrency, high-volume data applications such as artificial intelligence training, distributed database log replication, and real-time inference.

[0034] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] This invention provides a data transmission method, and the method is described in detail below with reference to its execution flow. See also... Figure 5 The flowchart illustrates a data transmission method according to an exemplary embodiment.

[0036] S101: Determine the time interval within which the data transfer request needs to wait for the shared memory device to process.

[0037] The time interval range refers to the time range required from when a data transfer request is sent to when the shared memory device completes processing it; it is usually an interval between a minimum and a maximum value.

[0038] The purpose of this step is to assess whether the shared memory device can handle upcoming data transfer requests in a timely manner. In this step, by monitoring the shared memory device's load, historical processing time, and current queue length, and combining this information with a pre-defined algorithm model, the time interval within which data transfer requests need to wait is calculated. For example, if there are a large number of requests for small data blocks in a short period, it may lead to longer processing times because frequent small data transfers increase system overhead. Therefore, this step can predict the processing time of data transfer requests in advance, providing a basis for selecting an appropriate data transfer path, thereby optimizing the overall system performance and reducing unnecessary waiting time.

[0039] As a feasible implementation method, determining the time interval range within which a data transmission request needs to wait for the shared memory device to process includes: determining the time interval range within which a data transmission request needs to wait for the shared memory device to process based on the data size of the data transmission request and the request frequency within a preset time period.

[0040] In practical implementation, the time interval is determined by monitoring the data volume of data transmission requests and the request frequency within a preset time period, combined with the processing capacity and current load of the shared memory device, and using a preset algorithm (such as a linear regression model or machine learning algorithm). For example, if the data volume is large and the request frequency is high, the time interval may be relatively long; conversely, it may be shorter. Therefore, this implementation method can more accurately predict the time required for the shared memory device to process data transmission requests, thereby allocating resources more rationally and improving the system's response speed and efficiency.

[0041] S102: Determine the memory copy processing time for direct data transfer requests between different hosts.

[0042] The memory copy processing time refers to the time required for the entire process, ideally, from the time the sending host prepares the data until the receiving host successfully receives the data and writes it into its memory.

[0043] In this step, the memory copy processing time for direct data transfer requests between different hosts is determined through simulation or historical data statistics. This can be calculated based on factors such as network bandwidth between hosts, data transfer protocols, and data size. For example, if the network bandwidth between hosts is 10Gbps and the data size is 1GB, the theoretical memory copy processing time can be estimated by dividing the data size by the bandwidth. Therefore, this step provides an accurate reference for subsequent decisions on whether to perform data transfer directly between hosts, thereby selecting the optimal data transfer path and improving system transmission efficiency.

[0044] As a feasible implementation method, determining the memory copy processing time for directly executing data transfer requests between different hosts includes: determining the memory copy processing time for directly executing data transfer requests between different hosts based on the data size of the data transfer request.

[0045] In practice, the memory copy processing time is calculated using a formula based on the size of the data transfer request, the network bandwidth between hosts, and the data transfer protocol. For example, assuming the network bandwidth between hosts is 10Gbps and the data size is 1GB, the memory copy processing time can be estimated by dividing the data size by the bandwidth, i.e., 1GB / 10Gbps = 0.1 seconds. This method allows for a quick and accurate estimation of the memory copy processing time for directly executing data transfer requests between hosts, providing reliable data support for subsequent path selection and thus optimizing the system's data transfer performance.

[0046] S103: If the memory copy processing time is greater than the maximum value of the time interval and there is no transmission abnormality in the shared memory device, then data transmission is performed between different hosts through the shared memory device.

[0047] Among them, transmission anomalies of shared memory devices include port busyness of shared memory devices, link blockage between the host and the shared memory device, and shared memory device failure. Port busyness means that the port of the shared memory device is currently processing a large number of requests and cannot respond to new requests in a timely manner; link blockage means that the communication link between the host and the shared memory device cannot transmit data normally due to network congestion or other reasons; device failure means that the shared memory device itself has a hardware or software failure and cannot work normally.

[0048] In this step, if the memory copy processing time is determined to be greater than the maximum value of the time interval and there are no transmission anomalies in the shared memory device, the data is transferred between different hosts through the shared memory device. That is, the data is written into the task queue of the shared memory device, and the shared memory device is responsible for transferring the data from one host to another.

[0049] It is evident that when shared memory devices can efficiently handle data transfer requests and there are no anomalies, data transfer through shared memory devices can fully leverage their advantages of high performance and low latency, improve overall system performance, and reduce the pressure of direct communication between hosts.

[0050] S104: If the memory copy processing time is less than the minimum value of the time interval or there is a transmission abnormality in the shared memory device, then data transmission will be performed between different hosts through the server.

[0051] In this step, if the memory copy processing time is less than the minimum of the time interval or if there is a transmission anomaly in the shared memory device, data transmission between different hosts will be performed via a server. The server acts as an intermediary node, receiving data requests, coordinating the data transmission process, and transferring data from the source host to the target host.

[0052] It is evident that when shared memory devices cannot efficiently process data transmission requests or are malfunctioning, transmitting data through a server can avoid transmission delays or failures caused by the limitations of shared memory devices, ensuring the reliability and efficiency of data transmission.

[0053] As one feasible implementation, data transmission between different hosts via a server includes: the server sending a first message to a third host; wherein the first host is a data requesting host, the first message includes the address of a fourth host and a sequence identifier of the requested data, and the fourth host is a data responding host; the server sending a second message to the fourth host; wherein the second message includes the address of the third host and a sequence identifier of the requested data; and the fourth host responding to the second message to transmit the requested data to the third host.

[0054] In practice, after receiving a data request, the server generates a first message and a second message based on the request information. The first message is sent to the requesting host, informing it of the address of the responding host and the sequence identifier of the requested data; the second message is sent to the responding host, informing it of the address of the requesting host and the sequence identifier of the requested data. The responding host then transmits the requested data to the requesting host based on the information in the second message. If the responding host's response is inconsistent or the data request fails, the transmission process via the shared memory device proceeds according to step S103. It is evident that through the coordinating role of the server, a data transmission path between hosts can be established quickly and accurately, improving the efficiency and reliability of data transmission and avoiding problems such as address resolution errors or communication failures that may occur during direct communication between hosts.

[0055] As one possible implementation, the fourth host responds to the second message to transmit the request data to the third host, including: the fourth host enters kernel mode through the control path and transmits the request data to the third host; after receiving the request data, the third host writes the request data into a designated memory.

[0056] In practical implementation, if data transfer is required between the third and fourth hosts, a host copy mode is entered. In this mode, after receiving the second message, the fourth host enters kernel mode through the control path, invokes the data transfer function provided by the operating system kernel, and transmits the requested data to the third host. Upon receiving the data, the third host writes it to the specified memory address using the appropriate memory management mechanism. Therefore, by entering kernel mode for data transfer, the operating system's kernel resources can be fully utilized, improving the efficiency and stability of data transfer, while ensuring that data is accurately written to the specified memory location, avoiding data loss or erroneous writes.

[0057] As one possible implementation, the fourth host responds to the second message to transmit the request data to the third host, including: the local device of the fourth host responds to the second message to transmit the request data to the local device of the third host.

[0058] In this context, "local device" refers to the computing devices within the host machine, such as GPUs and FPGAs. In practical implementation, if data transfer is required between the local devices of the third and fourth hosts, a device copy mode is entered. In this mode, after receiving the second message, the local device of the fourth host transmits the requested data to the local device of the third host via a direct device communication mechanism. This method bypasses the host's CPU and operating system kernel, enabling direct data transfer between devices. Therefore, direct data transfer between local devices reduces the involvement of the CPU and operating system kernel, lowers data transfer latency, and improves data transfer efficiency, making it particularly suitable for applications with high real-time and performance requirements.

[0059] The data transmission method provided in this invention achieves dynamic pathing by comparing the time interval during which a data transmission request needs to wait for processing by the shared memory device with the memory copy processing time of the data transmission request, and by monitoring transmission anomalies of the shared memory device in real time. The time interval during which a data transmission request needs to wait for processing by the shared memory device can be understood as the tolerable time interval for the shared memory device to complete the data transmission request. The memory copy processing time of the data transmission request can be understood as the time required for the host to directly execute the data transmission request under ideal conditions. The comparison result reflects the real-time availability of the shared memory device path. When the memory copy processing time exceeds the tolerable time interval or when the shared memory device experiences a transmission anomaly, the shared memory device is bypassed, and data transmission between hosts is coordinated by the server. Therefore, this invention adds a redundant transmission path when the shared memory device experiences a transmission anomaly, ensuring data reachability while maintaining low-latency transmission, thereby significantly improving the reliability and overall efficiency of data transmission between hosts.

[0060] This embodiment describes the process of data transfer via a shared device memory device, such as... Figure 6 As shown.

[0061] S201: The first host generates the data to be sent in the data transmission request, writes the data into the task queue of the shared memory device, and writes the sequence identifier corresponding to the data into the sequence queue of the shared memory device; wherein, the first host is the data sending host.

[0062] In this step, after receiving a data transmission request, the first host generates the data to be sent and writes it to the task queue through the interface provided by the shared memory device. Simultaneously, it generates a sequence identifier corresponding to the data and writes it to the sequence queue. The sequence identifier is used to identify the order of data blocks during data transmission, ensuring data integrity and consistency. By writing the data and sequence identifier to the task queue and sequence queue respectively, the data transmission process can be effectively organized, ensuring the order and integrity of data during transmission, and providing a basis for subsequent data verification and recovery.

[0063] S202: The server retrieves the sequence identifier corresponding to the data from the sequence queue.

[0064] In this step, the server periodically or on demand reads sequence identifiers from the sequence queue through the communication interface with the shared memory device. The server can use a polling mechanism or an interrupt mechanism to obtain sequence identifiers, ensuring timely access to the latest data transmission status. By obtaining sequence identifiers, the server can monitor the progress and status of data transmission in real time, providing support for subsequent data scheduling and anomaly handling, thereby improving the reliability and stability of the system.

[0065] S203: The shared memory device retrieves data from the task queue and retrieves the sequence identifier corresponding to the data from the sequence queue.

[0066] In this step, the shared memory device reads data from the task queue and the corresponding sequence identifier from the sequence queue through its internal queue management module. The shared memory device can process data and sequence identifiers according to queue priority or the First-In-First-Out (FIFO) principle. By retrieving data and sequence identifiers from the task queue and sequence queue respectively, the shared memory device can efficiently organize the data transmission process, ensuring data order and integrity, while improving system throughput and response speed.

[0067] S204: The shared memory device loads data into the memory of the second host and writes the sequence identifier into the sequence queue of the second host; wherein the second host is the data receiving host.

[0068] In this step, the shared memory device loads data into the second host's memory, such as DRAM, through a communication interface with the second host. Simultaneously, a sequence identifier is written into the second host's sequence queue so that the second host can perform data verification and processing. By loading data and sequence identifiers into the second host, data integrity and consistency are ensured, while providing the second host with necessary information for further data processing and verification, thus improving system reliability and stability.

[0069] This embodiment describes the process in host copy mode when data is transmitted through a server, such as... Figure 7 As shown.

[0070] S301: The server sends a first message to a third host; wherein the third host is the data requesting host, the first message includes the address of the fourth host and the sequence identifier of the requested data, and the fourth host is the data responding host.

[0071] In this step, the server sends a first message to the third host via the network communication interface. The first message contains the address of the fourth host and a sequence identifier for the requested data, so that the third host knows the source and destination of the data.

[0072] S302: The server sends a second message to the fourth host; wherein the second message includes the address of the third host and the sequence identifier of the requested data.

[0073] In this step, the server sends a second message to the fourth host via the network communication interface. This second message contains the address of the third host and a sequence identifier for the requested data, so that the fourth host knows the target host and the identifier of the data.

[0074] S303: The fourth host locates the local memory item based on the sequence identifier in the second message and prepares the requested data.

[0075] In this step, after receiving the second message, the fourth host searches for the corresponding memory item in its local memory based on the sequence identifier in the message and prepares the requested data. The fourth host can perform this operation through a memory management unit (MMU) or a dedicated memory lookup module.

[0076] S304: The fourth host enters kernel mode through the control path and creates a memory transfer queue.

[0077] In this step, the fourth host enters kernel mode via a control path (such as a system call or interrupt mechanism) and uses the functions provided by the operating system kernel to create a memory transfer queue. The kernel is responsible for notifying and maintaining the queue, ensuring the order and integrity of data transmission. By entering kernel mode to create and maintain the queue, the kernel resources of the operating system can be fully utilized, improving the efficiency and stability of data transmission while ensuring the order and integrity of data transmission.

[0078] S305: If the free space in the memory transfer queue meets the transfer requirements of the requested data, the fourth host will write the requested data into the task queue.

[0079] In this step, the fourth host checks the free space in the memory transfer queue to ensure there is enough space to accommodate the requested data. Then, it writes the requested data into the task queue, awaiting subsequent transfer operations. By checking for free space and writing the requested data accordingly, queue overflow can be avoided, ensuring smooth data transmission and improving system reliability.

[0080] S306: The fourth host notification sending hardware copies the request data in the task queue to the local cache, and encapsulates the request data in the local cache into a data packet and sends it to the third host.

[0081] In this step, the fourth host notifies the sending hardware of new task request information, task queue index, etc. The sending hardware reads task queue information, including opcode, local address, remote address, data length and access key, etc. The sending hardware copies the data to its own hardware cache, and then encapsulates the data packet according to the protocol and sends it to the third host.

[0082] S307: The third host receives and parses the data packet to obtain the request data. After verifying the request data according to the sequence identifier of the request data, it writes the request data into the specified memory.

[0083] In this step, the third host receives data packets through the network interface and parses them to obtain the requested data. The third host verifies the sequence number of the requested data; if the verification passes, the requested data is written to the designated memory.

[0084] This embodiment describes the process in device copy mode when transmitting data through a server, such as... Figure 8 As shown.

[0085] S401: The server sends a first message to a third host; wherein the first host is the data requesting host, the first message includes the address of the fourth host and the sequence identifier of the requested data, and the fourth host is the data responding host.

[0086] In this step, the server sends a first message to the third host via the network communication interface. The first message contains the address of the fourth host and a sequence identifier for the requested data, so that the third host knows the source and destination of the data.

[0087] S402: The server sends a second message to the fourth host; wherein the second message includes the address of the third host and the sequence identifier of the requested data.

[0088] In this step, the server sends a second message to the fourth host via the network communication interface. This second message contains the address of the third host and a sequence identifier for the requested data, so that the fourth host knows the target host and the identifier of the data.

[0089] S403: The local device of the fourth host creates a memory transfer queue.

[0090] In this step, the local device of the fourth host (such as a GPU or FPGA) creates the queue required for memory transfer and is responsible for operations such as notifying and maintaining the queue.

[0091] S404: If the free space in the memory transfer queue meets the transfer requirements of the requested data, the local device of the fourth host will write the requested data into the task queue.

[0092] In this step, the fourth host's local device checks the free space in the memory transfer queue to ensure there is enough space to accommodate the requested data. Then, it writes the requested data into the task queue, awaiting subsequent transfer operations. By checking for free space and writing the requested data accordingly, queue overflow can be avoided, ensuring smooth data transmission and improving system reliability.

[0093] S405: The local device of the fourth host notifies the sending hardware to copy the request data in the task queue to the local cache, and encapsulate the request data in the local cache into a data packet and send it to the local device of the third host.

[0094] In this step, the local device of the fourth host notifies the sending hardware of new task request information, task queue index, etc., the sending hardware reads task queue information, including opcode, local address, remote address, data length and access key, etc., the sending hardware copies the data to its own hardware cache, and then encapsulates the data packet according to the protocol and sends it to the local device of the third host.

[0095] S406: The local device of the third host receives and parses the data packet to obtain the request data. After verifying the request data according to the sequence identifier of the request data, it writes the request data into the specified memory.

[0096] In this step, the third host's local device receives data packets through the network interface and parses them to obtain the requested data. The third host's local device verifies the request data based on its sequence identifier; if the verification passes, the request data is written to the designated memory.

[0097] Based on the above embodiments, as a preferred implementation, it further includes: assigning priorities to data transmission requests; dynamically adjusting data transmission paths according to priorities; adjusting queue scheduling strategies according to priorities on shared memory device paths; optimizing resource allocation according to priorities on inter-host transmission paths; and monitoring system status in real time and dynamically adjusting priorities.

[0098] In implementation, each data transmission request is assigned a priority based on its type, urgency, and impact on system performance. Priorities are categorized into high, medium, and low levels. For example, data transmission requests with high real-time requirements (such as parameter synchronization in AI training) are assigned high priority, while data backup requests with low latency requirements are assigned low priority. The transmission path is dynamically selected based on the request's priority. For high-priority requests, low-latency shared memory device paths are prioritized; for low-priority requests, inter-host transmission paths coordinated by the server can be chosen to balance system load. Therefore, by dynamically adjusting the data transmission path, the optimal path can be flexibly selected based on priority and system status, improving system flexibility and adaptability, and further optimizing data transmission efficiency.

[0099] Furthermore, shared memory devices adjust the scheduling strategy of their internal queues based on the priority of data transfer requests. High-priority requests can be processed first, while low-priority requests may be temporarily delayed. Additionally, when data transfer requests travel via direct inter-host paths, the system optimizes resource allocation based on request priority. For example, high-priority requests may be allocated more bandwidth and CPU resources to ensure fast transmission.

[0100] Additionally, monitoring modules can be deployed on servers and shared memory devices to collect system status information in real time and dynamically adjust request priorities based on preset rules. For example, if a port on a shared memory device fails, the system can lower the priority of requests that depend on that port and reallocate transmission paths.

[0101] As can be seen, by introducing a priority scheduling mechanism, combined with dynamic path selection and resource allocation optimization, the data transmission efficiency and reliability of the CXL multi-head device memory-shared architecture in high-concurrency scenarios have been further improved. By rationally allocating priorities and dynamically adjusting transmission paths, the system can more flexibly respond to different data transmission needs, significantly improving overall performance.

[0102] The following describes the application examples provided by this invention. When training and processing very large models and GPU memory is insufficient, it is necessary to divide the model or data into multiple GPUs or multiple nodes. Some parameters or optimizer states (such as momentum, variance, learning rate, etc.) can be offloaded to a shared multi-head memory device, which can improve the effect in the following aspects.

[0103] (1) Parameter synchronization: Model parameters need to be collected from the memory of different devices. For example, in a scenario where four GPUs are trained on different hosts, the weight parameters of the model on each GPU are extended and stored on a shared device. At the start of synchronization, a master device (such as GPU 0) or a dedicated synchronization management module will initiate the synchronization operation. It will send requests to other devices, asking them to send their respective parameter values. By using a shared multi-head device, these parameter values ​​will be read from the memory of the shared multi-head device, without needing to be sent to the master device or synchronization management module over the network, which can reduce the latency caused by network transmission. If there is device congestion or failure during the transmission process, the server will report that other devices (such as GPU 1) have performed the same parameter synchronization process. At this time, the master device will send requests to other devices to reduce the additional latency caused by congestion or failure.

[0104] (2) Parameter Aggregation: For a large model containing millions of parameters, multi-device collaboration is required, and memory overflow should be avoided by storing the parameters on the host or other devices. This often involves a large number of parameter aggregation processes, with common aggregation methods including averaging and weighted summation. Taking averaging as an example, suppose the model has a parameter w, which is updated to w1, w2, w3, and w4 on four GPUs. In memory, these four values ​​are added together and then divided by 4 to obtain the new parameter value w' = (w1 + w2 + w3 + w4) / 4. Using a multi-device shared memory approach can greatly improve parameter aggregation efficiency. If GPU0 cannot aggregate parameters w3 and w4 due to congestion or failure, but these parameters are successfully aggregated on other GPUs, parameter aggregation can be performed through host memory copying or host local device memory copying. Of course, for larger models, large blocks of data will be aggregated in batches, not just one or two parameters as in the example above. Therefore, this approach can bring significant benefits.

[0105] In distributed databases within a cloud environment, to optimize performance, handle large-scale data, and achieve high availability and fault tolerance, there are situations where logs and other data are stored in extended memory and data is replicated across multiple nodes. For example, in master-slave replication mode, master and slave nodes can synchronize logs through multi-head shared memory. That is, the master node records operations in the shared memory log, and the slave nodes apply the logs sequentially to update their local copies. If a slave node with an anomaly fails to update its local copy in time, it can update its local log through master-slave memory copy mode.

[0106] In distributed inference scenarios, models need to respond quickly to user requests, especially in high-concurrency environments (such as real-time image recognition and speech recognition). In this case, model parameters and input data (or preprocessed data) are loaded onto multiple inference nodes. Each node loads the same model parameters but handles different user requests. Employing multi-device memory sharing and inter-host memory copying can improve overall efficiency.

[0107] Let the total latency of a shared memory operation task be T = n + t, where n is the time consumed by normal shared memory read / write operations, and t is the total time consumed by shared memory read / write operations when congestion or failure occurs. In the host memory copy method, the server processing latency is s, the maximum host response time is r, and the adjustment coefficient of r is a, where a is greater than 0 and less than or equal to 1. When a equals 1, it is host transmission mode, and the latency improvement effect is s + rt. When a is less than 1, it is device transmission mode, and the latency improvement effect varies depending on the local device type (such as FPGA, GPU, etc.) or the transmission protocol type (RDMA, TCP, etc.), and the latency improvement effect is s + r × at. For link or device failures, the recovery time is uncontrollable, and it is generally understood that the solution will be more effective in the event of failure.

[0108] Embodiments of the present invention also provide an electronic device, Figure 9 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 9 As shown, the electronic device includes: a communication interface 1, capable of exchanging information with other devices such as network devices; and a processor 2, connected to the communication interface 1 to enable information exchange with other devices and, when running a computer program, to execute the data transmission methods provided by one or more of the above-mentioned technical solutions. The computer program is stored on a memory 3.

[0109] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 9The general will label all buses as Bus System 4.

[0110] In this embodiment of the invention, memory 3 is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0111] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0112] The methods disclosed in the above embodiments of the present invention can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0113] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of the present invention. For the sake of brevity, these will not be described in detail here.

[0114] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data transmission method embodiments when it is run.

[0115] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0116] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by processor 2, implements the steps in any of the above-described data transmission method embodiments.

[0117] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by processor 2, implements the steps in any of the above-described data transmission method embodiments.

[0118] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0119] The present invention has provided a detailed description of a data transmission system, method, apparatus, device, medium, and product. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are merely illustrative of the methods and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A data transmission method, characterized in that, The system is applied to a data transmission system, which includes multiple hosts, one or more shared memory devices, and a server. The shared memory devices include multiple ports, with one port connected to one host. The server is connected to multiple hosts and one or more shared memory devices. The method includes: Determine the time interval within which the data transfer request needs to wait for the shared memory device to process; Determine the memory copy processing time for directly executing the data transfer request between different hosts; If the memory copy processing time is greater than the maximum value of the time interval and there is no transmission abnormality in the shared memory device, then data transmission is performed between different hosts through the shared memory device. If the memory copy processing time is less than the minimum value of the time interval or if the shared memory device has a transmission abnormality, then data transmission is performed between different hosts through the server. The time interval within which a data transfer request needs to wait for the shared memory device to process it includes: The time interval during which the data transmission request needs to wait for the shared memory device to process is determined based on the data size of the data transmission request and the request frequency within a preset time period.

2. The data transmission method according to claim 1, characterized in that, Determine the memory copy processing time for directly executing the data transfer request between different hosts, including: The memory copy processing time for directly executing the data transfer request between different hosts is determined based on the data size of the data transfer request.

3. The data transmission method according to claim 1, characterized in that, The transmission anomaly of the shared memory device includes any one or a combination of any of the following: the port of the shared memory device is busy, the link between the host and the shared memory device is blocked, or the shared memory device is faulty.

4. The data transmission method according to claim 1, characterized in that, Data transfer between different hosts via the shared memory device includes: The first host generates the data to be sent in the data transmission request, writes the data into the task queue of the shared memory device, and writes the sequence identifier corresponding to the data into the sequence queue of the shared memory device; wherein, the first host is the data sending host; The shared memory device obtains the data from the task queue and obtains the sequence identifier corresponding to the data from the sequence queue; The shared memory device loads the data into the memory of the second host and writes the sequence identifier into the sequence queue of the second host; wherein the second host is a data receiving host.

5. The data transmission method according to claim 4, characterized in that, After the first host generates the data to be sent in the data transmission request, writes the data into the task queue of the shared memory device, and writes the sequence identifier corresponding to the data into the sequence queue of the shared memory device, the process further includes: The server obtains the sequence identifier corresponding to the data from the sequence queue.

6. The data transmission method according to claim 1, characterized in that, Data transmission between different hosts via the server includes: The server sends a first message to a third host; wherein the third host is a data requesting host, the first message includes the address of a fourth host and a sequence identifier of the requested data, and the fourth host is a data responding host; The server sends a second message to the fourth host; wherein the second message includes the address of the third host and the sequence identifier of the requested data; The fourth host responds to the second message in order to transmit the request data to the third host.

7. The data transmission method according to claim 6, characterized in that, The fourth host responds to the second message to transmit the request data to the third host, including: The fourth host enters kernel mode through the control path and transmits the request data to the third host; After receiving the request data, the third host writes the request data into the designated memory.

8. The data transmission method according to claim 7, characterized in that, The fourth host enters kernel mode through a control path and transmits the request data to the third host, including: The fourth host searches for the local memory item based on the sequence identifier in the second message and prepares the requested data. Enter kernel mode through the control path and create a memory transfer queue; If the free space of the memory transfer queue meets the transfer requirements of the requested data, then the requested data is written into the task queue. The notification sending hardware copies the request data in the task queue to the local cache, and encapsulates the request data in the local cache into a data packet and sends it to the third host; Accordingly, after receiving the request data, the third host writes the request data into a designated memory, including: The third host receives and parses the data packet to obtain the request data. After verifying the request data according to the sequence identifier of the request data, it writes the request data into the designated memory.

9. The data transmission method according to claim 6, characterized in that, The fourth host responds to the second message to transmit the request data to the third host, including: The local device of the fourth host responds to the second message in order to transmit the request data to the local device of the third host.

10. The data transmission method according to claim 9, characterized in that, The local device of the fourth host responds to the second message to transmit the request data to the local device of the third host, including: The fourth host's local device creates a memory transfer queue; If the free space of the memory transfer queue meets the transfer requirements of the requested data, then the requested data is written into the task queue. The notification sending hardware copies the request data in the task queue to the local cache, and encapsulates the request data in the local cache into a data packet and sends it to the local device of the third host.

11. The data transmission method according to claim 10, characterized in that, After the local device of the fourth host encapsulates the request data in the local cache into a data packet and sends it to the local device of the third host, the transmission hardware further includes: The local device of the third host receives and parses the data packet to obtain the request data. After verifying the request data according to the sequence identifier of the request data, it writes the request data into the designated memory.

12. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to perform the steps of the data transmission method as described in any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the data transmission method as described in any one of claims 1 to 11.

14. A computer program product, characterized in that, It includes a computer program, which, when executed, implements the steps of the data transmission method as described in any one of claims 1 to 11.

Images