Data transmission method, apparatus, device, and medium

Abstract

The application discloses a data transmission method, device, equipment and medium, the method comprises the following steps: performing heartbeat detection on each computing node in real time through a single thread to obtain the network connection state of each computing node; and dynamically updating a first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node; transmitting data to each of the at least one first computing node according to the first list through multiple threads; when it is determined that there is at least one second computing node with failed data transmission in the process of transmitting data, reassigning a task amount corresponding to each second computing node according to the heartbeat time difference corresponding to each first computing node in the first list; and sending the re-assigned task amount corresponding to each second computing node to each first computing node in the first list. The embodiments provided in the application can improve the stability of the system.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a data transmission method, apparatus, device, and medium. Background Technology

[0002] Open MPI (Open Message Passing Interface) is an open-source, cross-platform implementation of the message passing interface (MPI). It is primarily used in high-performance computing environments, supporting multi-processor, multi-core, and distributed computing systems. Based on Open MPI, cluster systems of various sizes can be built for scenarios such as high-performance computing, big data analytics, and machine learning.

[0003] In Open MPI cluster systems, system stability is a crucial consideration, as it directly impacts the quality and efficiency of computational tasks. Common issues affecting Open MPI cluster system stability include various communication problems between nodes, and uneven task distribution within the cluster, where some nodes may be overloaded while others remain idle. Communication issues between nodes may include network congestion, latency, and packet loss. These problems can lead to unstable or even failed computational tasks; while load imbalance can degrade overall system performance, thus affecting system stability. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a data transmission method, apparatus, device and medium to solve the problems affecting the stability of existing Open MPI cluster systems.

[0005] In a first aspect, embodiments of the present invention provide a data transmission method for an Open MPI cluster system, the Open MPI cluster system including a master node and at least one compute node. The method is executed by the master node and includes: performing heartbeat detection on each compute node in real time using a separate thread to obtain the network connection status of each compute node; dynamically updating a first list based on at least one first compute node and the heartbeat time difference corresponding to each first compute node, wherein the first compute node is a compute node among the at least one compute node whose network connection status is normal; transmitting data to each of the at least one first compute node through multiple threads according to the first list; when it is determined during the data transmission process that there is at least one second compute node that has failed to transmit data, reallocating the task quantity corresponding to each second compute node according to the heartbeat time difference corresponding to each first compute node in the first list; and sending the reallocated task quantity corresponding to each second compute node to each first compute node in the first list.

[0006] Secondly, embodiments of the present invention also provide a data transmission device for an Open MPI cluster system, the Open MPI cluster system including a master node and at least one compute node, the device being configured in the master node, the device comprising: a heartbeat detection module, configured to perform heartbeat detection on each compute node in real time via a separate thread to obtain the network connection status of each compute node; and dynamically update a first list based on at least one first compute node and the heartbeat time difference corresponding to each first compute node, the first compute node being a compute node among the at least one compute node whose network connection status is normal; a data transmission module, configured to transmit data to each of the at least one first compute node via multiple threads according to the first list; a task allocation module, configured to, when it is determined during data transmission that there is at least one second compute node that has failed to transmit data, reallocate the amount of tasks corresponding to each second compute node according to the heartbeat time difference corresponding to each first compute node in the first list; the data transmission module is further configured to send the reallocated amount of tasks corresponding to each second compute node to each first compute node in the first list.

[0007] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the embodiments of the present invention.

[0008] Fifthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the embodiments of the present invention.

[0009] The beneficial effects of the technical solution provided by this invention are:

[0010] This invention provides a data transmission method, apparatus, device, and medium. The method is used in an Open MPI cluster system, which includes a master node and at least one compute node. The method is executed by the master node. It involves using a separate thread to perform heartbeat detection on each compute node in real time to obtain the network connection status of each compute node; dynamically updating a first list based on at least one first compute node and the heartbeat time difference corresponding to each first compute node; and transmitting data to each of the at least one first compute node using multiple threads according to the first list. If, during data transmission, at least one second compute node is found to have failed to transmit data, the workload corresponding to each second compute node is reallocated based on the heartbeat time difference corresponding to each first compute node in the first list. The reallocated workload is then sent to each first compute node in the first list. The embodiments provided by this invention achieve reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection technology and adjust the allocation of failed tasks according to the node's network communication capabilities, effectively improving the stability of the entire system and increasing the success rate of data transmission. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the data transmission method proposed in an embodiment of the present invention is shown;

[0013] Figure 2 A schematic flowchart of a data transmission method according to an embodiment of the present invention is shown;

[0014] Figure 3 A flowchart illustrating a data retransmission method for transmission failure according to another embodiment of the present invention is shown.

[0015] Figure 4 A schematic flowchart of a data retransmission method according to another embodiment of the present invention is shown;

[0016] Figure 5 A schematic diagram of the data transmission device provided in an embodiment of the present invention is shown;

[0017] Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] In Open MPI cluster systems, various factors can lead to node communication failures. For example, network congestion, node crashes, or insufficient underlying buffers in the Open MPI cluster system can all cause nodes to be unable to communicate. Currently, Open MPI cluster systems lack effective methods to directly and promptly determine whether nodes can communicate normally, and they also cannot assess the current network communication capabilities of nodes. This means that the existence of these problems can potentially lead to system crashes in Open MPI cluster systems.

[0021] This invention proposes a data transmission method that uses heartbeat detection technology to achieve reliable transmission between nodes in an Open MPI cluster system. It also adjusts the allocation of failed tasks based on the network communication capabilities of the nodes, thereby improving the stability of the entire Open MPI cluster system.

[0022] To better understand the inventive concept provided by this invention, the following is combined with... Figure 1-5 The image processing method proposed in this invention will be described in detail.

[0023] Please refer to Figure 1 , Figure 1 The diagram illustrates a data transmission method proposed in an embodiment of the present invention. This method is used in an Open MPI cluster system, which includes a master node and at least one compute node. The method can be implemented by a data transmission device configured in the master node.

[0024] The method includes:

[0025] Step S101: Perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically update the first list based on at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is the computing node whose network connection status is normal among at least one computing node.

[0026] In the above steps, the heartbeat time difference is determined based on the time when the master node sends the heartbeat packet and the time when it receives the heartbeat response data from each compute node.

[0027] The first list can be represented using a hash table. For example, the first list could be a heartbeat result hash table `heartBeatResult` maintained by the master node. In the heartbeat result hash table, the key value represents the process ID (rank) of the compute node in the Open MPI cluster, and the value represents the time difference between when the compute node starts sending a heartbeat and when it receives a heartbeat response.

[0028] In the steps described above, both sending and receiving heartbeat data can be asynchronous. For example, the MPI_Isend and MPI_Irecv functions can be used. Assume the master node uses the MPI_Send function to send data and the MPI_Wait function to check if the sending operation was successful. Heartbeat data can be in the form of heartbeat packets. When a compute node encounters an error, it can only wait for the next round of heartbeat checks. Only when the entire process of sending and receiving heartbeat data is error-free can it be added to the heartbeat result hash table.

[0029] Step S102: Data is transmitted to each of at least one first computing node via multiple threads according to the first list.

[0030] In the above steps, each thread is responsible for data transmission of one compute node. In the Open MPI cluster system, when the master node sends data, it is only necessary to determine whether the master node has sent the data correctly, without needing to care whether the compute nodes have received the data correctly, which effectively reduces communication complexity.

[0031] During the data reception process at the master node, the MPI_Irecv function, the MPI_Test function, and the heartbeat result hash table are used to jointly determine whether the asynchronous data reception from the compute node is normal.

[0032] Data is transmitted to at least one first computing node using multiple threads based on a first list. Multiple threads can be created for data sending and receiving. During data sending, the success of the data transmission operation is determined based on the API return value. If successful, the system asynchronously waits for the computing node to return a result; if it fails, the computing node is removed from the first list and recorded in the failed task list (failedTaskList). For asynchronous receiving, the MPI_Irecv function is used to asynchronously receive the result returned from the computing node. During the asynchronous result reception, a while loop uses MPI_Test to check if the asynchronous data reception is complete. If the result indicates that asynchronous reception is incomplete, the heartbeat result hash table `heartBeatResult` is queried to determine if the computing node exists in `heartBeatResult`. The heartbeat result hash table is also queried to check if the computing node's heartbeat is correct. If the computing node does not exist in `heartBeatResult`, the asynchronously received result corresponding to the computing node is discarded, and the failed task is recorded in the failed task list (failedTaskList).

[0033] Step S103: When it is determined during the data transmission process that there is at least one second computing node that has failed to transmit data, the task quantity corresponding to each second computing node is reallocated according to the heartbeat time difference between each first computing node in the first list.

[0034] Step S104: The reassigned task quantity, corresponding one-to-one with each second computing node, is sent to each first computing node in the first list.

[0035] In the above steps, after all data has been sent, all data that failed to be transmitted needs to be retransmitted. The failed tasks are redistributed to computing nodes with normal network connection status according to the proportion of heartbeat time difference and retransmitted.

[0036] The data transmission method provided in this invention is based on reliable transmission between nodes in an Open MPI cluster system using heartbeat detection, and reallocates failed transmission tasks according to the node's network communication capabilities, which can improve the success rate of data transmission and make reasonable use of cluster resources.

[0037] Please refer to Figure 2 , Figure 2 The diagram illustrates a data transmission method proposed in an embodiment of the present invention. This method is used in an Open MPI cluster system, which includes a master node and at least one compute node. The method can be implemented by a data transmission device configured in the master node.

[0038] The method includes:

[0039] Step S201: Perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically update the first list based on at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is the computing node whose network connection status is normal among at least one computing node.

[0040] In some embodiments, performing heartbeat detection on each compute node in real time via a separate thread may include sending heartbeat packets to each compute node asynchronously and cyclically via a separate thread, and receiving heartbeat response data from each compute node; determining the heartbeat time difference corresponding to each compute node based on the sending time of the heartbeat packets and the receiving time of the heartbeat response data from each compute node; determining that the network connection status of the compute node is normal when the heartbeat time difference corresponding to the compute node does not exceed a preset time threshold; or determining that the network connection status of the compute node is abnormal when the heartbeat time difference corresponding to the compute node exceeds a preset time threshold.

[0041] In a multi-threaded environment, a separate thread runs the heartbeat detection algorithm, and a heartbeat result hash table `heartBeatResult` is maintained on the master node. This hash table records the time spent on the entire heartbeat process for each compute node. If a heartbeat times out, it indicates a failure, and the value of that compute node is deleted from the hash table. This thread can also include a `while` loop to iteratively perform heartbeat detection on all compute nodes, and an external method to stop the loop is provided.

[0042] In some embodiments, when the first list is a heartbeat result hash table, dynamically updating the first list based on at least one first computing node and the heartbeat time difference corresponding to each first computing node may include creating a heartbeat result hash table; determining that the computing node is a first computing node when it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold; adding the process ID corresponding to the first computing node as a key value to the heartbeat result hash table; and adding the heartbeat time difference corresponding to the first computing node as a value value to the storage location corresponding to the process ID in the heartbeat result hash table.

[0043] Step S202: Data is transmitted to each of at least one first computing node via multiple threads according to the first list.

[0044] In some embodiments, when it is determined that the current process corresponding to multiple threads is included in a first list, data is sent to each first computing node; it is determined whether the operation of sending data to each first computing node is successful; if it is determined that the operation of sending data to the first computing node is successful, the system asynchronously waits to receive the result returned from the first computing node; and while asynchronously waiting to receive the result returned from the first computing node, it cyclically checks whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and a first function used to detect whether the non-blocking communication operation is completed; if the master node has completed the operation of asynchronously receiving data from the first computing node, the process ends; if the master node has not completed the operation of asynchronously receiving data from the first computing node, the process code corresponding to the first computing node is added to the failed task list; or, if it is determined that the operation of sending data to the first computing node has failed, the process code corresponding to the first computing node is added to the failed task list, and the first computing node is removed from the first list.

[0045] In some embodiments, detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node, based on a first list and a first function for detecting whether a non-blocking communication operation has been completed, may include: detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; if the master node has not completed the operation of asynchronously receiving data from the first computing node according to the first function, determining whether the process ID corresponding to the first computing node is included in the first list; if the process ID corresponding to the first computing node is included in the first list, returning to the method of detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; if the process ID corresponding to the first computing node is not included in the first list, discarding the asynchronously received data from the first computing node and adding the first computing node to the failed task list.

[0046] For the master node's receive operation, the `MPI_Irecv` function can be used to asynchronously receive the results returned from the first compute node. After the function returns the result, the `MPI_Test` function can be used in a `while` loop to check whether the asynchronously received data has been completely received. If the result is that the asynchronous reception is incomplete, the heartbeat result hash table `heartBeatResult` is queried to determine whether the compute node exists, thus checking whether the compute node's heartbeat is correct. If it does not exist, the asynchronously received result is discarded, and the failed task is recorded in the failed task list `failedTaskList`.

[0047] In step S203, when it is determined during the data transmission process that there is at least one second computing node that has failed to transmit data, the heartbeat time difference corresponding to each first computing node is converted into the same time unit.

[0048] Step S204: Calculate the ratio of the heartbeat time difference corresponding to each first calculation node.

[0049] Step S205: Normalize the comparison values ​​to obtain normalized ratios.

[0050] After the above steps, the first computing node and the normalized ratio corresponding to the first computing node are stored in the second list.

[0051] Step S206: reallocate the task quantity corresponding to each second computing node according to the normalized ratio.

[0052] Step S207: The reassigned task quantity, corresponding one-to-one with each second computing node, is sent to each first computing node in the first list.

[0053] In the above steps, the first list is a hash table `heartBeatResult` representing the heartbeat results. Heartbeat times are converted to the same unit, milliseconds, and the proportion of heartbeat times across all computation nodes is stored in a one-dimensional array. The reciprocal of this proportion is taken and normalized. The relationship between computation nodes and their proportions is then stored in the second list. This second list can be a hash table `powerList`.

[0054] Based on the failed task list (failedTaskList) and the heartbeat result hash table (heartBeatResult), the data corresponding to the failed tasks is redistributed to the nodes in the heartbeat result hash table according to the network capability value, and data transmission is restarted. After the data transmission is completed, the failed task list (failedTaskList) is checked again. If the failed task list is not empty, the process returns to continue retransmitting the failed tasks in the failed task list.

[0055] The following is combined with Figure 3This section further explains the process of reallocating and sending failed data. Step S301: Determine if the failed task list is empty. If not empty, proceed to step S302. If empty, end the process. Step S302: Traverse the heartbeat result hash table `heartBeatResult`, converting each heartbeat time difference to the same time unit. Step S303: Calculate the ratio of the heartbeat time differences corresponding to each compute node. Step S304: Take the reciprocal of each ratio to obtain a normalized ratio. Step S305: Store the compute node and its corresponding normalized ratio in the hash table `powerlist`. Step S306: Assign failed tasks to the first compute node included in the heartbeat result hash table according to the normalized ratio, and restart data transmission. This task reallocation method improves the success rate of data transmission and makes better use of cluster resources.

[0056] The data transmission method provided in this invention is based on reliable transmission between nodes in an Open MPI cluster system using heartbeat detection, and reallocates failed transmission tasks according to the node's network communication capabilities, which can improve the success rate of data transmission and make reasonable use of cluster resources.

[0057] Please refer to Figure 4 , Figure 4 The diagram illustrates a data transmission method proposed in an embodiment of the present invention. This method is used in an Open MPI cluster system, which includes a master node and at least one compute node. The method can be implemented by a data transmission device configured in the master node.

[0058] The method includes:

[0059] In step S401, a separate thread is used to send heartbeat packets to each computing node in an asynchronous manner and receive heartbeat response data from each computing node.

[0060] Step S402: Determine the heartbeat time difference corresponding to the computing node based on the sending time of the heartbeat packet and the receiving time of the heartbeat response data received from each computing node.

[0061] Step S403: Determine the network connection status of the computing node based on whether the heartbeat time difference corresponding to the computing node has timed out.

[0062] Step S404: When the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, the computing node is determined to be the first computing node, and the heartbeat time difference corresponding to the first computing node is added to the first list, wherein the first computing node is a computing node with a normal network connection status.

[0063] Step S405: When the heartbeat time difference corresponding to the computing node exceeds a preset time threshold, the network connection status of the computing node is determined to be abnormal, and the data corresponding to the computing node is deleted from the first list.

[0064] Step S406: Data is transmitted to each of at least one computing node through multiple threads.

[0065] Step S407: Determine whether the current process corresponding to the multiple threads is included in the first list.

[0066] Step S408: When it is determined that the current process corresponding to multiple threads is included in the first list, data is sent to each first computing node.

[0067] Step S409: Determine whether the operation of sending data to each first computing node was successful.

[0068] In step S410, if it is determined that the operation of sending data to the first computing node is successful, the master node asynchronously waits to receive the result returned from the first computing node.

[0069] Step S411: While asynchronously waiting to receive the result returned from the first computing node, the master node is cyclically checked to see if it has completed the operation of asynchronously receiving data from the first computing node, based on the first list and the first function used to detect whether the non-blocking communication operation has been completed.

[0070] Step S412: If the master node completes the operation of asynchronously receiving data from the first computing node, then the process ends.

[0071] Step S413: If the master node fails to complete the operation of asynchronously receiving data from the first computing node, the process code corresponding to the first computing node is added to the list of failed tasks.

[0072] Step S414: If it is determined that the operation of sending data to the first computing node has failed, the process code corresponding to the first computing node is added to the failed task list, and the first computing node is removed from the first list.

[0073] The data transmission method provided in this invention is based on reliable transmission between nodes in an Open MPI cluster system using heartbeat detection, and reallocates failed transmission tasks according to the node's network communication capabilities, which can improve the success rate of data transmission and make reasonable use of cluster resources.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] Alternatively, please refer to Figure 5 , Figure 5 A schematic diagram of the data transmission device provided in an embodiment of the present invention is shown. This device is used in an Open MPI cluster system, which includes a master node and at least one compute node. The device is characterized in that it is configured in the master node and includes:

[0076] The heartbeat detection module 501 is used to perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and to dynamically update the first list based on at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node whose network connection status is normal among at least one computing node.

[0077] Data transmission module 502 is used to transmit data to each of at least one first computing node through multiple threads according to a first list;

[0078] The task allocation module 503 is used to reallocate the task quantity corresponding to each second computing node according to the heartbeat time difference corresponding to each first computing node in the first list when it is determined that there is at least one second computing node that has failed to transmit data during the data transmission process.

[0079] The data transmission module 502 is also used to send the redistributed task quantity, which corresponds one-to-one with each second computing node, to each first computing node in the first list.

[0080] Optionally, the heartbeat detection module 501 is configured to send heartbeat packets to each computing node asynchronously and cyclically via a separate thread, and receive heartbeat response data from each computing node; determine the heartbeat time difference corresponding to the computing node based on the sending time of the heartbeat packets and the receiving time of the heartbeat response data from each computing node; determine that the network connection status of the computing node is normal when the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold; or determine that the network connection status of the computing node is abnormal when the heartbeat time difference corresponding to the computing node exceeds a preset time threshold.

[0081] Optionally, when the first list is a heartbeat result hash table, the heartbeat detection module 501 is also used to create the heartbeat result hash table; when it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, the computing node is determined to be the first computing node; the process number corresponding to the first computing node is added to the heartbeat result hash table as the key value; and the heartbeat time difference corresponding to the first computing node is added to the storage location corresponding to the process number in the heartbeat result hash table as the value value.

[0082] Optionally, the task allocation module 503 is further configured to: convert the heartbeat time difference corresponding to each first computing node into the same time unit; calculate the ratio of the heartbeat time difference corresponding to each first computing node; normalize the ratio to obtain a normalized ratio; and reallocate the task quantity corresponding to each second computing node according to the normalized ratio.

[0083] Optionally, the task allocation module 503 is further configured to: store the first computing node and the normalized ratio corresponding to the first computing node in a second list.

[0084] Optionally, the data transmission module 502 is further configured to: send data to each first computing node when it is determined that the current process corresponding to multiple threads is included in the first list; determine whether the operation of sending data to each first computing node is successful; if it is determined that the operation of sending data to the first computing node is successful, asynchronously wait to receive the result returned from the first computing node; and during the asynchronous waiting to receive the result returned from the first computing node, cyclically detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function used to detect whether the non-blocking communication operation is completed; if the master node has completed the operation of asynchronously receiving data from the first computing node, then the process ends; if the master node has not completed the operation of asynchronously receiving data from the first computing node, then the process code corresponding to the first computing node is added to the failed task list; or, if it is determined that the operation of sending data to the first computing node has failed, then the process code corresponding to the first computing node is added to the failed task list, and the first computing node is deleted from the first list.

[0085] Optionally, the data transmission module 502 is further configured to detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; if the master node has not completed the operation of asynchronously receiving data from the first computing node according to the first function, determine whether the process number corresponding to the first computing node is included in the first list; if the process number corresponding to the first computing node is included in the first list, return to the function indicating whether the master node has completed the operation of asynchronously receiving data from the first computing node; if the process number corresponding to the first computing node is not included in the first list, discard the asynchronously received data from the first computing node and add the first computing node to the failed task list.

[0086] The data transmission device provided in this embodiment of the invention enables reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection, and reallocates failed tasks according to the node's network communication capabilities, thereby improving the success rate of data transmission and making reasonable use of cluster resources.

[0087] The following is for reference. Figure 6 , Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. This electronic device may be a computer device. Figure 6 It does not impose limitations on the structure of electronic devices. For example... Figure 5 As shown, the electronic device includes at least a memory 601 and a processor 602. For example, the electronic device may also include... Figure 5 Showing more or fewer components (such as network interfaces, display devices, etc.).

[0088] In particular, according to the embodiments provided by the present invention, the above refers to the flowchart. Figure 1-3The described process can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the system of the present invention.

[0089] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of the methods and computer program products described according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0091] The units or modules described in the embodiments provided by this invention can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor may be described as including a heartbeat detection module, a data transmission module, and a task allocation module. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, a training sample acquisition module may also be described as "a module for performing heartbeat detection on each computing node using a separate thread."

[0092] In another aspect, embodiments of the present invention also provide a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the data transmission method described in the present invention.

[0093] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A data transmission method for an Open MPI cluster system, the Open MPI cluster system comprising a master node and at least one compute node, characterized in that, This method is executed by the master node and includes: The network connection status of each computing node is obtained by performing heartbeat detection on each computing node in real time through a separate thread; and the first list is dynamically updated according to at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is the computing node whose network connection status is normal among the at least one computing node. Data is transmitted to each of the at least one first computing node according to the first list via multiple threads; When it is determined during data transmission that there is at least one second computing node that has failed to transmit data, the task volume corresponding to each second computing node is reallocated according to the heartbeat time difference between each first computing node in the first list; The reassigned task quantity, corresponding one-to-one with each of the second computing nodes, is sent to each of the first computing nodes in the first list.

2. The method according to claim 1, characterized in that, The step of performing heartbeat detection on each computing node in real time via a separate thread includes: Heartbeat packets are sent asynchronously to each computing node in a separate thread, and heartbeat response data is received from each computing node. The heartbeat time difference corresponding to each computing node is determined based on the sending time of the heartbeat packet and the receiving time of the heartbeat response data from each computing node. If the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, the network connection status of the computing node is determined to be normal; or... When the heartbeat time difference corresponding to the computing node exceeds a preset time threshold, the network connection status of the computing node is determined to be abnormal.

3. The method according to claim 1, characterized in that, When the first list is a heartbeat result hash table, the first list is dynamically updated based on at least one first computing node and the heartbeat time difference corresponding to each first computing node, including: Create the hash table of the heartbeat results; When it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, the computing node is determined to be the first computing node; The process ID corresponding to the first computing node is used as the key value and added to the heartbeat result hash table; The heartbeat time difference corresponding to the first computing node is used as the value and added to the storage location corresponding to the process number in the heartbeat result hash table.

4. The method according to claim 1, characterized in that, Based on the heartbeat time difference corresponding to each of the first computing nodes in the first list, the task volume corresponding to each of the second computing nodes is reallocated, including: Convert the heartbeat time difference corresponding to each of the first computing nodes into the same time unit; Calculate the ratio of the heartbeat time difference corresponding to each of the first computing nodes; The ratio is normalized to obtain a normalized ratio. The workload is redistributed to each of the second computing nodes according to the normalized ratio.

5. The method according to claim 4, characterized in that, After normalizing the ratio to obtain a normalized ratio, the method includes: Store the first computing node and the normalized ratio corresponding to the first computing node in the second list.

6. The method according to claim 1, characterized in that, Data is transmitted to each of the at least one first computing node via multiple threads according to the first list, including: When it is determined that the current process corresponding to the plurality of threads is included in the first list, data is sent to each of the first computing nodes; Determine whether the operation of sending data to each of the first computing nodes was successful; If the operation of sending data to the first computing node is successful, the system asynchronously waits to receive the result returned from the first computing node; and while asynchronously waiting to receive the result returned from the first computing node, the system cyclically checks whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function used to detect whether the non-blocking communication operation has been completed. If the master node completes the operation of asynchronously receiving data from the first computing node, then the process ends; If the master node fails to complete the asynchronous data reception operation from the first computing node, then the process code corresponding to the first computing node is added to the failed task list; or, If it is determined that the operation of sending data to the first computing node has failed, the process code corresponding to the first computing node is added to the list of failed tasks, and the first computing node is removed from the first list.

7. The method according to claim 6, characterized in that, The step of cyclically detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function for detecting whether the non-blocking communication operation has been completed includes: The first function is used to detect whether the master node has completed the operation of asynchronously receiving data from the first computing node; If the master node fails to complete the operation of asynchronously receiving data from the first computing node according to the first function, determine whether the process number corresponding to the first computing node is included in the first list; If the process ID corresponding to the first computing node is included in the first list, return the function that detects whether the master node has completed the operation of asynchronously receiving data from the first computing node. If the process ID corresponding to the first computing node is not included in the first list, discard the data received asynchronously from the first computing node and add the first computing node to the list of failed tasks.

8. A data transmission apparatus for use in an Open MPI cluster system, the Open MPI cluster system comprising a master node and at least one compute node, characterized in that, The device is configured in the master node and includes: The heartbeat detection module is used to perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and to dynamically update the first list based on at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is the computing node among the at least one computing node whose network connection status is normal. The data transmission module is used to transmit data to each of the at least one first computing node according to the first list via multiple threads; The task allocation module is used to reallocate the task quantity corresponding to each second computing node based on the heartbeat time difference between each first computing node in the first list when it is determined that there is at least one second computing node that has failed to transmit data during the data transmission process. The data transmission module is further configured to send the redistributed task quantity, which corresponds one-to-one with each of the second computing nodes, to each of the first computing nodes in the first list.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-7.

Images