Data transmission method and device, equipment and medium

Abstract

The invention discloses a data transmission method and device, equipment and a medium, and the method comprises the steps: carrying out the heartbeat detection of each calculation node in real time through an independent thread, and obtaining the network connection state of each calculation node; dynamically updating the first list according to the at least one first computing node and the heartbeat time difference in one-to-one correspondence with the first computing nodes; transmitting data to each of the at least one first computing node through a plurality of threads according to the first list; when it is determined that at least one second computing node failed in data transmission exists in the data transmission process, task loads corresponding to the second computing nodes in a one-to-one mode are redistributed according to heartbeat time differences corresponding to the first computing nodes in a first list in a one-to-one mode; and sending the reallocated task load in one-to-one correspondence with each second computing node to each first computing node in the first list. The embodiment provided by the invention can improve the stability of the system.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a data transmission method, device, equipment and medium. Background Art

[0002] Open MPI (Open Message Passing Interface) is an open source, cross-platform message passing interface (MPI) implementation. It is mainly used in high-performance computing environments and supports multi-processor, multi-core and distributed computing systems. Based on Open MPI, cluster systems of various sizes can be built for high-performance computing, big data analysis, machine learning and other scenarios.

[0003] In an Open MPI cluster system, system stability is an important consideration because it directly affects the quality and efficiency of the completion of computing tasks. Common problems that affect the stability of an Open MPI cluster system include, for example, various problems that may occur in communication between nodes, or when computing tasks are unevenly distributed in a cluster system, some nodes may be overloaded while other nodes may be idle. Among them, problems that may be encountered in communication between nodes include network congestion, delay, packet loss, etc. These problems may cause unstable execution or even failure of computing tasks; and load imbalance will cause the overall performance of the system to decline, thus affecting the stability of the system. Summary of the invention

[0004] In view of the above defects or deficiencies in the prior art, it is desirable to provide a data transmission method, apparatus, device and medium to solve problems such as the stability of the existing Open MPI cluster system.

[0005] In a first aspect, an embodiment of the present invention provides a data transmission method, which is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The method is executed by the master node, and the method includes: performing heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically updating a first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node one by one, wherein the first computing node is a computing node with a normal network connection status among at least one computing node; transmitting data to each of the at least one first computing node according to the first list through multiple threads; when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process, reallocating the task amount corresponding to each second computing node one by one according to the heartbeat time difference corresponding to each first computing node in the first list; and sending the reallocated task amount corresponding to each second computing node one by one to each first computing node in the first list.

[0006] In a second aspect, an embodiment of the present invention further provides a data transmission device, which is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The device is configured in the master node, and the device includes: a heartbeat detection module, which is used to perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically update a first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node one by one, wherein the first computing node is a computing node with a normal network connection status among at least one computing node; a data transmission module, which is used to transmit data to each of the at least one first computing node according to the first list through multiple threads; a task allocation module, which is used to reallocate the task amount corresponding to each second computing node one by one according to the heartbeat time difference corresponding to each first computing node in the first list when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process; the data transmission module is also used to send the reallocated task amount corresponding to each second computing node one by one to each first computing node in the first list.

[0007] In a third aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present invention when executing the program.

[0008] In a fifth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the embodiment of the present invention is implemented.

[0009] The beneficial effects brought by the technical solution provided by the present invention are:

[0010] The present invention provides a data transmission method, device, equipment and medium. The method is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The method is executed by the master node. The method performs heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically updates a first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node one by one, wherein the first computing node is a computing node with a normal network connection status in at least one computing node; transmits data to each of the at least one first computing node according to the first list through multiple threads; when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process, the task amount corresponding to each second computing node one by one is reallocated according to the heartbeat time difference corresponding to each first computing node in the first list; and the reallocated task amount corresponding to each second computing node one by one is sent to each first computing node in the first list. The embodiment provided by the present invention realizes the reliability transmission between nodes in the Open MPI cluster system based on the heartbeat detection technology, and distributes and adjusts the tasks that fail to transmit according to the node network communication capability, which effectively improves the stability of the entire system and improves the success rate of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram showing a flow chart of a data transmission method proposed in an embodiment of the present invention is shown;

[0013] Figure 2 A schematic diagram showing a flow chart of a data transmission method proposed in an embodiment of the present invention;

[0014] Figure 3 A schematic diagram showing a flow chart of a method for retransmitting data when transmission fails, according to another embodiment of the present invention;

[0015] Figure 4 A schematic flow chart of a data retransmission method according to another embodiment of the present invention is shown;

[0016] Figure 5 A schematic diagram showing the structure of a data transmission device provided by an embodiment of the present invention is shown;

[0017] Figure 6A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the invention are shown in the accompanying drawings.

[0019] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] In an Open MPI cluster system, there may be various factors that cause node communication failures. For example, node network congestion, node crashes, or insufficient underlying cache in the Open MPI cluster system may cause nodes to fail to communicate. In an Open MPI cluster system, there is no effective method to directly and promptly determine whether nodes can communicate normally, nor is it possible to estimate the current network communication capabilities of nodes. As a result, the current Open MPI cluster system may crash due to these problems.

[0021] The present invention proposes a data transmission method, which realizes reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection technology, and allocates and adjusts tasks with transmission failures according to the network communication capabilities of the nodes, thereby improving the stability of the entire Open MPI cluster system.

[0022] In order to more clearly understand the inventive concept provided by the present invention, Figure 1-5 The image processing method proposed in the present invention is described.

[0023] Please refer to Figure 1 , Figure 1 The flowchart of the data transmission method proposed in the embodiment of the present invention is shown. The method is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The method can be implemented by a data transmission device configured in the master node.

[0024] The method includes:

[0025] Step S101, performing heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically updating the first list according to at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node with a normal network connection status among at least one computing node.

[0026] In the above steps, the heartbeat time difference is determined according to the sending time of the heartbeat packet sent by the master node and the receiving time of the heartbeat response data received from each computing node.

[0027] The first list can be represented by a hash table. For example, the first list is a heartbeat result hash table heartBeatResult maintained by the master node. In the heartbeat result hash table, the key value represents the process number rank of the computing node in the Open MPI cluster, and the value represents the time difference from when the computing node starts sending a heartbeat to when it receives a heartbeat reply.

[0028] In the above steps, the sending and receiving of heartbeat data can be done asynchronously. For example, the MPI_Isend function and the MPI_Irecv function can be used. Assume that the master node uses the MPI_Send function to send data and uses the MPI_Wait function to check whether the sending operation is successful. The heartbeat data can be in the form of a heartbeat packet. When an abnormality occurs in a computing node, it can only wait for the next round of heartbeat detection. Only when there is no error in the entire process of sending and receiving the heartbeat data can it be added to the heartbeat result hash table.

[0029] Step S102: transmitting data to each of the at least one first computing node according to the first list through multiple threads.

[0030] In the above steps, each thread is responsible for the data transmission of a computing node. In the Open MPI cluster system, for the master node to send data, it is only necessary to determine whether the master node sends the data correctly, without worrying about whether the computing node receives the data correctly, which effectively reduces the communication complexity.

[0031] When the master node receives data, the MPI_Irecv function, MPI_Test function and the heartbeat result hash table are used to determine whether the asynchronous reception of data from the computing node is normal.

[0032] Data is transmitted to each of at least one first computing node according to the first list through multiple threads, and multiple threads can be created to send and receive data. In the data sending operation process, it is determined whether the operation of sending data to the computing node is successful according to the API return value. If successful, the computing node is asynchronously waited for to return the result. If failed, the computing node is deleted from the first list and the computing node is recorded in the failed task list failedTaskList. For the asynchronous receiving operation, the MPI_Irecv function is used to asynchronously receive the result returned from the computing node. In the process of asynchronously receiving the returned result, MPI_Test is used to detect whether the data is asynchronously received through the while loop. If the result is that the asynchronous reception is not completed, the heartbeat result hash table heartBeatResult is queried to determine whether the computing node exists in the heartbeat result hash table heartBeatResult. Whether the computing node heartbeats correctly is detected by querying the heartbeat result hash table. If it does not exist in the heartbeat result hash table heartBeatResult, the asynchronously received result corresponding to the computing node is discarded, and the failed task is recorded in the failed task list failedTaskList.

[0033] Step S103, when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process, reallocate the task amount corresponding to each second computing node one-to-one according to the heartbeat time difference corresponding to each first computing node in the first list.

[0034] Step S104: Send the reallocated task amount corresponding to each second computing node one by one to each first computing node in the first list.

[0035] In the above steps, after all data is sent, all data that failed to be transmitted needs to be retransmitted, and the failed tasks are redistributed to the computing nodes with normal network connection status for retransmission according to the ratio of the heartbeat time difference.

[0036] The data transmission method provided by the embodiment of the present invention detects reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection, and reallocates tasks with failed transmission according to the node network communication capability, thereby improving the success rate of data transmission and reasonably utilizing cluster resources.

[0037] Please refer to Figure 2 , Figure 2 The flowchart of the data transmission method proposed in the embodiment of the present invention is shown. The method is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The method can be implemented by a data transmission device configured in the master node.

[0038] The method includes:

[0039] Step S201, performing heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically updating the first list according to at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node with a normal network connection status among at least one computing node.

[0040] In some embodiments, performing heartbeat detection on each computing node in real time through a separate thread may include cyclically sending heartbeat packets to each computing node in an asynchronous manner through a separate thread, and receiving heartbeat response data from each computing node; determining the heartbeat time difference corresponding to the computing node based on the sending time of the heartbeat packet and the receiving time of the heartbeat response data from each computing node; determining that the network connection status of the computing node is normal when the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold; or, determining that the network connection status of the computing node is abnormal when the heartbeat time difference corresponding to the computing node exceeds a preset time threshold.

[0041] In a multi-threaded environment, a single thread is used to run the heartbeat detection algorithm, and a heartbeat result hash table heartBeatResult is maintained in the master node. The heartbeat result hash table records the time spent on the entire heartbeat process of each computing node. If the heartbeat times out, it means that the heartbeat has failed, and the value of the computing node in the heartbeat result hash table is deleted. The thread can also include a while loop to cyclically perform heartbeat detection on all computing nodes and provide an external way to stop the loop.

[0042] In some embodiments, when the first list is a heartbeat result hash table, dynamically updating the first list based on at least one first computing node and the heartbeat time difference corresponding to the first computing node may include creating a heartbeat result hash table; determining that the computing node is the first computing node when it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold; adding the process number corresponding to the first computing node as a key value to the heartbeat result hash table; and adding the heartbeat time difference corresponding to the first computing node as a value to the storage location corresponding to the process number in the heartbeat result hash table.

[0043] Step S202: transmitting data to each of at least one first computing node according to the first list through multiple threads.

[0044] In some embodiments, when it is determined that the current process corresponding to multiple threads is included in the first list, data is sent to each first computing node; it is determined whether the operation of sending data to each first computing node is successful; if it is determined that the operation of sending data to the first computing node is successful, asynchronously wait for receiving the result returned from the first computing node; and in the process of asynchronously waiting for receiving the result returned from the first computing node, cyclically detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function for detecting whether the non-blocking communication operation is completed; if the master node completes the operation of asynchronously receiving data from the first computing node, then end; if the master node has not completed the operation of asynchronously receiving data from the first computing node, then add the process code corresponding to the first computing node to the failed task list; or, if it is determined that the operation of sending data to the first computing node fails, then add the process code corresponding to the first computing node to the failed task list, and delete the first computing node from the first list.

[0045] In some embodiments, cyclically detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node based on the first list and the first function for detecting whether the non-blocking communication operation is completed may include: detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node based on the first function; if it is detected according to the first function that the master node has not completed the operation of asynchronously receiving data from the first computing node, determining whether the process number corresponding to the first computing node is included in the first list; if the process number corresponding to the first computing node is included in the first list, returning to detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node based on the first function; if the process number corresponding to the first computing node is not included in the first list, discarding the data received asynchronously from the first computing node, and adding the first computing node to the failed task list.

[0046] For the receiving operation of the master node, the MPI_Irecv function can be used to asynchronously receive the result returned from the first computing node. After the function returns the result, the MPI_Test function can be used in the while loop to detect whether the asynchronously received data has been received. If the result is that the asynchronous reception is not completed, query the heartbeat result hash table heartBeatResult, and determine whether the computing node exists through the heartbeat result hash table, so as to detect whether the heartbeat of the computing node is correct. If it does not exist, the result of the asynchronous reception is discarded, and the failed task is recorded in the failed task list failedTaskList.

[0047] Step S203: when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process, the heartbeat time difference corresponding to each first computing node is converted into the same time unit.

[0048] Step S204, calculating the ratio of the heartbeat time differences corresponding to each first computing node one by one.

[0049] Step S205, normalizing the comparison value to obtain a normalized ratio.

[0050] After the above steps, the first computing node and the normalized ratio corresponding to the first computing node are stored in a second list.

[0051] Step S206: reallocate the task amount corresponding to each second computing node one by one according to the normalized ratio.

[0052] Step S207: Send the reallocated task amount corresponding to each second computing node one by one to each first computing node in the first list.

[0053] In the above steps, the first list is the heartbeat result hash table heartBeatResult, which converts the heartbeat time into the same unit of milliseconds, records the ratio of the heartbeat time of all computing nodes and saves it in a one-dimensional array, takes the inverse of the ratio, normalizes it, and then saves the relationship between the computing node and the ratio in the second list. The second list can be a hash table powerList.

[0054] According to the failed task list failedTaskList and the heartbeat result hash table heartBeatResult that record failed tasks, the data corresponding to the failed tasks are redistributed to each node in the heartbeat result hash table according to the network capability value, and data sending is restarted. After the data sending is completed, the failed task list failedTaskList continues to be judged. If the failed task list is not empty, it returns to continue to resend the failed tasks in the failed task list.

[0055] Combine the following Figure 3, further illustrating the process of reallocating and sending the data that failed to be sent. Step S301, determine whether the failed task list is empty. When it is not empty, go to step S302. When it is empty, end. Step S302, traverse the heartbeat result hash table heartBeatResult, and convert each heartbeat time difference into the same time unit. Step S303, calculate the ratio of the heartbeat time difference corresponding to each computing node. Step S304, take the inverse of each ratio to obtain a normalized ratio. Step S305, store the computing node and the normalized ratio corresponding to the computing node in the hash table powerlist. Step S306, assign the failed task to the first computing node contained in the heartbeat result hash table according to the normalized ratio, and restart data sending. By the above-mentioned method of reallocating tasks, the success rate of data transmission can be improved and cluster resources can be used reasonably.

[0056] The data transmission method provided by the embodiment of the present invention is based on the reliability transmission between nodes in the heartbeat detection Open MPI cluster system, and reallocates the tasks with transmission failure according to the node network communication capability, so as to improve the success rate of data transmission and reasonably utilize cluster resources.

[0057] Please refer to Figure 4 , Figure 4 The flowchart of the data transmission method proposed in the embodiment of the present invention is shown. The method is used in an Open MPI cluster system. The Open MPI cluster system includes a master node and at least one computing node. The method can be implemented by a data transmission device configured in the master node.

[0058] The method includes:

[0059] Step S401 , a heartbeat packet is cyclically sent to each computing node in an asynchronous manner through a separate thread, and heartbeat response data is received from each computing node.

[0060] Step S402: Determine the heartbeat time difference corresponding to the computing node according to the sending time of the heartbeat packet and the receiving time of the heartbeat response data received from each computing node.

[0061] Step S403: determining the network connection status of the computing node according to whether the heartbeat time difference corresponding to the computing node has timed out.

[0062] Step S404, when the heartbeat time difference corresponding to the computing node does not exceed the preset time threshold, determine that the computing node is a first computing node, and add the heartbeat time difference corresponding to the first computing node to a first list, wherein the first computing node is a computing node with a normal network connection status;

[0063] Step S405: When the heartbeat time difference corresponding to the computing node exceeds a preset time threshold, it is determined that the network connection state of the computing node is abnormal, and the data corresponding to the computing node is deleted from the first list.

[0064] Step S406: Transmit data to each of the at least one computing node through multiple threads respectively.

[0065] Step S407: determine whether the current process corresponding to the multiple threads is included in the first list.

[0066] Step S408: When it is determined that the current process corresponding to the multiple threads is included in the first list, data is sent to each first computing node.

[0067] Step S409: Determine whether the operation of sending data to each first computing node is successful.

[0068] Step S410: If it is determined that the operation of sending data to the first computing node is successful, the master node asynchronously waits to receive a result returned from the first computing node.

[0069] Step S411, in the process of asynchronously waiting to receive the result returned from the first computing node, cyclically detect whether the main node has completed the operation of asynchronously receiving data from the first computing node based on the first list and the first function for detecting whether the non-blocking communication operation is completed.

[0070] Step S412: If the master node completes the operation of asynchronously receiving data from the first computing node, the process ends.

[0071] Step S413: If the master node has not completed the operation of asynchronously receiving data from the first computing node, the process code corresponding to the first computing node is added to the failed task list.

[0072] Step S414: if it is determined that the operation of sending data to the first computing node fails, the process code corresponding to the first computing node is added to the failed task list, and the first computing node is deleted from the first list.

[0073] The data transmission method provided by the embodiment of the present invention detects reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection, and reallocates tasks with failed transmission according to the node network communication capability, thereby improving the success rate of data transmission and reasonably utilizing cluster resources.

[0074] The process diagram and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0075] Optionally, refer to Figure 5 , Figure 5 The structure diagram of the data transmission device provided by the embodiment of the present invention is shown. The device is used in an Open MPI cluster system, and the Open MPI cluster system includes a master node and at least one computing node, and is characterized in that the device is configured in the master node, and the device includes:

[0076] The heartbeat detection module 501 is used to perform heartbeat detection on each computing node in real time through a separate thread to obtain the network connection status of each computing node; and dynamically update the first list according to at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node with a normal network connection status among the at least one computing node;

[0077] A data transmission module 502, configured to transmit data to each of the at least one first computing node according to the first list through a plurality of threads;

[0078] The task allocation module 503 is used to reallocate the task amount corresponding to each second computing node one-to-one according to the heartbeat time difference corresponding to each first computing node in the first list when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process;

[0079] The data transmission module 502 is further configured to send the reallocated task amount corresponding to each second computing node one by one to each first computing node in the first list.

[0080] Optionally, the heartbeat detection module 501 is used to cyclically send heartbeat packets to each computing node in an asynchronous manner through a separate thread, and receive heartbeat response data from each computing node; determine the heartbeat time difference corresponding to the computing node based on the sending time of the heartbeat packet and the receiving time of the heartbeat response data from each computing node; when the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, determine that the network connection status of the computing node is normal; or, when the heartbeat time difference corresponding to the computing node exceeds a preset time threshold, determine that the network connection status of the computing node is abnormal.

[0081] Optionally, when the first list is a heartbeat result hash table, the heartbeat detection module 501 is also used to create a heartbeat result hash table; when it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, the computing node is determined to be the first computing node; the process number corresponding to the first computing node is added as the key value to the heartbeat result hash table; the heartbeat time difference corresponding to the first computing node is added as the value to the storage location corresponding to the process number in the heartbeat result hash table.

[0082] Optionally, the task allocation module 503 is also used to: convert the heartbeat time difference corresponding to each first computing node one by one into the same time unit; calculate the ratio of the heartbeat time difference corresponding to each first computing node one by one; normalize the comparison value to obtain a normalized ratio; and reallocate the task amount corresponding to each second computing node one by one according to the normalized ratio.

[0083] Optionally, the task allocation module 503 is further configured to: store the first computing node and the normalized ratio corresponding to the first computing node in a second list.

[0084] Optionally, the data transmission module 502 is also used to: when it is determined that the current process corresponding to multiple threads is included in the first list, send data to each first computing node; determine whether the operation of sending data to each first computing node is successful; if it is determined that the operation of sending data to the first computing node is successful, asynchronously wait for receiving the result returned from the first computing node; and in the process of asynchronously waiting for receiving the result returned from the first computing node, cyclically detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function for detecting whether the non-blocking communication operation is completed; if the master node completes the operation of asynchronously receiving data from the first computing node, then end; if the master node has not completed the operation of asynchronously receiving data from the first computing node, add the process code corresponding to the first computing node to the failed task list; or, if it is determined that the operation of sending data to the first computing node fails, add the process code corresponding to the first computing node to the failed task list, and delete the first computing node from the first list at the same time.

[0085] Optionally, the data transmission module 502 is also used to detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; if it is detected according to the first function that the master node has not completed the operation of asynchronously receiving data from the first computing node, determine whether the process number corresponding to the first computing node is included in the first list; if the process number corresponding to the first computing node is included in the first list, return to detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; if the process number corresponding to the first computing node is not included in the first list, discard the data received asynchronously from the first computing node, and add the first computing node to the failed task list.

[0086] The data transmission device provided by the embodiment of the present invention realizes reliable transmission between nodes in an Open MPI cluster system based on heartbeat detection, and reallocates tasks with failed transmission according to the node network communication capability, thereby improving the success rate of data transmission and rationally utilizing cluster resources.

[0087] Reference below Figure 6 , Figure 6 The schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown. The electronic device may be a computer device. Figure 6 It does not limit the structure of the electronic device. Figure 5 As shown, the electronic device at least includes a memory 601 and a processor 602. For example, the electronic device may also include Figure 5 More or fewer components (eg, network interfaces, display devices, etc.) may be shown.

[0088] In particular, according to the embodiment provided by the present invention, the above reference flow chart Figure 1-3The described process can be implemented as a computer software program. For example, the embodiments provided by the present invention include a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program contains program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the system of the present invention are executed.

[0089] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products described in various embodiments provided by the present invention. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] The units or modules involved in the embodiments described in the present invention may be implemented in software or hardware. The units or modules described may also be arranged in a processor. For example, they may be described as: a processor includes a heartbeat detection module, a data transmission module, and a task allocation module. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves. For example, the training sample acquisition module may also be described as "a module for performing heartbeat detection on each computing node through a separate thread."

[0092] As another aspect, an embodiment of the present invention further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently and not be assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the data transmission method described in the present invention.

[0093] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A data transmission method, the method being used in an Open MPI cluster system, the Open MPI cluster system comprising a master node and at least one computing node, characterized in that: The method is executed by a master node, and includes: Performing heartbeat detection on each computing node in real time through a separate thread to obtain a network connection status of each computing node; and dynamically updating a first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node with a normal network connection status among the at least one computing node; transmitting data to each of the at least one first computing node according to the first list by a plurality of threads; When it is determined during the data transmission process that there is at least one second computing node that fails to transmit data, reallocate the task amount corresponding to each of the second computing nodes one by one according to the heartbeat time difference corresponding to each of the first computing nodes in the first list; The reallocated task amount corresponding to each second computing node one by one is sent to each first computing node in the first list.

2. The method according to claim 1, characterized in that The performing of heartbeat detection on each computing node in real time through a separate thread includes: Sending heartbeat packets to each computing node cyclically in an asynchronous manner through a separate thread, and receiving heartbeat response data from each computing node; Determine the heartbeat time difference corresponding to the computing node according to the sending time of the heartbeat packet and the receiving time of the heartbeat response data received from each computing node; When the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, determining that the network connection state of the computing node is normal; or, When the heartbeat time difference corresponding to the computing node exceeds a preset time threshold, it is determined that the network connection state of the computing node is abnormal.

3. The method according to claim 1, characterized in that: When the first list is a heartbeat result hash table, dynamically updating the first list according to at least one first computing node and a heartbeat time difference corresponding to the first computing node one by one includes: Create the heartbeat result hash table; When it is determined that the heartbeat time difference corresponding to the computing node does not exceed a preset time threshold, determining that the computing node is the first computing node; Adding the process number corresponding to the first computing node as a key value to the heartbeat result hash table; The heartbeat time difference corresponding to the first computing node is taken as the value and added to the storage location corresponding to the process number in the heartbeat result hash table.

4. The method according to claim 1, characterized in that According to the heartbeat time difference in one-to-one correspondence with each of the first computing nodes in the first list, reallocating the task amount in one-to-one correspondence with each of the second computing nodes includes: Converting the heartbeat time difference corresponding to each of the first computing nodes one by one into the same time unit; Calculating a ratio of heartbeat time differences corresponding one-to-one to each of the first computing nodes; normalizing the ratio to obtain a normalized ratio; The task amount corresponding to each of the second computing nodes is redistributed according to the normalized ratio.

5. The method according to claim 4, characterized in that After normalizing the ratio to obtain a normalized ratio, the method includes: The first computing node and the normalized ratio corresponding to the first computing node are stored in a second list.

6. The method according to claim 1, characterized in that Transmitting data to each of the at least one first computing node according to the first list through a plurality of threads includes: When it is determined that the current processes corresponding to the plurality of threads are included in the first list, sending data to each of the first computing nodes; Determining whether an operation of sending data to each of the first computing nodes is successful; If it is determined that the operation of sending data to the first computing node is successful, asynchronously wait for receiving the result returned from the first computing node; and in the process of asynchronously waiting for receiving the result returned from the first computing node, cyclically detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function for detecting whether the non-blocking communication operation is completed; If the master node completes the operation of asynchronously receiving data from the first computing node, then the process ends; If the master node fails to complete the operation of asynchronously receiving data from the first computing node, adding the process code corresponding to the first computing node to a failed task list; or, If it is determined that the operation of sending data to the first computing node fails, the process code corresponding to the first computing node is added to the failed task list, and the first computing node is deleted from the first list.

7. The method according to claim 6, characterized in that The cyclically detecting whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first list and the first function for detecting whether the non-blocking communication operation is completed comprises: Detecting, according to the first function, whether the master node has completed an operation of asynchronously receiving data from the first computing node; If it is detected according to the first function that the master node has not completed the operation of asynchronously receiving data from the first computing node, determining whether the process number corresponding to the first computing node is included in the first list; If the process number corresponding to the first computing node is included in the first list, returning to detect whether the master node has completed the operation of asynchronously receiving data from the first computing node according to the first function; If the process number corresponding to the first computing node is not included in the first list, the data asynchronously received from the first computing node is discarded, and the first computing node is added to the failed task list.

8. A data transmission device, the device being used in an Open MPI cluster system, the Open MPI cluster system comprising a master node and at least one computing node, characterized in that: The device is configured in the master node, and includes: A heartbeat detection module is used to perform heartbeat detection on each of the computing nodes in real time through a separate thread to obtain the network connection status of each of the computing nodes; and dynamically update the first list according to at least one first computing node and the heartbeat time difference corresponding to the first computing node, wherein the first computing node is a computing node with a normal network connection status among the at least one computing node; a data transmission module, configured to transmit data to each of the at least one first computing node according to the first list through a plurality of threads; A task allocation module, configured to reallocate the task amount corresponding to each of the second computing nodes one by one according to the heartbeat time difference corresponding to each of the first computing nodes in the first list when it is determined that there is at least one second computing node that fails to transmit data during the data transmission process; The data transmission module is further configured to send the reallocated task amount corresponding one-to-one to each second computing node to each first computing node in the first list.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images