Distributed data transmission method and device, electronic equipment and storage medium

Abstract

The invention provides a distributed data transmission method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: dividing a distributed training task to obtain a plurality of sub-tasks; determining a transmission sequence of the sub-tasks according to the dependency relationship between the sub-tasks; determining a transmission path of each subtask according to the network state data, wherein any transmission path is used for describing a transmission path for transmitting any subtask to a corresponding computing node; and according to the transmission sequence and the transmission path, transmitting each sub-task to the corresponding computing node in sequence. According to the invention, the transmission sequence of each sub-task obtained by division can be determined based on the dependency relationship between the sub-tasks, and the transmission path of each sub-task is determined according to the network state data, so that the data transmission efficiency and reliability can be further improved.

Description

Background Art

[0002] With the development of computer technology, the training of large-scale models has become an important force in promoting technological progress. As the scale of models continues to expand, large models can be trained in a distributed manner. In a distributed architecture, multiple nodes are involved, and a large amount of data needs to be transmitted between nodes, which places extremely high demands on the efficiency and reliability of data transmission methods.

[0003] Therefore, as the scale of large model training continues to expand, there is an urgent need for a data transmission method that can operate efficiently in distributed training scenarios, so as to improve the efficiency and reliability of distributed data transmission.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a distributed data transmission method, device, electronic device and storage medium, which improve the efficiency and reliability of data transmission at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a distributed data transmission method is provided, including:

[0008] Divide the distributed training task into multiple subtasks;

[0009] Determine the transmission order of each subtask based on the dependencies between subtasks;

[0010] Determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node;

[0011] According to the transmission order and the transmission path, each subtask is transmitted to the corresponding computing node in sequence.

[0012] In some exemplary embodiments, determining the transmission path of each subtask according to the network status data includes:

[0013] Predict future network status data based on current network status data and historical network status data within a preset time period;

[0014] The transmission path of each subtask is determined according to the predicted future network status data.

[0015] In some exemplary embodiments, the distributed data transmission method provided by the embodiments of the present disclosure further includes:

[0016] Get current network status data in real time;

[0017] Adjusting the transmission path according to the current network status data to obtain an adjusted transmission path;

[0018] The transmitting of each subtask to a corresponding computing node in sequence according to the transmission order and the transmission path includes:

[0019] According to the transmission order and the adjusted transmission path, each subtask is transmitted to the corresponding computing node in sequence.

[0020] In some exemplary embodiments, the distributed data transmission method provided by the embodiments of the present disclosure further includes:

[0021] In response to detecting a transmission error of the first subtask, switching a transmission path of the first subtask;

[0022] The first subtask is retransmitted through the switched transmission path.

[0023] In some exemplary embodiments, determining the transmission order of each subtask according to the dependency relationship between the subtasks includes:

[0024] Determining the scheduling priority of each subtask according to the dependency relationship between the subtasks;

[0025] The transmission order of each subtask is determined by the scheduling priority.

[0026] In some exemplary embodiments, determining the scheduling priority of each subtask according to the dependencies between the subtasks includes:

[0027] The scheduling priority of each subtask is determined according to the dependency relationship between the subtasks and at least one of the following information: computational complexity, resource requirement, node load, and network bandwidth.

[0028] According to another aspect of the present disclosure, there is also provided a distributed data transmission device, including:

[0029] A subtask determination module is used to divide the distributed training task into multiple subtasks;

[0030] A transmission order determination module is used to determine the transmission order of each subtask according to the dependency relationship between subtasks;

[0031] A transmission path determination module, used to determine the transmission path of each subtask according to the network status data, wherein any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node;

[0032] The subtask transmission module is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and the transmission path.

[0033] In some exemplary embodiments, the transmission path determination module is used to predict future network status data based on current network status data and historical network status data within a preset time period; and determine the transmission path of each subtask based on the predicted future network status data.

[0034] In some exemplary embodiments, the distributed data transmission device provided by the embodiments of the present disclosure further includes:

[0035] The transmission path adjustment module is used to obtain the current network status data in real time; adjust the transmission path according to the current network status data to obtain the adjusted transmission path;

[0036] The subtask transmission module is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and the adjusted transmission path.

[0037] In some exemplary embodiments, the subtask transmission module is further configured to switch a transmission path of the first subtask in response to detecting a transmission error of the first subtask; and retransmit the first subtask through the switched transmission path.

[0038] In some exemplary embodiments, the transmission order determination module is used to determine the scheduling priority of each subtask according to the dependency relationship between the subtasks; and determine the transmission order of each subtask according to the scheduling priority.

[0039] In some exemplary embodiments, the transmission order determination module is used to determine the scheduling priority of each subtask based on the dependencies between the subtasks and at least one of the information of computational complexity, resource requirements, node load, and network bandwidth.

[0040] According to another aspect of the present disclosure, there is also provided a distributed data transmission system, including:

[0041] The coordination layer is used to divide the distributed training tasks into multiple subtasks;

[0042] The scheduling layer is used to determine the transmission order of each subtask according to the dependencies between the subtasks; determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node;

[0043] The network layer is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and the transmission path.

[0044] According to another aspect of the present disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned distributed data transmission methods by executing the executable instructions.

[0045] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above-mentioned distributed data transmission methods is implemented.

[0046] According to another aspect of the present disclosure, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the distributed data transmission method provided in various optional ways in any embodiment of the present disclosure.

[0047] The technical solution provided in the embodiment of the present disclosure can determine the transmission order of each subtask obtained by division based on the dependency relationship between subtasks, and determine the transmission path of each subtask according to network status data. Therefore, the embodiment of the present disclosure can further improve the efficiency and reliability of data transmission.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0050] Figure 1 A schematic diagram showing a system architecture in an embodiment of the present disclosure;

[0051] Figure 2 A flow chart of a distributed data transmission method in an embodiment of the present disclosure is shown;

[0052] Figure 3A schematic diagram of a distributed data transmission communication protocol architecture in an embodiment of the present disclosure is shown;

[0053] Figure 4 A flow chart of a distributed data transmission communication method in an embodiment of the present disclosure is shown;

[0054] Figure 5 A schematic diagram of a distributed data transmission device in an embodiment of the present disclosure is shown;

[0055] Figure 6 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0056] Figure 7 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0058] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0059] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0060] Figure 1 FIG. 2 shows an exemplary application system architecture diagram to which the distributed data transmission method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture 100 may include a first server 101 , a network 102 , and a second server 103 .

[0061] Among them, the first server 101 is used to divide the distributed training task to obtain multiple subtasks; determine the transmission order of each subtask according to the dependency relationship between the subtasks; determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node; according to the transmission order and the transmission path, each subtask is transmitted to the corresponding computing node in turn. The second server 103 can be any computing node, so the second server 103 can obtain the corresponding subtask and perform the corresponding calculation.

[0062] Exemplarily, the second server 103 may also return the calculation result to the first server 101 .

[0063] The network 102 is used as a medium for providing a communication link between the first server 101 and the second server 103 , and may be a wired network or a wireless network.

[0064] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0065] The first server 101 and the second server 103 can both be servers that provide various services, such as a background management server that provides support for devices operated by users using terminal devices. The background management server can analyze and process the received request data, and feed back the processing results to the terminal device.

[0066] Optionally, the first server 101 and the second server 103 can be independent physical servers, or a server cluster or distributed system composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0067] Those skilled in the art will know that Figure 1 The number of the first server 101, the network 102, and the second server 103 is only illustrative, and any number of the first server 101, the network 102, and the second server 103 may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.

[0068] Under the above system architecture, a distributed data transmission method is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0069] In some embodiments, the distributed data transmission method provided in the embodiments of the present disclosure can be executed by the first server of the above-mentioned system architecture; in other embodiments, the distributed data transmission method provided in the embodiments of the present disclosure can be implemented by the first server and the second server in the above-mentioned system architecture through interaction.

[0070] Figure 2 A flow chart of a distributed data transmission method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the distributed data transmission method provided in the embodiment of the present disclosure includes the following steps S202 to S208.

[0071] S202, dividing the distributed training task into multiple subtasks.

[0072] The embodiments of the present disclosure do not limit the content of the distributed training task. For example, the distributed training task may be for an image recognition model. In this case, distributed training may be performed in a data parallel manner. Specifically, the data set may be segmented and assigned to different computing nodes for corresponding calculations. Alternatively, the distributed training task may be for a language model. In this case, the various parts of the model may be assigned to different nodes for training.

[0073] In addition, the embodiment of the present disclosure does not limit the data volume of the subtasks, and the data volume of each subtask can be determined based on experience or application scenarios.

[0074] S204: Determine the transmission order of each subtask according to the dependency relationship between the subtasks.

[0075] In an exemplary embodiment, the dependency relationship between subtasks can be used to describe the order or interdependence between the subtasks. For example, if the output of subtask A is the input of subtask B, then subtask A and subtask B have a dependency relationship. Moreover, subtask A is a dependent task of subtask B. For another example, the logical execution order of tasks can be determined based on the hierarchical structure of the model. If subtask A needs to be executed before subtask B, then subtask A and subtask B have a dependency relationship, and subtask A is a dependent task of subtask B.

[0076] In some exemplary embodiments, determining the transmission order of each subtask according to the dependencies between subtasks includes: determining the scheduling priority of each subtask according to the dependencies between subtasks; and determining the transmission order of each subtask according to the scheduling priority.

[0077] For example, still taking the above subtask A and subtask B as an example, since subtask A is a dependent task of subtask B, the scheduling priority of subtask A needs to be higher than that of subtask B, so that the transmission order of subtask A is before subtask B.

[0078] In an exemplary embodiment, the scheduling priority can be determined for each subtask, so that the transmission order of each subtask is obtained according to the order of scheduling priority from high to low. The embodiment of the present disclosure does not limit the number of levels of scheduling priority.

[0079] In some exemplary embodiments, the scheduling priority of each subtask is determined based on the dependencies between subtasks, including: determining the scheduling priority of each subtask based on the dependencies between subtasks and at least one of the information including computational complexity, resource requirements, node load, and network bandwidth.

[0080] Exemplarily, when determining the scheduling priority, in addition to considering the dependencies between subtasks, at least one of the following information may also be considered: computational complexity, resource requirements, node load, and network bandwidth.

[0081] In a possible implementation, for computational complexity, if the computational amount of a subtask is higher than a preset computational amount threshold, the subtask can be assigned priority, thereby reducing the waiting time of other nodes. For resource requirements, if a subtask has high requirements for specific hardware resources, the subtask can be assigned priority. Specific hardware resources may be, for example, GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), etc. For node load, the priority of the subtask assigned to the node can be adjusted according to the node load of the current node to avoid node overload. The node load can be determined, for example, based on CPU (Central Processing Unit) and memory usage. For network bandwidth, if a subtask requires higher bandwidth, the subtask can be scheduled first when the network bandwidth is sufficient.

[0082] In an exemplary embodiment, each divided subtask may be in the form of a data packet. After determining the scheduling priority of each subtask, a corresponding priority tag may be added to the data packet corresponding to each subtask.

[0083] In a possible implementation, after determining the scheduling priority corresponding to each subtask, the subtask execution status of each node may be detected in real time, and the transmission order of the subtasks and the computing nodes corresponding to the subtasks may be adjusted according to the detection results.

[0084] S206, determining a transmission path for each subtask according to the network status data, wherein any transmission path is used to describe a transmission path for transmitting any subtask to a corresponding computing node.

[0085] In some possible embodiments, the network status data may include at least one of bandwidth, network delay, and packet loss rate.

[0086] For example, the current network status data can be detected in real time, and a transmission path with high bandwidth, low network delay, and low packet loss rate can be preferentially selected for transmission of subtasks. The selection of transmission paths based on network status data can effectively improve the reliability and efficiency of data transmission.

[0087] In some exemplary embodiments, the transmission path of each subtask is determined based on the network status data, including: predicting future network status data based on the current network status data and historical network status data within a preset time period; and determining the transmission path of each subtask based on the predicted future network status data.

[0088] In an exemplary embodiment, for example, if the bandwidth contained in the current network status data suddenly drops, or if there are multiple problems in the historical network status data within a preset time period, it can be predicted that the transmission path may have problems in the future, so other transmissions can be given priority to transmit the subtask.

[0089] In addition, the method provided by the embodiment of the present disclosure also includes: determining a cache adjustment strategy based on network status data. It should be noted that the embodiment of the present disclosure can dynamically optimize the cache size of data based on real-time bandwidth usage, network delay, packet loss rate and other information. For example, when problems such as sudden bandwidth drop occur, the cache of data on the node can be increased to avoid direct transmission of data through links with higher loads and select a more idle transmission path. Therefore, the embodiment of the present disclosure can effectively avoid delays caused by excessively large caches, or frequent retransmissions caused by too small caches.

[0090] In a possible implementation, the optimal transmission path and cache strategy may be adaptively selected based on the reinforcement learning technology according to the network delay and transmission efficiency to improve the transmission performance.

[0091] S208, transmitting each subtask to a corresponding computing node in sequence according to the transmission order and the transmission path.

[0092] In a possible implementation, each subtask may be transmitted to a corresponding node through a determined transmission sequence and a corresponding transmission path.

[0093] In some exemplary embodiments, the distributed data transmission method provided by the embodiments of the present disclosure may further include: acquiring current network status data in real time; and adjusting the transmission path according to the current network status data to obtain an adjusted transmission path.

[0094] In this case, each subtask is transmitted to the corresponding computing node in sequence according to the transmission order and the transmission path, including: each subtask is transmitted to the corresponding computing node in sequence according to the transmission order and the adjusted transmission path.

[0095] Exemplarily, after the transmission path of each subtask is determined, the transmission path can be adjusted in real time. For example, the transmission path of each subtask can be adaptively adjusted in real time based on a dynamic optimization algorithm, thereby minimizing network delay and packet loss rate.

[0096] In some exemplary embodiments, the distributed data transmission method provided by the embodiments of the present disclosure may further include: in response to detecting a transmission error of the first subtask, switching the transmission path of the first subtask; and retransmitting the first subtask through the switched transmission path.

[0097] In an exemplary embodiment, the first subtask may be any subtask. During the transmission of the subtask, the present disclosure may continuously perform error detection and trigger an error correction mechanism when necessary. Exemplarily, the error detection is implemented through a multi-level checksum and redundancy strategy, and the error correction mechanism may be completed based on a retransmission mechanism and a switching of a transmission path.

[0098] The method provided by the embodiment of the present disclosure can determine the transmission order of each subtask obtained by division based on the dependency relationship between subtasks, and determine the transmission path of each subtask according to network status data. Therefore, the embodiment of the present disclosure can further improve the efficiency and reliability of data transmission.

[0099] For example, a distributed data transmission communication protocol architecture diagram provided by an embodiment of the present disclosure can be as follows: Figure 3 shown.

[0100] In this Figure 3 In the protocol architecture, the protocol architecture may include a coordination layer, a scheduling layer, a network layer, and an underlying hardware layer.

[0101] Among them, the coordination layer includes the task allocation module and the task scheduling and management module. The scheduling layer includes the priority scheduling module, the cache and routing optimization module, and the error detection and correction module. The network layer includes the transmission protocol stack module, the data grouping and packet processing module, and the real-time network monitoring scheduling module. The underlying hardware layer includes the computing resource management module and the hardware interface.

[0102] Exemplarily, the coordination layer can be the top layer of the protocol architecture, which is used to coordinate and manage the entire distributed training task. The coordination layer can interact with the large model training framework. Among them, the task allocation module can be used to divide the distributed training task, obtain multiple subtasks, and allocate these subtasks to each node according to the computing power of the node and the network status information. The task scheduling and management module can be used to monitor the execution of the node's subtasks in real time and adjust the node allocation in real time.

[0103] Exemplarily, the scheduling layer can be the core of the protocol architecture, responsible for managing and scheduling data transmission across nodes. Through a series of optimization strategies, it is ensured that data can be transmitted according to priority, real-time, and reliability. Among them, the priority scheduling module is used to determine the transmission order of each subtask according to the dependency relationship between subtasks. The cache and routing optimization module can be used to determine the transmission path of each subtask according to the network status data. In addition, the error detection and error correction module can be used to perform error detection and switch the transmission path of the first subtask when a transmission error of the first subtask is detected.

[0104] Exemplarily, the network layer can transmit each subtask to the corresponding computing node in sequence according to the transmission order and the transmission path. Among them, the transmission protocol stack module can be used to process the encapsulation, grouping, and network protocol control of data packets to ensure the integrity and efficient transmission of data. The data grouping and packet processing module can be used to allocate subtask data to different transmission paths to achieve maximum parallelism and minimize transmission delay. The real-time network monitoring and scheduling module can be used to detect network status data and adaptively adjust the transmission path of the subtask packet.

[0105] Exemplarily, the computing resource management module in the underlying hardware layer can be used to directly interact with the underlying hardware resources to ensure that the communication protocol can adapt to various hardware architectures. For example, for GPUs, TPUs, CPU clusters, etc., the protocol architecture provided by the embodiments of the present disclosure can adjust the computing and transmission tasks according to the hardware characteristics. In addition, the hardware interface can be a unified interface that can adapt to different hardware architectures, thereby achieving direct interaction with the underlying hardware.

[0106] For example, a flow chart of a distributed data transmission communication method provided by an embodiment of the present disclosure may be as follows: Figure 4 As shown. Figure 4 In the embodiment, the distributed data transmission method may include the following steps S402 to S410.

[0107] S402, the distributed training task starts.

[0108] S404, the distributed training task is divided into various subtasks, and the scheduling priority of each subtask is calculated.

[0109] S406, determining a transmission path and a cache optimization strategy.

[0110] S408, perform error detection and error correction.

[0111] S410, the distributed training task is completed.

[0112] It should be noted that the implementation of the above steps S402 to S410 can refer to the corresponding descriptions in the above steps S202 to S208, which will not be repeated here.

[0113] It should be noted that the acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations, and various types of data such as personal identity data, operation data, behavioral data, etc. related to individuals, customers, and groups obtained in the embodiments of this disclosure have all been authorized.

[0114] Based on the same inventive concept, the present disclosure also provides a distributed data transmission device in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0115] Figure 5 A schematic diagram of a distributed data transmission device in an embodiment of the present disclosure is shown. Figure 5 As shown, the device comprises:

[0116] A subtask determination module 501 is used to divide the distributed training task into multiple subtasks;

[0117] The transmission order determination module 502 is used to determine the transmission order of each subtask according to the dependency relationship between the subtasks;

[0118] A transmission path determination module 503 is used to determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node;

[0119] The subtask transmission module 504 is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and transmission path.

[0120] In some exemplary embodiments, the transmission path determination module 503 is used to predict future network status data based on current network status data and historical network status data within a preset time period; and determine the transmission path of each subtask based on the predicted future network status data.

[0121] In some exemplary embodiments, the distributed data transmission device provided by the embodiments of the present disclosure further includes:

[0122] The transmission path adjustment module is used to obtain the current network status data in real time; adjust the transmission path according to the current network status data to obtain the adjusted transmission path;

[0123] The subtask transmission module 504 is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and the adjusted transmission path.

[0124] In some exemplary embodiments, the subtask transmission module 504 is further configured to switch the transmission path of the first subtask in response to detecting a transmission error of the first subtask; and retransmit the first subtask through the switched transmission path.

[0125] In some exemplary embodiments, the transmission order determination module 502 is used to determine the scheduling priority of each subtask according to the dependency relationship between the subtasks; and determine the transmission order of each subtask according to the scheduling priority.

[0126] In some exemplary embodiments, the transmission order determination module 502 is used to determine the scheduling priority of each subtask based on the dependency between subtasks and at least one of the following information: computational complexity, resource requirements, node load, and network bandwidth.

[0127] It should be noted that the above subtask determination module 501, transmission order determination module 502, transmission path determination module 503, and subtask transmission module 504 correspond to S202 to S208 in the method embodiment, and the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer executable instructions.

[0128] The device provided by the embodiment of the present disclosure can determine the transmission order of each subtask obtained by division based on the dependency relationship between subtasks, and determine the transmission path of each subtask according to network status data. Therefore, the embodiment of the present disclosure can further improve the efficiency and reliability of data transmission.

[0129] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0130] The embodiment of the present disclosure provides an electronic device, which exemplarily includes: a processor and a memory. The memory can be used to store executable instructions of the processor. The processor is configured to execute the above executable instructions to implement the distributed data transmission device method provided by the embodiment of the present disclosure.

[0131] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present disclosure is described. Figure 6 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0132] like Figure 6As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).

[0133] The storage unit stores a program code, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 610 can perform the following steps of the above method embodiment:

[0134] The distributed training task is divided into multiple subtasks; the transmission order of each subtask is determined according to the dependency relationship between the subtasks; the transmission path of each subtask is determined according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node; according to the transmission order and the transmission path, each subtask is transmitted to the corresponding computing node in turn.

[0135] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0136] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0137] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0138] The electronic device 600 may also communicate with one or more external devices 640 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0139] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0140] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above-mentioned distributed data transmission method when executed by a processor.

[0141] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the distributed data transmission method provided in the embodiment of the present disclosure can be implemented. The computer-readable storage medium can be a readable signal medium or a readable storage medium.

[0142] Figure 7 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. Figure 7As shown, a program product capable of implementing the above method of the present disclosure is stored on the computer-readable storage medium 700. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0143] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0146] In a specific implementation, the program code for performing the operation of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0147] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0148] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0149] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0150] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope of the present disclosure is indicated by the appended claims.

Claims

1. A distributed data transmission method, characterized in that: include: Divide the distributed training task into multiple subtasks; Determine the transmission order of each subtask based on the dependencies between subtasks; Determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node; According to the transmission order and the transmission path, each subtask is transmitted to the corresponding computing node in sequence.

2. The distributed data transmission method according to claim 1, characterized in that: Determining the transmission path of each subtask according to the network status data includes: Predict future network status data based on current network status data and historical network status data within a preset time period; The transmission path of each subtask is determined according to the predicted future network status data.

3. The distributed data transmission method according to claim 1, characterized in that: The method further comprises: Get current network status data in real time; Adjusting the transmission path according to the current network status data to obtain an adjusted transmission path; The transmitting of each subtask to a corresponding computing node in sequence according to the transmission order and the transmission path includes: According to the transmission order and the adjusted transmission path, each subtask is transmitted to the corresponding computing node in sequence.

4. The distributed data transmission method according to claim 1, characterized in that: The method further comprises: In response to detecting a transmission error of the first subtask, switching a transmission path of the first subtask; The first subtask is retransmitted through the switched transmission path.

5. The distributed data transmission method according to any one of claims 1 to 3, characterized in that: The step of determining the transmission order of each subtask according to the dependency relationship between the subtasks includes: Determining the scheduling priority of each subtask according to the dependency relationship between the subtasks; The transmission order of each subtask is determined by the scheduling priority.

6. The distributed data transmission method according to claim 4, characterized in that: Determining the scheduling priority of each subtask according to the dependency relationship between the subtasks includes: The scheduling priority of each subtask is determined according to the dependency relationship between the subtasks and at least one of the following information: computational complexity, resource requirement, node load, and network bandwidth.

7. A distributed data transmission system, characterized in that: include: The coordination layer is used to divide the distributed training tasks into multiple subtasks; The scheduling layer is used to determine the transmission order of each subtask based on the dependencies between subtasks; Determine the transmission path of each subtask according to the network status data, and any transmission path is used to describe the transmission path for transmitting any subtask to the corresponding computing node; The network layer is used to transmit each subtask to the corresponding computing node in sequence according to the transmission order and the transmission path.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the distributed data transmission method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the distributed data transmission method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the distributed data transmission method as described in any one of claims 1 to 6.

Images