Data transmission method, device, equipment and medium

Abstract

To provide a data transmission method and device, apparatus and medium in the technical field of computers, in particular the technical field of chips and data transmission.SOLUTION: An implementation scheme comprises: acquiring a plurality of data transmission requests, where each data transmission request comprises to-be-transmitted data, a data sending node and a data receiving node; determining a target transmission length of each data transmission request, where the target transmission length indicates a minimum value of the number of nodes between the data sending node and the data receiving node of the data transmission request; based on the target transmission length of each data transmission request, dividing the plurality of data transmission requests into a plurality of sets such that the target transmission lengths of at least one data transmission request included in each set are equal; and determining a sequence formed by the plurality of sets so as to sequentially execute a data transmission operation for each set in the plurality of sets such that the data to be transmitted in the at least one data transmission request included in each set is transmitted in parallel.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of computer technology, in particular to chip technology and data transmission technology, and more particularly to a data transmission method, device, electronic device, computer-readable storage medium and computer program product. [Background technology]

[0002] Artificial intelligence is a field that studies how computers can imitate some human thought processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.), and includes both hardware and software technologies. AI hardware technologies generally include sensors, AI-specific chips, cloud computing, distributed storage, and big data processing, while AI software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0003] With the development of artificial intelligence technology, more and more applications are relying on artificial intelligence technology to achieve results far exceeding those of traditional algorithms. Deep learning is a data- and computation-intensive algorithm. To improve the training and inference speed of large-scale deep learning models, data processing can be performed using a distributed system containing multiple nodes to meet the computing power requirements. The data exchange process in a distributed system directly affects the efficiency of distributed computing.

[0004] The approaches described in this section are not necessarily approaches that have been previously conceived or adopted. Unless otherwise noted, any approach described in this section should not be considered prior art merely because it is included in this section. Likewise, unless otherwise noted, the problems addressed in this section should not be considered to be acknowledged in the prior art. Summary of the Invention

[0005] The present disclosure provides a data transmission method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, there is provided a data transmission method applicable to a distributed system including a plurality of nodes forming a ring topology or a linear topology, the method comprising: receiving a plurality of data transmission requests, each of the plurality of data transmission requests including data to be transmitted, a data transmitting node, and a data receiving node; determining a target transmission length for each of the plurality of data transmission requests, the target transmission length indicating the minimum number of nodes between the data transmitting node of the data transmission request and the data receiving node of the data transmission request; dividing the plurality of data transmission requests into a plurality of sets based on the target transmission lengths of each of the plurality of data transmission requests, wherein at least one data transmission request included in each of the plurality of sets has the same target transmission length; determining a sequence consisting of the plurality of sets, and sequentially performing a data transmission operation for each of the plurality of sets, wherein the data transmission operation includes transmitting data to be transmitted in each data transmission request from the data transmitting node of the data transmission request to the data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0007] According to one aspect of the present disclosure, there is provided a data transmission device applied to a distributed system including a plurality of nodes forming a ring topology or a linear topology, the data transmission device including: an acquisition unit configured to acquire a plurality of data transmission requests, each of the plurality of data transmission requests including data to be transmitted, a data transmitting node, and a data receiving node; a determination unit configured to determine a target transmission length for each of the plurality of data transmission requests, the target transmission length indicating the minimum number of nodes between the data transmitting node of the data transmission request and the data receiving node of the data transmission request; and a determination unit configured to determine the target transmission length for each of the plurality of data transmission requests. a dividing unit configured to divide the plurality of data transmission requests into a plurality of sets based on the above, wherein at least one data transmission request included in each of the plurality of sets has an equal target transmission length; and a transmitting unit configured to determine a sequence consisting of the plurality of sets and sequentially perform a data transmission operation on each of the plurality of sets, wherein the data transmission operation includes transmitting data to be transmitted in each data transmission request from a data transmitting node of the data transmission request to a data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0008] According to one aspect of the present disclosure, there is provided a chip including the data transmission device described above.

[0009] According to one aspect of the present disclosure, there is provided an electronic device including at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to perform the above-described data transmission method.

[0010] According to one aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause the computer to perform the data transmission method.

[0011] According to one aspect of the present disclosure, there is provided a computer program product including a computer program, which, when executed by a processor, is capable of implementing the data transmission method described above.

[0012] According to one or more embodiments of the present disclosure, data transmission efficiency may be improved.

[0013] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, and are not intended to limit the scope of protection of the present disclosure. Other features of the present disclosure will be easily understood from the following description. [Brief explanation of the drawings]

[0014] The drawings illustratively illustrate examples, constitute a part of the specification, and together with the written description serve to explain exemplary embodiments of the examples. The illustrated examples are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar, but not necessarily identical, elements. [Figure 1] 1 is a schematic diagram of an example system capable of implementing various methods described herein, according to an example embodiment of the present disclosure. [Figure 2] 1 is a flowchart illustrating a data transmission method according to an exemplary embodiment of the present disclosure. [Figure 3A] FIG. 1 illustrates a data transmission procedure according to an exemplary embodiment of the present disclosure. [Figure 3B] FIG. 1 illustrates a data transmission procedure according to an exemplary embodiment of the present disclosure. [Figure 3C] FIG. 1 illustrates a data transmission procedure according to an exemplary embodiment of the present disclosure. [Figure 3D] FIG. 1 illustrates a data transmission procedure according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is a block diagram illustrating a configuration of a data transmission device according to an exemplary embodiment of the present disclosure. [Figure 5] 1 is a block diagram showing a configuration of an exemplary electronic device for implementing an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015]

[0023] The following description will be made in conjunction with the drawings to illustrate exemplary embodiments of the present disclosure. Various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the following description omits descriptions of known functions and structures.

[0016] In this disclosure, unless otherwise specified, the use of terms such as "first," "second," etc. to describe various elements is not intended to limit the location, timing, or importance of these elements. Such terms are used only to distinguish one element from another. In some instances, a first element and a second element may refer to the same instance of the element, or in some cases, may refer to different instances based on the context.

[0017] The terms used in the description of various examples of the present disclosure are intended only to describe particular examples and are not intended to be limiting. Unless the context clearly indicates otherwise, and unless a specific number of elements is not limited, the element may be one or more. Furthermore, as used in this disclosure, the term "and / or" covers any and all possible combinations of the listed items.

[0018] Related technologies typically involve designing data communication algorithms for private networks such as mesh and NVlink, or realizing data exchange in distributed systems based on wormhole routing technology, but these involve high hardware costs.

[0019] Therefore, the present disclosure provides a data transmission method, which divides data transmission requests into multiple sets based on a target transmission length, and the communication links corresponding to the data transmission requests in each set do not collide, so that link collisions can be avoided by transmitting data to the multiple sets sequentially, and the data transmission requests in each set are executed in parallel to improve data transmission efficiency, thereby making full use of the transmission links and optimizing data transmission efficiency.

[0020] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0021] 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented, according to embodiments of the present disclosure. Referring to FIG. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to run one or more applications.

[0022] In an embodiment of the present disclosure, the server 120 may operate one or more services or software applications capable of implementing data transmission methods.

[0023] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized and virtualized environments. In some embodiments, these services may be provided as web-based or cloud services, for example, provided to users of client devices 101, 102, 103, 104, 105, and / or 106 in a Software as a Service (SaaS) model.

[0024] In the configuration shown in FIG. 1 , server 120 may include one or more assemblies that implement the functionality performed by server 120. These assemblies may include software assemblies, hardware assemblies, or a combination thereof, executable on one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may utilize one or more client applications to interact with server 120 to access services provided by these assemblies. It should be understood that a variety of different system configurations are possible and may differ from system 100. Thus, FIG. 1 is intended to be illustrative of an example system for implementing various methods described herein and is not intended to be limiting.

[0025] A user may use client devices 101, 102, 103, 104, 105, and / or 106 to send a data transmission request. The client devices may provide an interface through which a user of the client device interacts with the client device. The client devices may also output information to the user through the interface. Although only six client devices are shown in FIG. 1, one skilled in the art will understand that the present disclosure can support any number of client devices.

[0026] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computing devices, such as portable handheld devices, general-purpose computers such as personal computers and laptop computers, workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors and other sensing devices, etc. These computing devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., Google Chrome OS), or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include mobile phones, intelligent phones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (e.g., smart glasses) and other devices. Gaming systems may include various handheld gaming devices, Internet-enabled gaming devices, etc. The client device may run a variety of applications, such as Internet-related applications, communication applications (eg, email applications), and short message service (SMS) applications, and may use a variety of communication protocols.

[0027] Network 110 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including, but not limited to, TCP / IP, SNA, IPX, etc. By way of example, one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token loop, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0028] Server 120 may include one or more general-purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframes, server clusters, or any other suitable arrangement and / or combination. Server 120 may also include one or more virtual machines running virtual operating systems or other computing architectures involving virtualization (e.g., one or more flexible pools of virtualized logical storage devices to maintain the server's virtual storage devices). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0029] The computing units in server 120 may run one or more operating systems, including any of the operating systems listed above and any commercial server operating system. Server 120 may also run any one of a variety of additional server and / or middle-tier applications, such as an HTTP server, an FTP server, a CGI server, a JAVA server, a database server, etc.

[0030] In some embodiments, server 120 may include one or more applications for analyzing and consolidating data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0031] In some embodiments, server 120 may be a server in a distributed system or a server incorporating blockchain. Server 120 may be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product in a cloud computing service system that solves the drawbacks of traditional physical hosts and virtual private server (VPS) services, such as high management difficulty and poor business scalability.

[0032] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data or other information. For example, one or more of databases 130 may be used to store information such as audio files or video files. Databases 130 may be located in a variety of locations. For example, a database used by server 120 may be local to server 120 or may be remote from server 120 and in communication with server 120 over a network or dedicated connection. Databases 130 may be of different types. In some embodiments, a database used by server 120 may be a relational database. One or more of these databases may store, update, and retrieve data from the databases in response to instructions.

[0033] In some embodiments, one or more of databases 130 may be used by an application to store data for the application. The databases used by the application may be different types of databases, such as key-value repositories, object repositories, general-purpose repositories backed by a file system, etc.

[0034] The system 100 of FIG. 1 can be configured and operated in a variety of ways to accommodate the various methods and apparatus described in accordance with this disclosure.

[0035] 2 is a flowchart of a data transmission method 200 applied to a distributed system including a plurality of nodes, the plurality of nodes forming a ring topology or a linear topology, according to an exemplary embodiment of the present disclosure. As shown in FIG. 2, the method 200 includes: Step S201: obtaining a plurality of data transmission requests, each of the plurality of data transmission requests including data to be transmitted, a data transmitting node, and a data receiving node; Step S202: for each of the plurality of data transmission requests, determine a target transmission length of the data transmission request, the target transmission length indicating the minimum number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request; Step S203: Dividing the plurality of data transmission requests into a plurality of sets according to a target transmission length of each data transmission request of the plurality of data transmission requests, wherein the target transmission length of at least one data transmission request included in each of the plurality of sets is equal; and step S204 of determining a sequence consisting of the plurality of sets and sequentially performing a data transmission operation for each of the plurality of sets, the data transmission operation including transmitting data to be transmitted in each data transmission request from a data transmitting node of the data transmission request to a data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0036] Generally speaking, data to be transmitted in a distributed system is transmitted sequentially from a data transmitting node along a specific path to a data receiving node, and then to a node corresponding to the next hop. By applying the above data transmission method, multiple data transmission requests in a ring topology or a linear topology can be divided into multiple sets according to a target transmission length, and the data transmission requests in each set do not collide with each other on the communication links occupied by each hop. By sequentially transmitting data to multiple sets, link collisions between data transmission operations in different sets can be effectively avoided. By concurrently executing the data transmission requests in each set, data transmission efficiency can be effectively improved, and the transmission links can be fully utilized to optimize data transmission efficiency.

[0037] In some examples, a sequence of multiple sets can be determined in order of target transmission length from largest to smallest or smallest to largest, but is not limited to this as long as non-simultaneous transmission of data transmission requests in different sets can be achieved.

[0038] In some examples, when multiple nodes form a linear topology, the shortest communication path between a data sending node and a data receiving node is uniquely determined, which makes it possible to conveniently determine a target transmission length for each data transmission request.

[0039] According to some embodiments, when the plurality of nodes form a ring topology, performing a data transmission operation for each of the plurality of sets includes: in response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determining, for each data transmission request of the at least one data transmission request, a data transmission direction of the data transmission request, the data transmission direction including a clockwise direction and a counterclockwise direction; and transmitting the data to be transmitted in the data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request along the data transmission direction. In a distributed system with a ring topology, when the target transmission length of a data transmission request is a half ring, the shortest transmission path between the data sending node and the data receiving node is not unique, i.e., it includes shortest transmission paths in two transmission directions, clockwise and counterclockwise. By applying the above method, a uniquely determined data transmission path can be obtained by restricting the transmission direction, thereby avoiding link collisions and ensuring normal data transmission.

[0040] In some examples, a pre-configured rule may limit data transmission to a fixed direction (e.g., clockwise or counterclockwise) if the target transmission length of a data transmission request is half the total length of the ring topology.

[0041] According to some embodiments, determining the data transmission direction of the data transmission request includes, in response to determining that the set includes a plurality of data transmission requests, dividing at least one data transmission request included in the set into a first subset and a second subset, determining that data sending nodes of any two data transmission requests of the first subset and the second subset are adjacent nodes, determining that the data transmission direction of at least one data transmission request included in the first subset is clockwise, and determining that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise, thereby dividing the plurality of nodes in the ring topology into two groups according to an interleaving rule, and adjacent nodes respectively transmit data in clockwise and counterclockwise directions, with a target transmission length of half the total length of the ring topology, thereby more uniformly allocating link resources.

[0042] In some examples, the nodes in the ring topology may be numbered sequentially, with any node being designated as node 1, and the parity of the node numbers may be used to determine the data transmission direction, with the target transmission length being half the total length of the ring topology. For example, odd-numbered nodes may transmit clockwise, and even-numbered nodes may transmit counterclockwise.

[0043] According to some embodiments, when the plurality of nodes form a ring topology, determining a target transmission length for each of the plurality of data transmission requests includes determining a number of nodes between a data sending node of the data transmission request and a data receiving node of the data transmission request in the ring topology along a clockwise direction, the number being denoted as a first value, determining a number of nodes between the data sending node of the data transmission request and a data receiving node of the data transmission request in the ring topology along a counterclockwise direction, the number being denoted as a second value, and determining a smaller value of the first value and the second value as the target transmission length. In this way, the number of nodes between the data sending node and the data receiving node can be determined based on each of the clockwise and counterclockwise directions, and the target transmission length of the data transmission request can be simply and efficiently determined based on the minimum value.

[0044] According to some embodiments, the plurality of data transmission requests include a first data transmission request and a second data transmission request for any two nodes among the plurality of nodes, where a data sending node of the first data transmission request is the first node of the two nodes, a data receiving node of the first data transmission request is the second node of the two nodes, a data sending node of the second data transmission request is the second node, and a data receiving node of the second data transmission request is the first node. Thus, by using the above method, data exchange between any two nodes in a network can be realized simply and efficiently, i.e., an efficient full exchange algorithm can be realized, data transmission efficiency can be optimized when communication complexity is high, and performance of a distributed system can be improved.

[0045] In some cases, the send buffer and receive buffer of each node are divided into multiple groups, and the jth group of data in the send buffer of node i is sent to node j, and node j places the received data block from node i in the ith position of its receive buffer. This achieves full distribution of all data across all nodes, supporting deep learning distributed training tasks such as model parallelization, and fully meeting the needs of practical application scenarios.

[0046] In the above example, if the length of data to be transmitted from each node to other multiple nodes is m, the time overhead for the above plan is (outside 1) TIFF0007761194000001.tif7119, where (outside 2) TIFF0007761194000002.tif6119 is the start time of communication, (Outside 3) TIFF0007761194000003.tif6119 is the time to transmit a unit length of a message. The entire exchange algorithm realized by applying the above method 200 is on a bidirectional one-dimensional ring of length N, and the first stage of the algorithm is (outside 4) TIFF0007761194000004.tif7119 group communication needs to be performed, and the second stage of the algorithm is (outside 5) TIFF0007761194000005.tif6119 group communication must be performed, and therefore the total communication time complexity is (outside 6) TIFF0007761194000006.tif8119, and the main term of the communication volume is (outside 7) TIFF0007761194000007.tif7119, which can reach the theoretical lower bound of communication time complexity, effectively optimize the entire switching algorithm in the ring topology, and improve the data switching performance of the distributed system.

[0047] 3A-3D are diagrams illustrating a data transmission procedure according to an exemplary embodiment of the present disclosure. In this example, eight nodes in a distributed system form a ring topology, and a data transmission request exists between any two nodes in the system.

[0048] By applying the above method 200, multiple data transmission requests can be divided into four sets, i.e., the target transmission lengths are 1, 2, 3, and 4, respectively. The data transmission requests in the four sets are executed in order, i.e., corresponding to the following process:

[0049] Step S10: All nodes perform one-hop communication operations in parallel to process messages with a target transmission length of 1. Node i receives the message (outside 8) TIFF0007761194000008.tif7119 to node (outer 9) TIFF0007761194000009.tif7119 and message (Outside 10) TIFF0007761194000010.tif8119 to node (Outside 11) Transmit to TIFF0007761194000011.tif6119.

[0050] Step S20, all nodes execute the following two-hop communication operations in parallel to process messages with a target transmission length of 2; step S21, node i processes messages (Outside 12) TIFF0007761194000012.tif8119 to node (Outside 13) TIFF0007761194000013.tif6119, message (Outside 14) TIFF0007761194000014.tif9119 to node (Outside 15) TIFF0007761194000015.tif5119; step S22, node i transmits the message (Outside 16) TIFF0007761194000016.tif8119 to node (Outside 17) TIFF0007761194000017.tif6116 and message (Outside 18) TIFF0007761194000018.tif8116 to node (Outside 19) Transmit to TIFF0007761194000019.tif5116.

[0051] Step S30, all nodes execute the following three-hop communication operations in parallel to process messages with a target transmission length of 3: Step S31, node i sends message (outside 20) TIFF0007761194000020.tif9116 to node (outside 21) TIFF0007761194000021.tif6116 and message (outside 22) TIFF0007761194000022.tif8116 to node (outside 23) TIFF0007761194000023.tif6116; step S32, node i transmits the message (outside 24) TIFF0007761194000024.tif9116 to node (Outside 25) TIFF0007761194000025.tif6116 and message (outside 26) TIFF0007761194000026.tif8116 to node (outside 27) TIFF0007761194000027.tif6116; step S33, node i transmits the message (outside 28) TIFF0007761194000028.tif9116 to node (outside 29) TIFF0007761194000029.tif6116 and message (Outside 30) TIFF0007761194000030.tif8116 to node (Outside 31) Transmit to TIFF0007761194000031.tif6116.

[0052] Step S40, all nodes are grouped according to the subscript parity, and execute the following four-hop communication operations in parallel to process messages with a transmission distance of 4 in parallel; step S41, node 2k sends message (Outside 32) TIFF0007761194000032.tif8116 to node (Outside 33) TIFF0007761194000033.tif6116 and the node (Outside 34) TIFF0007761194000034.tif6114 is a message (Outside 35) TIFF0007761194000035.tif8114 is transmitted to node 2k; step S42, node 2k receives the message (Outside 36) TIFF0007761194000036.tif9114 to node (Outside 37) Transmit to TIFF0007761194000037.tif6114, node (Outside 38) TIFF0007761194000038.tif6114 is a message (Outside 39) TIFF0007761194000039.tif9114 is transmitted to node 2k; step S43, node 2k receives the message (outside 40) TIFF0007761194000040.tif8114 to node (outside 41) TIFF0007761194000041.tif6114, node (outside 42) TIFF0007761194000042.tif6114 is a message (outside 43) TIFF0007761194000043.tif8114 is transmitted to node 2k; in step S44, node 2k receives the message (outside 44) TIFF0007761194000044.tif8116 to node (outside 45) TIFF0007761194000045.tif7116 and the node (outside 46) TIFF0007761194000046.tif6116 is the message (outside 47) Transmit TIFF0007761194000047.tif9116 to node 2k.

[0053] According to one aspect of the present disclosure, there is further provided a data transmission device applicable to a distributed system including a plurality of nodes forming a ring topology or a linear topology. Fig. 4 is a block diagram showing the configuration of a data transmission device 400 according to an exemplary embodiment of the present disclosure. As shown in Fig. 4, the device 400 includes: an acquiring unit 401 configured to acquire a plurality of data transmission requests, each of the plurality of data transmission requests including data to be transmitted, a data sending node, and a data receiving node; a determining unit 402 configured to, for each data transmission request of the plurality of data transmission requests, determine a target transmission length of the data transmission request, wherein the target transmission length indicates a minimum number of nodes between a data sending node of the data transmission request and a data receiving node of the data transmission request; a dividing unit 403 configured to divide the plurality of data transmission requests into a plurality of sets according to a target transmission length of each data transmission request of the plurality of data transmission requests, wherein the target transmission length of at least one data transmission request included in each of the plurality of sets is equal; and a unit 404 for determining a sequence consisting of the plurality of sets and sequentially performing a data transmission operation for each of the plurality of sets, the data transmission operation including transmitting data to be transmitted in each data transmission request from a data sending node of the data transmission request to a data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0054] According to some embodiments, when the plurality of nodes form a ring topology, the transmission unit 404 includes: a determination subunit configured to, in response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determine, for each of the at least one data transmission request, a data transmission direction of the data transmission request, the data transmission direction including a clockwise direction and a counterclockwise direction; and a transmission subunit configured to transmit data to be transmitted in the data transmission request from a data sending node of the data transmission request to a data receiving node of the data transmission request along the data transmission direction.

[0055] According to some embodiments, in response to determining that the set includes a plurality of data transmission requests, the determination subunit is configured to divide at least one data transmission request included in the set into a first subset and a second subset, determine that the data sending nodes of any two data transmission requests of the first subset and the second subset are not adjacent nodes, determine that the data transmission direction of at least one data transmission request included in the first subset is clockwise, and determine that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise.

[0056] According to some embodiments, when the plurality of nodes form a ring topology, the determination unit 402 is configured to determine the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a clockwise direction, and denote this as a first value; determine the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a counterclockwise direction, and denote this as a second value; and determine the smaller value of the first value and the second value as the target transmission length.

[0057] According to some embodiments, the plurality of data transmission requests include a first data transmission request and a second data transmission request for any two nodes among the plurality of nodes, where the data sending node of the first data transmission request is the first node of the two nodes, the data receiving node of the first data transmission request is the second node of the two nodes, the data sending node of the second data transmission request is the second node, and the data receiving node of the second data transmission request is the first node.

[0058] According to one aspect of the present disclosure, there is further provided a chip including the data transmission device described above.

[0059] According to one aspect of the present disclosure, there is further provided an electronic device including at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to perform the data transmission method described above.

[0060] According to one aspect of the present disclosure, there is further provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause the computer to perform the data transmission method described above.

[0061] According to one aspect of the present disclosure, there is also provided a computer program product including a computer program which, when executed by a processor, implements the data transmission method described above.

[0062] Next, referring to FIG. 5 , a block diagram of an electronic device 500 functioning as a server or client of the present disclosure will be described, which is an example of a hardware device applicable to each aspect of the present disclosure. The electronic device may represent various forms of digital electronic computers, such as laptop computers, desktop computers, stage computers, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing devices, mobile phones, intelligent phones, wearable devices, and other similar computing devices. The components, their connections, and their functions shown herein are merely exemplary and do not limit the implementation of the present disclosure as described and / or claimed herein.

[0063] 5, the device 500 includes a computing unit 501, which can perform various appropriate operations and processes according to a computer program stored in a read-only memory (ROM) 502 or loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may store various programs and data required to operate the device 500. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0064] The components within device 500 are connected to I / O interface 505, which includes input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 may be any type of device capable of inputting information into device 500. Input unit 506 can receive input numeric or character information and generate key signal input for user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackboard, trackball, joystick, microphone, and / or remote control. Output unit 507 may be any type of device capable of presenting information, and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 508 may include, but is not limited to, a magnetic disk, an optical disk, etc. The communication unit 509 enables the device 500 to exchange information / data with other devices via a computer network, e.g., the Internet, and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, e.g., a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0065] The computing unit 501 may be any of a variety of general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 may include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs each of the methods and processes described above, such as the data transmission method. For example, in some embodiments, the data transmission method may be embodied as a computer software program tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, it may perform one or more steps of the data transmission method described above. Alternatively, in other embodiments, the computing unit 501 may be configured (eg, by firmware) to perform the data transmission method in any other suitable manner.

[0066] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being embodied in one or more computer programs that may be executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor, and may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0067] Program code implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that when executed by the processor or controller, the program code performs the functions / operations specified in the flowcharts and / or block diagrams. The program code may be entirely executed by machine, partially executed by machine, partially executed by machine and partially executed by a remote machine as a separate software package, or entirely executed on a remote machine or server.

[0068] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may include or store a program for use in or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection with one or more leads, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0069] To provide for user interaction, a computer may implement the systems and techniques described herein and include a display device (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user may provide input to the computer. Other types of devices may also be used to provide for user interaction, for example, providing feedback to a user in any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and receiving input from a user in any form (including sound input, speech input, or tactile input).

[0070] The systems and techniques described herein may be implemented in a computing system including backstage components (e.g., as a data server), middleware components (e.g., as an application server), front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with the system or technique implementation), or any combination of backstage components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0071] The computer system may include a client and a server. The client and the server are generally remote from each other and usually interact via a communication network. The client-server relationship is created by running computer programs on corresponding computers. The server may be a cloud server, a server in a distributed system, or a server combined with a blockchain.

[0072] It should be understood that the various forms of flow described above may be used to rearrange the order, add or remove steps, etc. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, as long as the technical solutions disclosed in this disclosure can achieve the desired results, and the present disclosure is not limited thereto.

[0073] Although embodiments or examples of the present disclosure have been described with reference to the drawings, it should be understood that the above-described methods, systems, and devices are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples. Various elements of the embodiments or examples may be omitted or replaced by equivalent elements. Furthermore, steps may be performed in an order different from that described in this disclosure. Furthermore, various elements of the embodiments or examples may be combined in various ways. It is important to note that as technology evolves, many elements described herein may be replaced by equivalent elements that appear later in this disclosure.

Claims

1. A data transmission method applied to a distributed system including a plurality of nodes forming a ring topology or a linear topology, the method comprising: Obtaining a plurality of data transmission requests, each of the plurality of data transmission requests including data to be transmitted, a data sending node, and a data receiving node; For each of the plurality of data transmission requests, determine a target transmission length of the data transmission request, wherein the target transmission length indicates a minimum number of nodes between a data sending node of the data transmission request and a data receiving node of the data transmission request; Dividing the plurality of data transmission requests into a plurality of sets based on a target transmission length of each data transmission request of the plurality of data transmission requests, wherein the target transmission length of at least one data transmission request included in each of the plurality of sets is equal; determining a sequence consisting of the plurality of sets, and sequentially executing a data transmission operation for each of the plurality of sets, wherein the data transmission operation includes transmitting data to be transmitted in each data transmission request from a data transmission node of the data transmission request to a data reception node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel; When the plurality of nodes form a ring topology, performing a data transmission operation on each of the plurality of sets includes: In response to determining that a target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, for each data transmission request of the at least one data transmission request: determining a data transmission direction of the data transmission request, the data transmission direction including a clockwise direction and a counterclockwise direction; transmitting data to be transmitted in the data transmission request from a data transmitting node of the data transmission request to a data receiving node of the data transmission request along the data transmission direction; determining the data transmission direction of the data transmission request includes: In response to determining that the set includes a plurality of data transmission requests, dividing at least one data transmission request included in the set into a first subset and a second subset, and data sending nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; determining that the data transmission direction of at least one data transmission request included in the first subset is clockwise; determining that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise.

2. When the plurality of nodes form a ring topology, determining a target transmission length of each of the plurality of data transmission requests includes: determining the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a clockwise direction, and denoting the number as a first value; determining the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a counterclockwise direction, and denoting the number as a second value; and establishing the smaller of the first and second values ​​as the target transmission length.

3. The plurality of data transmission requests:

2. The method of claim 1, wherein a first data transmission request and a second data transmission request exist for any two nodes among the plurality of nodes, and wherein the following conditions are satisfied: a data sending node of the first data transmission request is a first node of the two nodes, a data receiving node of the first data transmission request is a second node of the two nodes, a data sending node of the second data transmission request is the second node, and a data receiving node of the second data transmission request is the first node.

4. A data transmission device applied to a distributed system including a plurality of nodes forming a ring topology or a linear topology, the device comprising: an acquiring unit configured to acquire a plurality of data transmission requests, each data transmission request of the plurality of data transmission requests including data to be transmitted, a data sending node, and a data receiving node; a determination unit configured to determine, for each data transmission request of the plurality of data transmission requests, a target transmission length of the data transmission request, wherein the target transmission length indicates a minimum number of nodes between a data sending node of the data transmission request and a data receiving node of the data transmission request; a dividing unit configured to divide the plurality of data transmission requests into a plurality of sets according to a target transmission length of each data transmission request of the plurality of data transmission requests, wherein the target transmission length of at least one data transmission request included in each of the plurality of sets is equal; a transmission unit configured to determine a sequence consisting of the plurality of sets and sequentially perform a data transmission operation for each of the plurality of sets, the data transmission operation including transmitting data to be transmitted in each data transmission request from a data transmission node of the data transmission request to a data reception node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel; When the plurality of nodes form a ring topology, the transmission unit a determining subunit configured to, in response to determining that a target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determine, for each data transmission request of the at least one data transmission request, a data transmission direction of the data transmission request, wherein the data transmission direction includes a clockwise direction and a counterclockwise direction; a transmission subunit configured to transmit data to be transmitted in the data transmission request from a data sending node of the data transmission request to a data receiving node of the data transmission request along the data transmission direction; The defined subunit is In response to determining that the set includes a plurality of data transmission requests, divide at least one data transmission request included in the set into a first subset and a second subset, and data transmission nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; determining that the data transmission direction of at least one data transmission request included in the first subset is clockwise; The data transmission device applied to a distributed system is configured to determine that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise.

5. When the plurality of nodes form a ring topology, the determination unit: Determine the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a clockwise direction, and denote this as a first value; Determine the number of nodes between the data sending node of the data transmission request and the data receiving node of the data transmission request in the ring topology along a counterclockwise direction, and denote this as a second value; The apparatus of claim 4 , configured to establish the smaller of the first and second values ​​as the target transmission length.

6. The plurality of data transmission requests:

5. The device of claim 4, wherein a first data transmission request and a second data transmission request exist for any two nodes among the plurality of nodes, and wherein the following conditions are satisfied: a data sending node of the first data transmission request is a first node of the two nodes, a data receiving node of the first data transmission request is a second node of the two nodes, a data sending node of the second data transmission request is the second node, and a data receiving node of the second data transmission request is the first node.

7. A chip comprising a data transmission device according to any one of claims 4 to 6.

8. An electronic device, at least one processor; and a memory communicatively coupled to the at least one processor, wherein: An electronic device, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the at least one processor to perform the method of any one of claims 1 to 3.

9. A non-transitory computer readable storage medium having stored thereon computer instructions, the computer instructions being used to cause a computer to perform the method of any one of claims 1 to 3.

10. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 3.

Images