Data transmission methods, apparatus, equipment and media

Abstract

This disclosure provides a data transmission method, apparatus, device, and medium, relating to the field of computer technology, and particularly to chip technology and data transmission technology. The implementation scheme includes: acquiring multiple data transmission requests, each data transmission request including data to be transmitted, a data sending node, and a data receiving node; determining the target transmission length for each data transmission request, the target transmission length indicating the minimum number of nodes between the data sending node and the data receiving node of that data transmission request; dividing the multiple data transmission requests into multiple sets based on the target transmission length of each data transmission request, each set including at least one data transmission request with the same target transmission length; and determining a sequence composed of the multiple sets to sequentially perform data transmission operations on each set, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to chip technology and data transmission technology, specifically to a data transmission method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] With the development of artificial intelligence (AI) technology, more and more applications are achieving results far exceeding those of traditional algorithms. Deep learning is a data-intensive and computationally intensive algorithm. To improve the training and inference speeds of large-scale deep learning models, distributed systems with multiple nodes can be used to perform data processing and meet computing power requirements. The data exchange process within a distributed system directly impacts the efficiency of distributed computing.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] This disclosure provides a data transmission method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0006] According to one aspect of this disclosure, a data transmission method for a distributed system is provided, the distributed system including multiple nodes forming a ring topology or a linear topology, the method comprising: acquiring multiple data transmission requests, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node; for each of the multiple data transmission requests, determining a target transmission length for the data transmission request, the target transmission length indicating the minimum number of nodes between the data sending node and the data receiving node of the data transmission request; based on the target transmission length of each of the multiple data transmission requests, dividing the multiple data transmission requests into multiple sets, wherein at least one data transmission request included in each of the multiple sets has the same target transmission length; and determining a sequence composed of the multiple sets to sequentially perform a data transmission operation for each of the multiple sets, the data transmission operation including: transmitting the data to be transmitted in each data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0007] According to one aspect of this disclosure, a data transmission apparatus for a distributed system is provided, the distributed system including multiple nodes forming a ring topology or a linear topology, the apparatus comprising: an acquisition unit configured to acquire multiple data transmission requests, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node; a determination unit configured to determine, for each of the multiple data transmission requests, a target transmission length for the data transmission request, the target transmission length indicating the minimum number of nodes between the data sending node and the data receiving node of the data transmission request; a partitioning unit configured to partition the multiple data transmission requests into multiple sets based on the target transmission length of each of the multiple data transmission requests, wherein at least one data transmission request included in each of the multiple sets has the same target transmission length; and a transmission unit configured to determine a sequence consisting of the multiple sets to sequentially perform a data transmission operation for each of the multiple sets, the data transmission operation including: transmitting the data to be transmitted in each data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0008] According to one aspect of this disclosure, a chip is provided, including the data transmission device as described above.

[0009] According to one aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data transmission method described above.

[0010] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the above-described data transmission method.

[0011] According to one aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, is capable of implementing the above-described data transmission method.

[0012] According to one or more embodiments of this disclosure, data transmission efficiency can be improved.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0015] Figure 1 A schematic diagram of an exemplary system in which various methods described herein may be implemented, according to exemplary embodiments of the present disclosure;

[0016] Figure 2 A flowchart of a data transmission method according to an exemplary embodiment of the present disclosure is shown;

[0017] Figures 3A-3D A schematic diagram of a data transmission process according to an exemplary embodiment of the present disclosure is shown;

[0018] Figure 4 A structural block diagram of a data transmission apparatus according to an exemplary embodiment of the present disclosure is shown;

[0019] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0022] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0023] In related technologies, corresponding data communication algorithms are usually designed for dedicated networks such as mesh or NVlink, or data exchange in distributed systems is achieved based on worm-eating routing technology, which has a high hardware cost.

[0024] Based on this, this disclosure provides a data transmission method that divides data transmission requests into multiple sets based on the target transmission length. The communication links corresponding to the data transmission requests in each set do not conflict, thereby avoiding link conflicts by executing data transmission sequentially for multiple sets and improving data transmission efficiency by executing data transmission requests in each set in parallel. This maximizes the utilization of transmission links and optimizes data transmission efficiency.

[0025] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0026] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0027] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of data transmission methods.

[0028] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105, and / or 106 under a Software as a Service (SaaS) model.

[0029] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0030] Users can use client devices 101, 102, 103, 104, 105, and / or 106 to send data transmission requests. The client devices can provide an interface that allows users to interact with the client devices. The client devices can also output information to the user through this interface. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0031] Client devices 101, 102, 103, 104, 105, and / or 106 may include various categories of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various categories and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices can run a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), short message service (SMS) applications, and can use various communication protocols.

[0032] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0033] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0034] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0035] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0036] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0037] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different categories. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0038] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be different categories of databases, such as key-value stores, object stores, or regular stores supported by a file system.

[0039] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0040] Figure 2 A flowchart of a data transmission method 200 according to an exemplary embodiment of the present disclosure is shown. Method 200 is applied to a distributed system comprising multiple nodes forming a ring topology or a linear topology. Figure 2 As shown, method 200 includes:

[0041] Step S201: Obtain multiple data transmission requests, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node;

[0042] Step S202: For each of the plurality of data transmission requests, determine the target transmission length of the data transmission request, wherein the target transmission length indicates the minimum number of nodes between the data sending node and the data receiving node of the data transmission request;

[0043] Step S203: Based on the target transmission length of each of the plurality of data transmission requests, divide the plurality of data transmission requests into a plurality of sets, wherein each of the plurality of sets includes at least one data transmission request with the same target transmission length; and

[0044] Step S204: Determine a sequence consisting of the plurality of sets, and perform a data transmission operation sequentially for each of the plurality of sets. The data transmission operation includes: transmitting the data to be transmitted in each data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0045] In a distributed system, data to be transmitted typically originates from the sending node and proceeds sequentially along a specific path to the next hop, ending at the receiving node. By applying the aforementioned data transmission method, multiple data transmission requests in a ring or linear topology can be divided into multiple sets based on the target transmission length. Data transmission requests within each set do not conflict with each hop's communication link. By sequentially executing data transmission for multiple sets, link conflicts between different sets can be effectively avoided. Parallel execution of data transmission requests within each set can significantly improve data transmission efficiency, thereby maximizing the utilization of transmission links and optimizing data transmission performance.

[0046] In some examples, the sequence of the multiple sets can be determined based on the order of the target transmission length from largest to smallest or from smallest to largest, but it is not limited to this, as long as it can realize that the data transmission requests in different sets are not transmitted at the same time.

[0047] In some examples, when the multiple nodes form a linear topology, the shortest communication path between the data sending node and the data receiving node is uniquely determined, thereby making it easy to determine the target transmission length for each data transmission request.

[0048] According to some embodiments, when the plurality of nodes form a ring topology, performing a data transmission operation for each of the plurality of sets includes: in response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determining the data transmission direction of each data transmission request, the data transmission direction including a clockwise direction and a counterclockwise direction; and transmitting the data to be transmitted in the data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request along the data transmission direction. In a distributed system with a ring topology, when the target transmission length of the data transmission request is half a ring, the shortest transmission path between the data sending node and the data receiving node is not unique, that is, it includes shortest transmission paths in both clockwise and counterclockwise transmission directions. By applying the above method, a uniquely determined data transmission path can be obtained by limiting the transmission direction, avoiding link conflicts and ensuring normal data transmission.

[0049] In some examples, data transmission can be performed in a fixed direction (e.g., clockwise or counterclockwise) when the target transmission length of the data transmission request is half the total length of the ring topology, by pre-configuring rules.

[0050] According to some embodiments, determining the data transmission direction of the data transmission request includes: in response to determining that the set includes multiple data transmission requests, dividing at least one data transmission request included in the set into a first subset and a second subset, wherein the data transmission nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; determining that the data transmission direction of at least one data transmission request included in the first subset is clockwise; and determining that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise. Thus, multiple nodes in a ring topology can be divided into two groups based on an interleaving rule, with adjacent nodes transmitting data with a target transmission length of half the total length of the ring topology in clockwise and counterclockwise directions respectively, to more evenly distribute link resources.

[0051] In some examples, multiple nodes can be numbered sequentially, with any node in the ring topology designated as node 1. This allows the direction of data transmission, where the target transmission length is half the total length of the ring topology, to be determined based on the parity of the node numbers. For instance, clockwise transmission could be performed for odd-numbered nodes, and counterclockwise transmission for even-numbered nodes.

[0052] According to some embodiments, when the plurality of nodes form a ring topology, determining the target transmission length for each of the plurality of data transmission requests includes: determining the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a clockwise direction and recording it as a first value; determining the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a counterclockwise direction and recording it as a second value; and determining the smaller of the first value and the second value as the target transmission length. Thus, the number of nodes between the data sending node and the data receiving node can be determined based on both clockwise and counterclockwise directions, allowing for a simple and efficient determination of the target transmission length of the data transmission request based on the minimum value.

[0053] According to some embodiments, the plurality of data transmission requests satisfy the following conditions: for any two nodes among the plurality of nodes, there exists a first data transmission request and a second data transmission request, wherein the data sending node of the first data transmission request is the first node among the two nodes, and the data receiving node of the first data transmission request is the second node among the two nodes; and wherein the data sending node of the second data transmission request is the second node, and the data receiving node of the second data transmission request is the first node. Therefore, the above method can be used to simply and efficiently realize the exchange of data between any two nodes in the network, that is, to implement an efficient full-switching algorithm, optimize data transmission efficiency under high communication complexity, and improve the performance of the distributed system.

[0054] In some examples, the send and receive buffers of each node can be divided into several groups. The j-th group of data in node i's send buffer is sent to node j, and node j places the data block received from node i in the i-th position of its own receive buffer. This achieves full distribution of all data across all nodes, supporting deep learning distributed training tasks such as model parallelism and fully meeting the needs of practical application scenarios.

[0055] In the example above, when the length of data to be sent from each node to multiple other nodes is m, the time cost of the above scheme is T = t. s+m*t w , where t s t is the communication start time. w Let N be the time required to send a message of unit length. Using the full-switching algorithm implemented by method 200 on a bidirectional one-dimensional ring of length N, the first stage of the algorithm requires N / 2*(N / 2-1) / 2 communication sets, while the second stage requires N / 2 communication sets. Therefore, the total communication time complexity is (N... 2 / 8+N / 4)*(t s +m*t w The main term of communication volume is N. 2 / 8*m, thus reaching the theoretical lower limit of communication time complexity, effectively optimizing the full-switching algorithm in ring topology, and improving the data exchange performance of distributed systems.

[0056] Figures 3A-3D A schematic diagram of a data transmission process according to an exemplary embodiment of the present disclosure is shown. In this example, eight nodes in a distributed system form a ring topology, and there is a data transmission request between any two nodes in the system.

[0057] By applying the method 200 described above, multiple data transmission requests can be divided into four sets, with target transmission lengths of 1, 2, 3, and 4, respectively. Executing the data transmission requests in these four sets sequentially corresponds to the following process:

[0058] Step S10: All nodes perform 1-hop communication actions in parallel, processing messages with a target transmission length of 1. Node i sends the message... Pass the message to node i+1. Pass it to node i-1;

[0059] Step S20: All nodes perform the following two-hop communication actions in parallel, processing messages with a target transmission length of 2 in parallel: Step S21: Node i sends the message Pass the message to node i+1. The message is passed to node i-1; Step S22, node i will pass the message... Pass the message to node i+1. Pass it to node i-1.

[0060] Step S30: All nodes execute the following 3-hop communication actions in parallel, processing messages with a target transmission length of 3 in parallel: Step S31: Node i sends the message Pass the message to node i+1. The message is passed to node i-1; Step S32, node i passes the message... Pass the message to node i+1. The message is passed to node i-1; Step S33, node i will pass the message... Pass the message to node i+1. Pass it to node i-1.

[0061] Step S40: All nodes are grouped according to the parity of their indices and perform the following 4-hop communication actions in parallel, processing messages with a transmission distance of 4 in parallel: Step S41: Node 2k sends the message The message is passed to node 2k+1, and node 2k+1 will then forward the message. The message is passed to node 2k; Step S42, node 2k sends the message. The message is passed to node 2k-1, and node 2k-1 will then forward the message. The message is passed to node 2k; Step S43: Node 2k passes the message... The message is passed to node 2k+1, and node 2k+1 will then forward the message. The message is passed to node 2k; Step S44: Node 2k passes the message... The message is passed to node 2k-1, and node 2k-1 will then forward the message. Pass it to node 2k.

[0062] According to one aspect of this disclosure, a data transmission apparatus for a distributed system is also provided, the distributed system comprising multiple nodes that form a ring topology or a linear topology. Figure 4 A structural block diagram of a data transmission apparatus 400 according to an exemplary embodiment of the present disclosure is shown. Figure 4 As shown, the device 400 includes:

[0063] The acquisition unit 401 is configured to acquire multiple data transmission requests, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node;

[0064] The determining unit 402 is configured to determine, for each of the plurality of data transmission requests, a target transmission length for the data transmission request, wherein the target transmission length indicates the minimum number of nodes between the data sending node and the data receiving node of the data transmission request.

[0065] The partitioning unit 403 is configured to partition the plurality of data transmission requests into a plurality of sets based on the target transmission length of each of the plurality of data transmission requests, wherein each of the plurality of sets includes at least one data transmission request with the same target transmission length; and

[0066] Transmission unit 404 is configured to determine a sequence consisting of the plurality of sets to sequentially perform a data transmission operation for each of the plurality of sets, the data transmission operation including: transmitting data to be transmitted in each data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request, wherein the data to be transmitted in at least one data transmission request included in each set is transmitted in parallel.

[0067] According to some embodiments, when the plurality of nodes form a ring topology, the transmission unit 404 includes: a determining subunit configured to, in response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determine a data transmission direction for each of the at least one data transmission request, the data transmission direction including a clockwise direction and a counterclockwise direction; and a transmission subunit configured to transmit the data to be transmitted in the data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request along the data transmission direction.

[0068] According to some embodiments, the determining subunit is configured to: in response to determining that the set includes multiple data transmission requests, divide at least one data transmission request included in the set into a first subset and a second subset, wherein the data transmission nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; determine that the data transmission direction of at least one data transmission request included in the first subset is clockwise; and determine that the data transmission direction of at least one data transmission request included in the second subset is counterclockwise.

[0069] According to some embodiments, when the plurality of nodes form a ring topology, the determining unit 402 is configured to: determine the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a clockwise direction and record it as a first value; determine the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a counterclockwise direction and record it as a second value; and determine the smaller of the first value and the second value as the target transmission length.

[0070] According to some embodiments, the plurality of data transmission requests satisfy the following conditions: for any two of the plurality of nodes, there exists a first data transmission request and a second data transmission request, wherein the data sending node of the first data transmission request is the first node among the two nodes, the data receiving node of the first data transmission request is the second node among the two nodes, and wherein the data sending node of the second data transmission request is the second node, and the data receiving node of the second data transmission request is the first node.

[0071] According to one aspect of this disclosure, a chip is also provided, including the data transmission device described above.

[0072] According to one aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data transmission method described above.

[0073] According to one aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to cause the computer to perform the data transmission method described above.

[0074] According to one aspect of this disclosure, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, implements the above-described data transmission method.

[0075] refer to Figure 5 The present invention describes a structural block diagram of an electronic device 500 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0076] like Figure 5As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0077] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 can be any type of device capable of inputting information to device 500. Input unit 506 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 507 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 508 may include, but is not limited to, a hard disk and an optical disk. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0078] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data transfer methods. For example, in some embodiments, the data transfer method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the data transfer method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the data transfer method by any other suitable means (e.g., by means of firmware).

[0079] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0080] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0082] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0083] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0084] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0085] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0086] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited to these embodiments or examples. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A data transmission method applied to a distributed system, the distributed system comprising multiple nodes forming a ring topology or a linear topology, the method comprising: Obtain multiple data transmission requests in the distributed system, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node; For each of the plurality of data transmission requests, a target transmission length is determined, wherein the target transmission length indicates the minimum number of nodes between the data sending node and the data receiving node of the data transmission request; Based on the target transmission length of each of the plurality of data transmission requests, the plurality of data transmission requests are divided into a plurality of sets, wherein each of the plurality of sets includes at least one data transmission request with the same target transmission length; Determine the sequence consisting of the plurality of sets; and Based on the sequence, data transfer operations are performed for each of the plurality of sets, wherein the data transfer operations for each set include: The data to be transmitted in each data transmission request in the set is transmitted in parallel from the data sending node of the data transmission request to the data receiving node of the data transmission request.

2. The method as described in claim 1, wherein, When the plurality of nodes form a ring topology, performing data transmission operations for each of the plurality of sets includes: In response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, for each of the at least one data transmission request, The data transmission direction of the data transmission request is determined, including clockwise and counterclockwise directions; and The data to be transmitted in the data transmission request is transmitted from the data sending node of the data transmission request to the data receiving node of the data transmission request along the data transmission direction.

3. The method as described in claim 2, wherein, Determining the data transmission direction of the data transmission request includes: In response to determining that the set includes multiple data transmission requests, at least one data transmission request included in the set is divided into a first subset and a second subset, wherein the data transmission nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; It is determined that the data transmission direction of at least one data transmission request included in the first subset is clockwise; and The data transmission direction of at least one data transmission request included in the second subset is determined to be counterclockwise.

4. The method according to any one of claims 1-3, wherein, When the plurality of nodes form a ring topology, determining the target transmission length for each of the plurality of data transmission requests includes: The number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology is determined in a clockwise direction and recorded as the first value; Determine the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a counterclockwise direction, and record it as the second value; and The smaller of the first value and the second value is determined as the target transmission length.

5. The method according to any one of claims 1-3, wherein, The multiple data transmission requests satisfy the following conditions: For any two of the plurality of nodes, there exists a first data transmission request and a second data transmission request, wherein the data sending node of the first data transmission request is the first node among the two nodes, the data receiving node of the first data transmission request is the second node among the two nodes, and wherein the data sending node of the second data transmission request is the second node, and the data receiving node of the second data transmission request is the first node.

6. A data transmission apparatus for a distributed system, the distributed system comprising multiple nodes forming a ring topology or a linear topology, the apparatus comprising: The acquisition unit is configured to acquire multiple data transmission requests in the distributed system, each of the multiple data transmission requests including data to be transmitted, a data sending node, and a data receiving node; The determining unit is configured to determine, for each of the plurality of data transmission requests, a target transmission length for the data transmission request, the target transmission length indicating the minimum number of nodes between the data sending node and the data receiving node of the data transmission request; A partitioning unit is configured to partition the plurality of data transmission requests into a plurality of sets based on the target transmission length of each of the plurality of data transmission requests, wherein each of the plurality of sets includes at least one data transmission request with an equal target transmission length; and The transmission unit is configured as follows: Determine the sequence consisting of the plurality of sets; and Based on the sequence, data transfer operations are performed for each of the plurality of sets, wherein the data transfer operations for each set include: The data to be transmitted in each data transmission request in the set is transmitted in parallel from the data sending node of the data transmission request to the data receiving node of the data transmission request.

7. The apparatus of claim 6, wherein, When the plurality of nodes form a ring topology, the transmission unit includes: A subunit is configured to, in response to determining that the target transmission length of at least one data transmission request included in the set is half the total length of the ring topology, determine, for each of the at least one data transmission requests, the data transmission direction of that data transmission request, said data transmission direction including clockwise and counterclockwise directions; and The transmission subunit is configured to transmit the data to be transmitted in the data transmission request from the data sending node of the data transmission request to the data receiving node of the data transmission request along the data transmission direction.

8. The apparatus of claim 7, wherein, The determining subunit is configured as follows: In response to determining that the set includes multiple data transmission requests, at least one data transmission request included in the set is divided into a first subset and a second subset, wherein the data transmission nodes of any two data transmission requests in the first subset and the second subset are not adjacent nodes; It is determined that the data transmission direction of at least one data transmission request included in the first subset is clockwise; and The data transmission direction of at least one data transmission request included in the second subset is determined to be counterclockwise.

9. The apparatus according to any one of claims 6-8, wherein, When the plurality of nodes form a ring topology, the determining unit is configured as follows: The number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology is determined in a clockwise direction and recorded as the first value; Determine the number of nodes between the data sending node and the data receiving node of the data transmission request in the ring topology in a counterclockwise direction, and record it as the second value; and The smaller of the first value and the second value is determined as the target transmission length.

10. The apparatus according to any one of claims 6-8, wherein, The multiple data transmission requests satisfy the following conditions: For any two of the plurality of nodes, there exists a first data transmission request and a second data transmission request, wherein the data sending node of the first data transmission request is the first node among the two nodes, the data receiving node of the first data transmission request is the second node among the two nodes, and wherein the data sending node of the second data transmission request is the second node, and the data receiving node of the second data transmission request is the first node.

11. A chip comprising the data transmission device according to any one of claims 6-10.

12. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

14. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method according to any one of claims 1-5.

Images