Distributed system, data transmission method and device and storage medium

Abstract

The invention provides a distributed system, a data transmission method and device, and a storage medium, and the system comprises the steps: each cross-regional switch cluster communicates with a plurality of transverse extension track switches, and each transverse extension track switch communicates with a plurality of fusion extension switches, each fusion extension switch communicates with a plurality of processing nodes corresponding to at least one processing node server in the plurality of processing node servers; and each fusion extension switch is used for transmitting a data packet sent by each processing node in the at least one processing node server to at least one processing node in the at least one processing node server. According to the embodiment of the invention, the data packet of a certain processing node in each fusion extension switch is transmitted to other processing nodes of the fusion extension switch at the same time, so that the data packets of the processing nodes of the distributed system are transmitted to other processing nodes of the distributed system at the same time; therefore, the resources of each processing node in the distributed system are fully utilized.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a distributed system, a data transmission method, a device and a storage medium. Background Art

[0002] When the training model is relatively large or the amount of data in the training data set is relatively large, it will bring greater training pressure to the training equipment, causing the training process to be lengthy and affecting the application of the model.

[0003] In the face of the above situation, a distributed training network can generally be used to train the above training model, that is, the training tasks of the training model are distributed to different processing nodes of the distributed training network, and the training process is completed by transmitting data between different processing nodes. Therefore, the distributed training network can significantly accelerate the training process of the model. In addition, the distributed training network can process large-scale data sets and complex models that exceed the memory limit of a single computing node by dividing the data and model of the training set and distributing them on different processing nodes, and then transmitting data between different processing nodes to complete the training process.

[0004] In addition to the model training network, the intelligent computing center also needs to be equipped with a model inference network so that the real-time data in the real environment can be used to obtain the final application results through the trained model. However, the resource utilization rate of the intelligent computing center network in related technologies is not high, resulting in a waste of resources. Summary of the invention

[0005] The present application proposes a distributed system, a data transmission method, a device and a storage medium, which can solve the technical problem of low resource utilization rate and waste of resources in the intelligent computing center network in the related technology.

[0006] The first embodiment of the present application provides a distributed system, including:

[0007] at least one cross-zone switch cluster, a plurality of horizontal expansion track switches, a plurality of converged expansion switches, and a plurality of processing node servers;

[0008] Each processing node server includes a plurality of processing nodes;

[0009] Each cross-zone switch cluster communicates with the multiple horizontal expansion track switches respectively, each horizontal expansion track switch communicates with the multiple converged expansion switches respectively, and each converged expansion switch communicates with multiple processing nodes corresponding to at least one processing node server among the multiple processing node servers;

[0010] Each of the fusion extension switches is used to send data packets sent by each processing node in the at least one processing node server to at least one processing node in the at least one processing node server.

[0011] The second aspect of the present application provides a data transmission method.

[0012] Any switch unit in the first converged extension switch in the distributed system, wherein the first converged extension switch is any converged extension switch of multiple converged extension switches in the distributed system, and the method includes:

[0013] Receive data packets;

[0014] Determining the data packet type of the data packet;

[0015] If the data packet is a training data packet, sending the data packet to the connected processing node;

[0016] If the data packet is an inference data packet, confirming that the inference task label matches the inference task of the connected processing node;

[0017] If not, discard the data packet and issue an alarm;

[0018] If it matches, remove the inference task label and send the data packet to the connected processing node

[0019] The second data packet is transmitted to at least one processing node in at least one processing node server corresponding to the first converged extension switch.

[0020] An embodiment of the third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the second aspect above.

[0021] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the method described in the second aspect above.

[0022] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0023] The present application proposes a distributed system, a data transmission method, a device and a storage medium, the system comprising: at least one cross-zone switch cluster, a plurality of horizontally extended track switches, a plurality of fusion extended switches and a plurality of processing node servers; each processing node server comprises a plurality of processing nodes; each cross-zone switch cluster communicates with a plurality of horizontally extended track switches respectively, each horizontally extended track switch communicates with a plurality of fusion extended switches respectively, each fusion extended switch communicates with a plurality of processing nodes respectively corresponding to at least one processing node server in a plurality of processing node servers; each fusion extended switch is used to transmit a data packet sent by each processing node in at least one processing node server to at least one processing node in at least one processing node server. The embodiment of the present application transmits a data packet of a processing node in each fusion extended switch to other processing nodes of the fusion extended switch at the same time, so as to realize the simultaneous transmission of data packets of processing nodes of the distributed system to other processing nodes of the distributed system, so as to make full use of the resources of each processing node in the distributed system.

[0024] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0026] In the attached picture:

[0027] Figure 1 A schematic diagram of the structure of a distributed system provided by the related art is shown;

[0028] Figure 2 A schematic diagram showing the structure of another distributed system provided by the related art is shown;

[0029] Figure 3 A structural diagram of another distributed system provided by the related art is shown;

[0030] Figure 4 A schematic diagram of the structure of a distributed system provided by an embodiment of the present application is shown;

[0031] Figure 5 A schematic diagram of the structure of a converged extension switch provided by an embodiment of the present application is shown;

[0032] Figure 6A flow chart of a data transmission method provided by an embodiment of the present application is shown;

[0033] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0034] Figure 8 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0036] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0037] Based on the above background technology, the current distributed system such as Figure 1 As shown in the figure, each server in a distributed training network area of ​​an intelligent computing center will have multiple parallel processing nodes, and also includes a built-in high-bandwidth switch chip to interconnect the processing nodes of the same server. The industry generally refers to this as a vertical expansion network. In addition, there are multiple track switches in each area, and each track switch can connect the processing nodes of the same track in different servers. For example, the track 1 switch is used to connect the first processing node in each server, and the track 2 switch is used to connect the second processing node, and so on. These track switches form what the industry calls a horizontal expansion network. If you need to build a larger training network, you can interconnect multiple areas through a cross-region switch cluster on the next level. The main function of the vertical expansion network plus the horizontal network is to transmit the parameters, number of layers, or tensors of each processing node model in the distributed intelligent computing training process.

[0038] Usually, the training network transmission process is to first transmit the vertical expansion network, and then transmit the horizontal expansion network. Therefore, the resources of the vertical expansion network and the horizontal expansion network are not fully utilized.

[0039] Another distributed system such as Figure 2 As shown, Figure 1The only difference is that the vertical expansion network is no longer a high-bandwidth switch chip built into each GPU server, but multiple (for example, M) built-in switch chips are stacked together to form an 8*M vertical expansion network to accelerate the transmission of the vertical expansion network between 8*M GPUs.

[0040] However, in the early stages of intelligent computing, there will be a variety of new application models that need to be established and improved. However, as the application models are gradually established and improved, the inference process will not involve the resources in the training process, and the utilization rate of the training network resources will gradually decrease, resulting in a waste of resources.

[0041] Another distributed system such as Figure 3 As shown in the figure, after the intelligent computing model training is completed, for the sake of versatility, applicability and reasonable use of computing resources, the model will usually go through a streamlined process and then use the streamlined model to infer the real scene. Therefore, compared with the training network, the GPU server in the inference network is usually relatively low-end, so each GPU server will no longer have a built-in high-bandwidth switch chip, but only the PCIE bus that appears in the traditional CPU server. Therefore, the interconnection of GPUs in the inference network can only be connected through a full-mesh network composed of TOR (Top of Rack) switches and Spine switches. Similarly, if you need to build a larger inference network, you can interconnect multiple regions through the cross-region switch cluster on the next level.

[0042] When the application model is gradually established and improved, the real application will begin to be fully launched. Therefore, the resources of the inference network will grow rapidly, far exceeding the current scale and may cause the current inference network to be demolished and rebuilt, which will also result in a waste of resources.

[0043] In order to solve the above technical problems, the present application proposes a distributed system, a data transmission method, a device and a storage medium, the system comprising: at least one cross-zone switch cluster, a plurality of horizontally extended track switches, a plurality of fusion extension switches and a plurality of processing node servers; each processing node server comprises a plurality of processing nodes; each cross-zone switch cluster communicates with a plurality of horizontally extended track switches respectively, each horizontally extended track switch communicates with a plurality of fusion extension switches respectively, each fusion extension switch communicates with a plurality of processing nodes corresponding to at least one processing node server in a plurality of processing node servers; each fusion extension switch is used to transmit a data packet sent by each processing node in at least one processing node server to at least one processing node in at least one processing node server. The embodiment of the present application transmits a data packet of a processing node in each fusion extension switch to other processing nodes of the fusion extension switch at the same time, so as to realize the simultaneous transmission of data packets of processing nodes of the distributed system to other processing nodes of the distributed system, so as to make full use of the resources of each processing node in the distributed system.

[0044] A distributed system proposed according to an embodiment of the present application is described below in conjunction with the accompanying drawings. The present application takes a distributed system including a cross-region switch cluster as an example to illustrate the distributed system.

[0045] like Figure 4 As shown, the distributed system includes: a cross-zone switch cluster, a plurality of horizontally expanded track switches, a plurality of converged expansion switches and a plurality of processing node servers;

[0046] Each processing node server includes a plurality of processing nodes;

[0047] Each cross-zone switch cluster communicates with a plurality of horizontal expansion track switches respectively, each horizontal expansion track switch communicates with a plurality of converged expansion switches respectively, and each converged expansion switch communicates with a plurality of processing nodes corresponding to at least one processing node server among a plurality of processing node servers;

[0048] Each fusion extension switch is used to transmit a data packet sent by each processing node in at least one processing node server to at least one processing node in at least one processing node server.

[0049] Among them, the processing node can be a graphics processing unit (GPU), a central processing unit (CPU), a tensor processing unit (TPU), etc.

[0050] It is understandable that in the early stage of intelligent computing, there is little training task data. The system includes a horizontal expansion track switch, a fusion expansion switch, a small number of processing node servers, and the number of processing nodes in the processing node servers is also relatively small. However, as the training task data increases, the number of processing nodes also increases accordingly. Each processing node server will have a processing node number threshold. When the number of processing nodes exceeds the processing node number threshold, a new processing node server will be set up, and the new processing node server will also communicate with the above-mentioned fusion expansion switch, but the communication capacity of the fusion expansion switch is limited. Therefore, each fusion expansion switch will have a corresponding processing node server number threshold. When the number of processing node servers exceeds the processing node number threshold, a new fusion expansion switch will be set up to communicate with the newly added processing node server.

[0051] As the number of tasks increases, the final system structure is as follows Figure 4 shown.

[0052] It can be understood that since each fusion extension switch communicates with multiple processing nodes corresponding to at least one processing node server among multiple processing node servers, that is, each fusion extension switch communicates with multiple processing nodes corresponding to at least one corresponding processing node server, each fusion extension switch can receive a data packet sent by each processing node and transmit the data packet to at least one processing node in at least one processing node server.

[0053] Therefore, the distributed system proposed in the embodiment of the present application can simultaneously transmit the data packets of a processing node in each fusion extension switch to other processing nodes of the fusion extension switch, so as to simultaneously transmit the data packets of the processing nodes of the distributed system to other processing nodes of the distributed system, so as to fully utilize the resources of each processing node in the distributed system. Figure 1 The corresponding distributed system in the intelligent computing training network first expands the network transmission vertically, and then expands the network transmission horizontally. The inability to transmit at the same time results in the resources of the vertically expanded network or the horizontally expanded network not being fully utilized.

[0054] The present application proposes a distributed system, a data transmission method, a device and a storage medium, the system comprising: each cross-zone switch cluster communicates with multiple horizontal expansion track switches respectively, each horizontal expansion track switch communicates with multiple fusion expansion switches respectively, each fusion expansion switch communicates with multiple processing nodes corresponding to at least one processing node server in multiple processing node servers; each fusion expansion switch is used to transmit a data packet sent by each processing node in at least one processing node server to at least one processing node in at least one processing node server. The embodiment of the present application transmits data packets of a processing node connected to each fusion expansion switch to other processing nodes connected to the fusion expansion switch at the same time, so as to realize simultaneous transmission of data packets of processing nodes of the distributed system to other processing nodes of the distributed system, so as to make full use of the resources of each processing node in the distributed system.

[0055] In some embodiments, each converged extension switch includes multiple first switch groups and multiple second switch groups, each first switch group and each second switch group includes multiple switch units, each first switch group and each second switch group includes an identical switch unit; the switch units correspond one-to-one to the processing nodes to which they are connected; each switch unit is used to receive data packets sent by the corresponding processing node; transmit the data packets to at least one switch unit in the same switch group; and send the received data packets to the corresponding processing node.

[0056] The switching unit is used to communicate with the corresponding processing node.

[0057] like Figure 5 As shown, the internal structure of a converged expansion switch that can connect N 8-GPU servers is shown, including N first switch groups and 8 second switch groups, and each first switch group includes 8 switch units, and each second switch group includes N switch units. Each first switch group and each second switch group include an identical switch unit. During the intelligent computing training process, each first switch group is used to perform the function of GPU vertical expansion network transmission, and each second switch group is used to perform the function of GPU horizontal expansion network transmission on the same track.

[0058] In some embodiments, the communication process of multiple processing nodes corresponding to the fusion extension switch can be implemented as follows:

[0059] When the fusion extension switch receives a task data packet, it determines at least one target processing node corresponding to the task data packet, sends the task data packet to at least one switch unit corresponding to the target processing node, and the switch unit determines the destination processing node of the data packet.

[0060] If the target processing node is all the processing nodes of the system, the switch unit transmits the data packet to other switch units in the first switch group and other switch units in the second switch group. Further, other switch units perform the same operation to transmit the data packet to all switch units of the fusion extension switch, and each switch unit sends the data packet to the corresponding processing node.

[0061] If the target processing node is at least one processing node of the system, the switch unit transmits the data packet to the switch unit corresponding to each of the at least one processing node, so that the at least one switch unit sends the data packet to the corresponding processing node.

[0062] In some embodiments, Figure 5 As shown, the multiple switch units of the first switch group or the multiple switch units of the second switch group correspond one-to-one to the multiple processing nodes of the same processing node server.

[0063] That is, the number of switch units in the first switch group is the same as the number of processing nodes in the corresponding processing node server, and each switch unit is connected to a unique processing node.

[0064] The number of switch units in the second switch group is the same as the number of processing node servers, and each switch unit is connected to a unique processing node.

[0065] In some embodiments, during the distributed network training process of the intelligent computing center, the workload or data volume corresponding to each training task is relatively large. Therefore, in most cases, all processing nodes in the network are required to jointly process the same training task. Therefore, the fusion switch sends each training data packet to multiple target processing nodes corresponding to at least one processing node server corresponding to each fusion extension switch.

[0066] In the distributed network reasoning process of the intelligent computing center, the workload or data volume corresponding to the reasoning task is relatively small. Therefore, in most cases, multiple reasoning tasks will be carried out simultaneously in the network. In order to ensure the integrity and privacy of each reasoning task, when the switch unit in the fusion extension switch receives the reasoning data packet sent from the node processor, it will add its reasoning task label to the reasoning data packet, and then send the reasoning data packet to at least one corresponding switch unit. When the corresponding switch unit receives the reasoning data packet sent from other switch units, it will first detect whether the reasoning task label on the reasoning data packet matches the reasoning task assigned by its corresponding processing node. If it does not match, the received reasoning data packet is discarded and an alarm is issued; if it matches, the reasoning task label is first removed and then the reasoning data packet is sent to its corresponding target processing node.

[0067] In some embodiments, the horizontal expansion track switch is used to send a data packet sent by each processing node of each converged expansion switch to at least one processing node of another converged expansion switch.

[0068] If during the training process, the processing capacity of the processing node connected to a converged extension switch is insufficient to process the training task, the processing nodes of other converged extension switches can be combined to process the training task. Therefore, the horizontal extension track switch is used to transmit the data packets sent by each processing node of each converged extension switch to at least one processing node of other converged extension switches.

[0069] In some embodiments, the cross-zone switch cluster is used to transmit a data packet sent by each processing node of each horizontal expansion track switch to at least one processing node of other cross-zone switch clusters.

[0070] Based on the same principle, if the processing capacity of a processing node of a horizontally expanded track switch is insufficient to process the training task, the training task can be processed in conjunction with the processing nodes of other horizontally expanded track switches. Therefore, the cross-zone switch cluster is used to transmit the data packets sent by each processing node of each horizontally expanded track switch to at least one processing node of other cross-zone switch clusters.

[0071] The following is a description of a data transmission method proposed according to an embodiment of the present application in conjunction with the accompanying drawings. The method is applied to Figure 4 Any switch unit in a first converged extension switch in a distributed system, where the first converged extension switch is any converged extension switch of multiple converged extension switches in the distributed system.

[0072] like Figure 6 As shown, the method comprises the following steps:

[0073] S601: Receive a data packet.

[0074] The data packet may be a data packet for an inference task or a data packet for a training task.

[0075] The data packet may be sent from the corresponding fusion extension switch to the switch unit, or may be sent from other switch units to the switch unit.

[0076] S602: Determine the data packet type of the data packet.

[0077] The data packet type may be a data packet of an inference task or a data packet of a training task, wherein the data packet of the inference task includes an inference task label, and the data packet type of the data packet may be determined by detecting whether the data packet includes the inference task label.

[0078] S603: If the data packet is a training data packet, send the data packet to the connected processing node.

[0079] S604: If the data packet is an inference data packet, confirm that the inference task label conforms to the inference task of the connected processing node.

[0080] S605: If not, discard the data packet and issue an alarm.

[0081] S606: If it is in compliance, remove the inference task label and send the data packet to the connected processing node.

[0082] When receiving an inference data packet, the switch unit will first detect whether the inference task label on the inference data packet matches the inference task assigned by its corresponding processing node. If not, the received inference data packet will be discarded and an alarm will be issued; if it matches, the inference task label will be removed first and then the inference data packet will be sent to its corresponding target processing node.

[0083] The present application also provides an electronic device to perform the above data transmission method. Figure 7 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 7 As shown, the electronic device 7 includes: a processor 700, a memory 701, a bus 702 and a communication interface 703, and the processor 700, the communication interface 703 and the memory 701 are connected via the bus 702; the memory 701 stores a computer program that can be run on the processor 700, and when the processor 700 runs the computer program, it executes the data transmission method provided in any of the aforementioned embodiments of the present application.

[0084] The memory 701 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 703 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0085] The bus 702 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 701 is used to store programs, and the processor 700 executes the program after receiving an execution instruction. The data transmission method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 700, or implemented by the processor 700.

[0086] The processor 700 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 700. The above processor 700 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 701, and the processor 700 reads the information in the memory 701 and completes the steps of the above method in combination with its hardware.

[0087] The electronic device provided in the embodiment of the present application and the data transmission method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0088] The present application also provides a computer-readable storage medium corresponding to the data transmission method provided in the above embodiment. Figure 8 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the data transmission method provided by any of the aforementioned embodiments will be executed.

[0089] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0090] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the data transmission method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0091] It should be noted that:

[0092] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0093] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0094] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0095] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A distributed system, characterized in that: include: at least one cross-zone switch cluster, a plurality of horizontal expansion track switches, a plurality of converged expansion switches, and a plurality of processing node servers; Each processing node server includes a plurality of processing nodes; Each cross-zone switch cluster communicates with the multiple horizontal expansion track switches respectively, each horizontal expansion track switch communicates with the multiple converged expansion switches respectively, and each converged expansion switch communicates with multiple processing nodes corresponding to at least one processing node server among the multiple processing node servers; Each of the fusion extension switches is used to transmit a data packet sent by each processing node in the at least one processing node server to at least one processing node in the at least one processing node server.

2. The system according to claim 1, characterized in that Each of the converged extension switches includes a plurality of first switch groups and a plurality of second switch groups, each of the first switch groups and each of the second switch groups includes a plurality of switch units, and each of the first switch groups and each of the second switch groups includes a same switch unit; The switch units correspond one-to-one with the processing nodes they are connected to; Each switch unit is used to receive a data packet sent by a corresponding processing node; transmit the data packet to at least one switch unit in the same switch group; and send the received data packet to the corresponding processing node.

3. The system according to claim 1, characterized in that The multiple switch units of the first switch group or the multiple switch units of the second switch group correspond one-to-one to the multiple processing nodes of the same processing node server.

4. The system according to claim 1, characterized in that When receiving multiple training tasks, each of the fusion extension switches sends each training task to multiple first target processing nodes corresponding to at least one processing node server corresponding to each of the fusion extension switches; When receiving multiple inference tasks, each fusion extension switch sends multiple target inference tasks among the multiple inference tasks to second target processing nodes corresponding to at least one processing node server corresponding to each fusion extension switch.

5. The system according to claim 1, characterized in that When receiving multiple training tasks, each fusion extension switch divides the task data corresponding to each training task to obtain multiple sub-task data; and sends the multiple sub-task data of each training task to the first target processing node corresponding to each training task.

6. The system according to claim 1, characterized in that The horizontal expansion track switch is used to transmit the data packet sent by each processing node of each fusion expansion switch to at least one processing node of other fusion expansion switches.

7. The system according to claim 1, characterized in that The cross-zone switch cluster is used to transmit the data packet sent by each processing node of each horizontal expansion track switch to at least one processing node of other cross-zone switch clusters.

8. A data transmission method, characterized in that: Applied to any switch unit in a first converged extension switch in a distributed system according to any one of claims 1 to 7, wherein the first converged extension switch is any converged extension switch of a plurality of converged extension switches in the distributed system, the method comprising: Receive data packets; Determining the data packet type of the data packet; If the data packet is a training data packet, sending the data packet to the connected processing node; If the data packet is an inference data packet, confirming that the inference task label matches the inference task of the connected processing node; If not, discard the data packet and issue an alarm; If it matches, remove the inference task label and send the data packet to the connected processing node The second data packet is transmitted to at least one processing node in at least one processing node server corresponding to the first converged extension switch.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the computer program to implement the method according to claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to claim 8.

Images