High-performance RoCE network based on optical connection

Abstract

The invention relates to the technical field of computing centers, in particular to a high-performance RoCE network based on optical connection, which comprises a storage module, a model training module and a model reasoning module which are mutually connected through an optical network, model training data are pre-stored in the storage module; the model training module adopts the model training data to train a to-be-trained model to obtain a business model; and the model reasoning module generates a corresponding processing result by adopting a business model after receiving an external processing demand and returns the processing result. In order to solve the problems that in the prior art, an intelligent computing center lacks scheduling during application, and the same computing task is possibly dispersed on clusters at different positions, so that time delay is increased, in the scheme, a server cluster is planned, three modules are divided according to different storage requirements of an artificial intelligence model, and the time delay is increased; the tasks of the same type are arranged and scheduled in a centralized manner respectively corresponding to three parts of model labeling, training and application, so that the transmission distance is shortened, and the efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computing centers, and in particular to a high-performance RoCE network based on optical connection. Background Art

[0002] Intelligent computing dedicated cloud, also known as intelligent computing center, refers to the use of large-scale heterogeneous computing resources, including general computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.), mainly for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning and other scenarios) to provide the required computing power, data and algorithm facilities. Compared with traditional data centers, intelligent computing centers can meet more targeted needs, as well as larger computing volume and faster computing speed, and provide AI computing power for various vertical industry scenarios such as large model training reasoning, autonomous driving, AIGC, etc. Users access the intelligent computing center through the Internet and deploy models. The management center schedules computing power according to business needs and provides network support.

[0003] For example, Chinese patent CN202311663029.3 discloses a networking system for an intelligent computing center, which includes: a terminal device module that receives and sends network data based on the RoCE network and carries an internal computing module for data processing, and an internal computing module that calculates and processes data based on the DPU and in combination with the GPU and transmits it; an optical fiber transmission module that uses optical fiber intelligent networking to transmit network data, and a satellite transmission module that performs emergency networking based on the EMSTM interactive platform to transmit network data; a receiving and processing module that pre-processes the received computing data based on the mobile terminal software before viewing it, and an authentication module that identifies and authenticates users who query computing data. The system solves the problem of the singleness of existing transmission networking methods through intelligent detection and switching of two transmission types of the data transmission module. In the event of optical fiber damage or large-scale disconnection, it can be transmitted through satellite communication networking, ensuring that the intelligent computing center can continue to provide efficient intelligent computing services.

[0004] However, during the actual implementation process, the inventors found that this type of technical solution usually only interconnects servers through the network, and does not distinguish between different regional functions. This leads to complex load scheduling during the application process. For example, model calculation tasks are dispersed across multiple clusters, resulting in increased transmission delays. Summary of the invention

[0005] In view of the above problems existing in the prior art, a high-performance RoCE network based on optical connection is now provided.

[0006] The specific technical solutions are as follows:

[0007] A high-performance RoCE network based on optical connection, including a storage module, a model training module and a model inference module interconnected by an optical network;

[0008] The storage module pre-stores model training data;

[0009] The model training module uses the model training data to train the model to be trained to obtain a business model;

[0010] After receiving external processing requirements, the model reasoning module uses the business model to generate and return corresponding processing results.

[0011] In another aspect, the optical network includes a spine switch and a leaf switch;

[0012] The storage module, the model training module and the model reasoning module are respectively connected to a first end of a leaf layer switch;

[0013] The second end of each leaf layer switch is respectively connected to the spine layer switch;

[0014] The leaf layer switch operates in remote memory direct access mode.

[0015] On the other hand, the optical network includes a plurality of network groups, each of which includes one spine switch and three leaf switches connected to each other;

[0016] Each of the network packets exchanges data on the leaf-layer switch corresponding to the storage module, the model training module, or the model inference module.

[0017] On the other hand, the leaf layer switches include a first type of leaf layer switches and a second type of leaf layer switches;

[0018] The first type of leaf switch is located at the top of the rack and is directly connected to the server in the storage module or the model training module or the model reasoning module;

[0019] The second type leaf layer switches are respectively connected to the first type leaf layer switches and other second type leaf layer switches;

[0020] The second type leaf layer switch is also connected to the spine layer switch.

[0021] On the other hand, the second type of leaf layer switches includes intra-module switches, and the intra-module switches are not connected to the pole layer switches;

[0022] The intra-module switch is used for data exchange within the storage module, the model training module, or the model reasoning module.

[0023] On the other hand, the storage module comprises:

[0024] An object storage module, wherein the object storage module is configured with original data;

[0025] A parallel file storage module, wherein the parallel file storage module stores the marked data;

[0026] The object storage module and the parallel file storage module exchange data through the leaf layer switch.

[0027] On the other hand, the model training module and the model reasoning module include multiple GPU clusters, and the GPU clusters exchange data through the leaf layer switch.

[0028] On the other hand, the GPU cluster includes a first network card and a second network card, and the second type of leaf layer switch includes an inter-module switch;

[0029] The first network card is connected to the inter-module switch, and the second network card is connected to the intra-module switch;

[0030] The port rate of the second network card is higher than that of the first network card.

[0031] The above technical solution has the following advantages or beneficial effects:

[0032] In order to address the problem that the intelligent computing center in the existing technology lacks scheduling when applied, the same computing task may be dispersed on clusters in different locations, resulting in increased latency. In this solution, the server cluster is planned and divided into three modules according to the different storage requirements of the artificial intelligence model, corresponding to the three parts of model annotation, training and application. Tasks of the same type are centrally arranged and scheduled, thereby shortening the transmission distance and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0034] Figure 1 It is an overall schematic diagram of an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of network grouping in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of a leaf layer switch in an embodiment of the present invention;

[0037] Figure 4 Schematic diagram of a switch in a module according to an embodiment of the present invention;

[0038] Figure 5 Schematic diagram of a storage module in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0042] The present invention comprises:

[0043] A high-performance RoCE network based on optical connections, such as Figure 1 As shown, it includes a storage module A1, a model training module A2 and a model reasoning module A3 which are interconnected by an optical network;

[0044] The storage module A1 stores model training data in advance;

[0045] The model training module A2 uses the model training data to train the model to be trained to obtain a business model;

[0046] After receiving external processing requirements, the model reasoning module A3 uses the business model to generate and return the corresponding processing results.

[0047] Specifically, in view of the problem that the intelligent computing center in the prior art lacks scheduling when applied, and the same computing task may be dispersed on clusters in different locations, resulting in increased latency, this solution plans the server cluster and divides it into three modules according to the different storage requirements of the artificial intelligence model, including storage module A1, model training module A2 and model reasoning module A3. Multiple corresponding server clusters are configured in each module to undertake the tasks on the corresponding module.

[0048] For example, a storage server is deployed in the storage module A1, which has a disk array, read-write cache and parallel file system, and can achieve high-concurrency file storage and read-write work. It is mainly used for the cleaning of raw data, data annotation and data set generation process during model training. In the subsequent model training process, files are also read through this module.

[0049] The model training module A2 is mainly equipped with a GPU cluster, which has high computing power to train the model. It reads the data set in the storage module A1 through the optical network for training.

[0050] The model reasoning module A3 is configured with a corresponding GPU cluster, which is used to receive the model and user analysis requirements after the model training module A2 completes the model training, and returns the results to the user after making inferences and predictions based on the business model.

[0051] In one embodiment, the optical network includes a spine switch 1 and a leaf switch 2;

[0052] The storage module A1, the model training module A2 and the model reasoning module A3 are respectively connected to a first end of a leaf layer switch 2;

[0053] The second end of each leaf switch 2 is connected to the spine switch 1 respectively;

[0054] Leaf switch 2 operates in remote memory direct access mode.

[0055] Specifically, in order to achieve better data access efficiency, in this embodiment, the above switch networking structure is selected. Specifically, at least one group of leaf switches 2 is configured on each functional partition, and the leaf switches 2 are connected to each server in the cluster through an optical port, and then connected to the upper spine switch 1 through the optical port on the WAN side to exchange data in different functional areas.

[0056] At the same time, in order to achieve better read and write efficiency, the leaf switch 2 is set as a switch with RoCE function (RDMA over Converged Ethernet, RDMA based on converged Ethernet), and the relevant access parameters are configured on the corresponding cluster. Based on the remote memory direct access mode, the leaf switch 2 can directly read and write the memory of each server in the cluster, including the storage module A1, the model training module A2 and the model reasoning module A3, to improve the access efficiency.

[0057] For example, when the model training module A2 receives the model to be trained uploaded by the user, it needs to call the data set stored in the storage module A1 for training. The storage module A1 then searches for the corresponding data set through the file system on the server, reads it out, and sends it to the leaf switch 2 on the side of the storage module A1. Then the leaf switch 2 on the side of the storage module A1 forwards the data to the leaf switch 2 on the side of the model training module 2 via the spine switch 1. The leaf switch 2 on the side of the model training module 2 directly writes to the memory of the server with the model training task, and then the processor of the server accesses the memory to obtain data, and stores the trained model through local storage. Better read and write efficiency is achieved by directly operating on the memory.

[0058] In one embodiment, Figure 2 As shown, the optical network includes a plurality of network groups, each of which includes a spine switch 1 and at least three leaf switches 2 that are interconnected;

[0059] Each network packet exchanges data on the leaf layer switch 2 corresponding to the storage module A1, the model training module A2, or the model inference module A3.

[0060] Specifically, in order to achieve better exchange efficiency for a large number of server clusters, in this embodiment, multiple network groups are created for server clusters that exceed the number of LAN side optical ports of the leaf layer switch 2. Each network group includes a spine switch 1 and at least three leaf layer switches 2 that are interconnected to achieve exchange between functional areas. Accordingly, in order to achieve data exchange between functional modules, such as multiple server clusters on the storage module A1, especially the exchange of server clusters across network groups, the leaf layer switch 2 is also internally networked. When any two servers in the storage module A1 need to exchange data, they can self-organize the network through the leaf layer switches 2 at this location, determine a host, read all the servers connected to the slaves, and then forward them centrally.

[0061] In one embodiment, Figure 3 As shown, the leaf layer switch 2 includes a first type leaf layer switch 21 and a second type leaf layer switch 22;

[0062] The first type of leaf switch 21 is located at the top of the rack and directly connected to the server in the storage module A1 or the model training module A2 or the model reasoning module A3;

[0063] The second type leaf switches 22 are respectively connected to the first type leaf switches 21 and other second type leaf switches 22;

[0064] The second type of leaf layer switch 22 is also connected to the spine layer switch 1.

[0065] Specifically, in order to achieve better intra-group switching efficiency, in this embodiment, on the basis of constructing the above-mentioned network grouping, a first type leaf layer switch 21 and a second type leaf layer switch 22 are also designed.

[0066] Normally, the server is installed on a rack, and the top of the rack is provided with structures such as cable troughs and wiring racks. Since the leaf switch 2 works in remote memory access mode, in order to achieve faster memory read and write efficiency, in this embodiment, some first-class leaf switches 21 are configured to be located at the top of the rack and directly connected to the servers in the storage module A1 or the model training module A2 or the model reasoning module A3, and the speed of read and write control is improved through a one-to-one connection. Then, according to the exchange requirements between the servers in the module, other second-class leaf switches 22 are set to network the first-class leaf switches 21 to realize data exchange within the cluster and forwarding to a higher network segment.

[0067] In one embodiment, Figure 4 As shown, the second type leaf switch 22 includes an intra-module switch 23, and the intra-module switch 23 is not connected to the first type leaf switch 21;

[0068] The intra-module switch 23 is used for data exchange within the storage module A1 or the model training module A2 or the model reasoning module A3.

[0069] Specifically, in the storage module A1 and the model training module A2, there are relatively frequent reading, writing and data exchange requirements during data labeling and model training. In this embodiment, an additional intra-module switch 23 is configured in the second type leaf switch 22 installed on the rack. The intra-module switch 23 is not connected to the first type leaf switch 21, and is only used for the exchange of multiple clusters within the module, thereby improving the efficiency of intra-group data exchange. Taking model training as an example, when training a large-scale model, the GPU cluster can perform a data exchange process equivalent to the bus rate through the intra-module switch 23, which is convenient for improving the speed of model training.

[0070] In one embodiment, the GPU cluster includes a first network card and a second network card, and the second type of leaf switch includes a non-intra-module switch;

[0071] The first network card is connected to a non-module switch, and the second network card is connected to a module switch 23;

[0072] The port rate of the second network card is higher than that of the first network card.

[0073] Specifically, in order to achieve lower equipment costs, in this embodiment, for the GPU cluster that needs to perform intra-module switching, a second network card with a higher rate is configured near the side of the intra-module switch to improve the data transmission efficiency during parallel computing and reduce equipment costs.

[0074] In one embodiment, Figure 5 As shown, the storage module 1A includes:

[0075] An object storage module 1A1, wherein the object storage module 1A1 is configured with original data;

[0076] A parallel file storage module 1A2, wherein the parallel file storage module 1A2 stores the marked data;

[0077] The object storage module 1A1 and the parallel file storage module 1A2 exchange data through the leaf layer switch 2.

[0078] Specifically, in order to improve the overall efficiency of the model training process, in this embodiment, after the original data is collected, it is first stored through the object storage module 1A1. Then, in the data annotation process, when the annotated data is generated, the data is transferred to the parallel file storage module 1A2 through the leaf layer switch 2, and then read and written by the model training module A2.

[0079] Those skilled in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, may be specifically implemented as systems, methods, or computer program products. Therefore, various aspects of the present invention, or possible implementations of various aspects, may take the form of complete hardware embodiments, complete software embodiments (including firmware, resident software, etc.), or embodiments of combined software and hardware aspects, all collectively referred to herein as "circuits," "modules," or "systems." In addition, various aspects of the present invention, or possible implementations of various aspects, may take the form of computer program products, which refer to computer instructions stored in a memory.

[0080] The memory may be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable read-only memory (CD-ROM).

[0081] The processor in the computer reads the computer instructions stored in the memory, so that the processor can execute the functional actions specified in each step or the combination of steps in the flowchart; and generate a device for implementing the functional actions specified in each block or the combination of blocks in the block diagram.

[0082] It should be understood that the processor in the computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components implemented to execute the aforementioned computer instructions.

[0083] Computer instructions can be executed completely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or completely on a remote computer or server. It should also be noted that in some alternative embodiments, the functions noted in each step in the flow chart or each block in the block diagram may not occur in the order noted in the figure. For example, depending on the functions involved, two steps or two blocks shown in succession may actually be executed roughly simultaneously, or these blocks may sometimes be executed in reverse order.

[0084] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-performance RoCE network based on optical connection, characterized in that: It includes a storage module, a model training module and a model reasoning module interconnected by an optical network; The storage module pre-stores model training data; The model training module uses the model training data to train the model to be trained to obtain a business model; After receiving external processing requirements, the model reasoning module uses the business model to generate and return corresponding processing results.

2. The high performance RoCE network according to claim 1, characterized in that: The optical network includes a spine switch and a leaf switch; The storage module, the model training module and the model reasoning module are respectively connected to a first end of a leaf layer switch; The second end of each leaf layer switch is respectively connected to the spine layer switch; The leaf layer switch operates in remote memory direct access mode.

3. The high performance RoCE network according to claim 2, characterized in that: The optical network includes a plurality of network groups, each of which includes one spine switch and at least three leaf switches connected to each other; Each of the network packets exchanges data on the leaf-layer switch corresponding to the storage module, the model training module, or the model inference module.

4. The high performance RoCE network according to claim 2, characterized in that: The leaf layer switches include a first type of leaf layer switch and a second type of leaf layer switch; The first type of leaf switch is located at the top of the rack and is directly connected to the server in the storage module or the model training module or the model reasoning module; The second type leaf layer switches are respectively connected to the first type leaf layer switches and other second type leaf layer switches; The second type leaf layer switch is also connected to the spine layer switch.

5. The high performance RoCE network according to claim 4, characterized in that: The second type of leaf layer switches includes an intra-module switch, and the intra-module switch is not connected to the first type of leaf layer switch; The intra-module switch is used for data exchange within the storage module, the model training module, or the model reasoning module.

6. The high performance RoCE network according to claim 1, characterized in that: The storage module comprises: An object storage module, wherein the object storage module is configured with original data; A parallel file storage module, wherein the parallel file storage module stores the marked data; The object storage module and the parallel file storage module exchange data through the leaf layer switch.

7. The high performance RoCE network according to claim 5, characterized in that: The model training module and the model reasoning module include multiple GPU clusters, and the GPU clusters exchange data through the leaf layer switch.

8. The high performance RoCE network according to claim 7, characterized in that: The GPU cluster includes a first network card and a second network card, and the second type of leaf layer switch includes a non-intra-module switch; The first network card is connected to the non-module switch, and the second network card is connected to the module switch; The port rate of the second network card is higher than that of the first network card.

Images