Method and system for selecting optimal network topology structures of different types of large models

Abstract

The invention discloses an optimal network topology structure selection method and system for different types of large models, and belongs to the technical field of large models. The method comprises the following steps: expanding a search object into a large model, selecting the large model, expanding an Embedding operator for each selected large model, and adding a data processing dimension; determining several kinds of candidate cluster topological structures, and performing training time simulation on each candidate cluster topological structure on a coarse granularity level; performing time measurement on the key operator by using CUDA (Compute Unified Device Architecture), wherein the time measurement is used as an input parameter in a simulation process; the simulation training time under different cluster topological structures is compared, and whether an obvious topology selection tendency exists or not is determined according to a simulation result; and selecting the cluster topological structure with the shortest training time as an optimal solution. The topological structure can provide the fastest training speed under a given large model and a data processing task; and the efficiency of the selection process is greatly improved.

Description

Technical Field

[0001] This application proposes a method and system for selecting the optimal network topology structure of different types of large models, belonging to the technical field of large models. Background Art

[0002] In recent years, due to the demand for massive data processing, the continuous optimization of deep learning technology, and task scenarios in fields such as computer vision, natural language processing, and text dialogue, deep learning models have been continuously expanding, and the model parameters have increased exponentially, posing challenges to model training.

[0003] To optimize the model training time, one can start from the model's own structure. Currently, there are relatively mature technologies such as pruning, quantization, and model distillation. There are also distributed training technology frameworks such as Microsoft's Deepspeed framework that uses the ZERO technology, Google's DistBelief framework that uses the DP MP distributed strategy, and the TeraPipe framework that uses the PP distributed strategy.

[0004] Moreover, the resource network topology connection in the training cluster will also significantly affect the training efficiency. Among them, TopoOpt, based on the Flexflow project (a technology focused on searching for the optimal parallel strategy of deep learning models), targets several deep learning networks such as DLRM, VGG, ResNet, NCF, CANDLE, etc. By iteratively optimizing the network topology and parallel strategy of the cluster, the training time of the model is minimized.

[0005] Existing technologies such as TopoOpt mainly target deep learning models. It is not difficult to analyze that their network structures are relatively simple, the number of model layers is small, and the number of model parameters is small, resulting in relatively small network traffic. Now, deep learning technology has developed to large models for various processing and application scenarios based on massive data. Whether its iterative search optimization is applicable to the characteristics of complex model structures, massive parameters, and ultra-large traffic remains to be verified.

[0006] On the other hand, for large models based on the Transformer structure, the underlying implementations of Encoder, Decoder, and Encoder-Decoder have emerged. Whether their different underlying implementations have different tendencies in network topology selection during training also needs to be considered. Furthermore, for models such as the GPT series (GPT2), Llama series (Llama2), and MOE model, they have similar underlying implementations. However, after analyzing the models, it can be seen that the MOE model has a relatively sparse structure due to its selective activation characteristics, while the GPT2 and Llama2 models have a relatively dense structure. Whether their dense and sparse model structures also have different tendencies in network topology selection during training. Summary of the Invention

[0007] In view of the limitations of the prior art, the present application proposes a method and system for selecting the optimal network topology structure for different types of large models. This topology structure can provide the fastest training speed under a given large model and data processing task, and greatly improves the efficiency of the selection process.

[0008] To solve the above technical problems, the technical solution adopted in the present application is as follows: In a first aspect, the present application provides a method for selecting the optimal network topology structure for different types of large models, including: Extend the search object to the large model, select the large model, expand the Embedding operator for each selected large model, and increase the data processing dimension; Determine several candidate cluster topology structures. At a coarse-grained level, perform training time simulation on each candidate cluster topology structure; use CUDA to measure the time of key operators, and the time measurement will be used as an input parameter in the simulation process; Compare the simulation training times under different cluster topology structures. According to the simulation results, determine whether there is an obvious topology selection tendency; select the cluster topology structure with the shortest training time as the optimal solution.

[0009] As a further improvement of the present application, the performing training time simulation on each candidate cluster topology structure at a coarse-grained level includes: Construct a Taskgraph for operator Task to reflect the data dependency relationship and calculation order between operators; in the Taskgraph, add a parallel strategy, and according to the parallel strategy, add traffic communication.

[0010] As a further improvement of the present application, the simulation includes forward propagation, backward propagation, and weight update.

[0011] As a further improvement of the present application, the parallel strategy includes data parallelism and model parallelism.

[0012] As a further improvement of the present application, the traffic communication is for weight update and synchronization between DPs, and data forward and backward transmission between TPs, MPs, and PPs.

[0013] As a further improvement of the present application, the comparing the simulation training times under different cluster topology structures, and according to the simulation results, determining whether there is an obvious topology selection tendency; selecting the cluster topology structure with the shortest training time as the optimal solution includes: After setting the topology type of the cluster, calculate the operation time in the imported model file, randomly select a parallelization scheme for each operator, calculate according to the data flow direction, and finally obtain the current simulation training time. Then randomly select an operator to change its parallelization strategy and recalculate the new training time. If the new time is better than the simulation time of the previous step, update the parallelization strategy of this operator; otherwise, continue the search until the search times are reached or the result cannot be optimized further. Define the training time at this time as the optimal training time of the current topology.

[0014] In a second aspect, the present application provides a system for selecting the optimal network topology structure of different types of large models, including: An expansion module for expanding the search object into the large model, selecting the large model, expanding the Embedding operator for each selected large model, and increasing the data processing dimension; A simulation module for determining several candidate cluster topology structures, and performing training time simulation on each candidate cluster topology structure at a coarse-grained level; using CUDA to measure the time of key operators, and the time measurement will be used as an input parameter in the simulation process; A selection module for comparing the simulation training times under different cluster topology structures, and determining whether there is an obvious topology selection tendency according to the simulation results; selecting the cluster topology structure with the shortest training time as the optimal solution.

[0015] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for selecting the optimal network topology structure of different types of large models is implemented.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method for selecting the optimal network topology structure of different types of large models is implemented.

[0017] In a fifth aspect, the present application provides a computer program product, including computer instructions, and the computer instructions direct a computer to execute the method for selecting the optimal network topology structure of different types of large models.

[0018] The beneficial effects of the present application compared with the prior art are: A method for selecting the optimal network topology structure of different types of large models provided by this application expands the search object to the large model and clarifies the scope of the selected large model. For each selected large model, the Embedding operator is expanded and the data processing dimension is increased to adapt to more complex network environments and data processing requirements. Multiple candidate cluster topology structures are determined. These structures are simulated for training time at a coarse-grained level to evaluate their performance in practical applications. CUDA (Compute Unified Device Architecture) is used to measure the time of key operators. These time measurements will be used as input parameters in the simulation process to more accurately simulate the training time under different topologies. Compare the simulated training times under different cluster topologies. According to the simulation results, determine whether there is an obvious topology selection tendency, that is, whether a certain topology has a significant advantage in training time. Select the cluster topology structure with the shortest training time as the optimal solution. This topology can provide the fastest training speed under the given large model and data processing tasks. Through simulation and CUDA acceleration, the performance of different topologies can be quickly evaluated, so as to select the optimal topology, greatly improving the efficiency of the selection process. Using CUDA to measure the time of key operators can more accurately reflect the performance differences of different topologies during training. This makes the selection result more reliable. This method for selecting the optimal network topology structure of different types of large models has advantages such as high efficiency, accuracy, flexibility, and scalability, and can provide the optimal network topology structure selection scheme for different types of large models and data processing tasks. Brief Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following introduces the accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings in the following introduction are only for conveniently and clearly presenting some embodiments of the technical solutions in this application. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a flowchart of a method for selecting the optimal network topology structure of different types of large models provided by this application; Figure 2 It is a schematic diagram of the Taskgraph task flow in Flexflow provided by the embodiment of this application; among them, (a) is an example of a parallelization strategy, (b) is the corresponding task flow chart, and (c), (d) are test processes; Figure 3(a) shows the experimental results of GPT2 simulation; Figure 3 (b) shows the experimental results of Llama2 simulation; Figure 3 (c) shows the experimental results of MOE simulation; Figure 4 A method and device for selecting the optimal network topology structure of different types of large models provided by this application; Figure 5 A schematic diagram of an electronic device provided by this application. Detailed implementation manners

[0021] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as a limitation to the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0022] In the description of the present application, unless otherwise clearly defined, words such as setting, installation, and connection should be understood in a broad sense. Those skilled in the art can reasonably determine the specific meanings of the above words in the present application in combination with the specific content of the technical solution.

[0023] The present application relates to a method for selecting the optimal network topology structure of different types of large models. The basic content of the present application is divided into two parts. One is that TopoOpt expands the search object to large models, realizes the expansion of the Embedding operator and the data processing dimension, and simultaneously realizes the implementation of GPT2, Llama2, and MOE models. The other is that the three models perform training time simulation on several cluster topology structures at a coarse-grained level, measure the time of the operator based on CUDA, build a Taskgraph of the operator Task according to the model definition, build the dependency relationship between Tasks through the data dependency relationship, and add traffic communication between parallel strategies such as Allreduce (used for weight update and synchronization between DPs, and forward and backward data transfer between TPs, MPs, and PPs). Finally, compare the simulated training time to determine whether there is a topology selection tendency. As Figure 1 shown, the specific steps are as follows: Step 1. Expand the search object to large models; 1. Select the scope of large models: Select the types of large models to be covered, such as GPT2, Llama2, and MOE, etc.

[0024] 2. Embedding Operator and Data Processing Dimension Expansion: For each selected large model, expand its Embedding operator to support a wider range of data inputs. Increase the data processing dimension to accommodate datasets of different scales and types.

[0025] 3. Model Implementation: Implement the specific algorithms and architectures of models such as GPT2, Llama2, and MOE.

[0026] Step 2: Determine the Optimal Network Topology Based on Coarse-Grained Simulation 1. Select Cluster Topologies: Identify several candidate cluster topologies, such as Fat-Tree, Torus, HyperCube, etc.

[0027] 2. Coarse-Grained Training Time Simulation: At the coarse-grained level, perform training time simulations for each candidate cluster topology. The simulation should cover the main stages of model training, including forward propagation, backward propagation, and weight updates, etc.

[0028] 3. CUDA Operator Time Measurement: Use CUDA to measure the time of key operators (such as matrix multiplication, convolution, etc.). These time measurements will serve as important input parameters during the simulation process.

[0029] 4. Build Taskgraph: According to the model definition, build the Taskgraph of operator Tasks. The Taskgraph should reflect the data dependency relationships and computational order among the operators.

[0030] 5. Add Parallel Strategies and Traffic Communication: In the Taskgraph, add parallel strategies such as data parallelism, model parallelism, etc. According to the parallel strategies, add traffic communication such as Allreduce operations for weight updates and synchronization, and data forward and backward passing between TP, MP, and PP.

[0031] 6. Comparison of Simulation Results and Topology Selection: Compare the simulation training times under different cluster topologies. Based on the simulation results, determine whether there is an obvious tendency in topology selection. Select the cluster topology with the shortest training time as the optimal solution.

[0032] Furthermore, expand the search object to large models and determine the optimal network topology through coarse-grained simulation. During the simulation process, key technologies such as CUDA operator time measurement, Taskgraph construction, parallel strategy addition, and traffic communication addition are used. Finally, the optimal cluster topology is determined by comparing the simulation training times.

[0033] Furthermore, the network topology determines the data transmission paths and methods in the network, thus affecting the network performance. Different topologies vary in terms of data transmission efficiency, communication latency between nodes, fault tolerance, etc. Through coarse-grained simulation, the performance of different topologies in practical applications can be approximately evaluated. This simulation method can compare and select multiple topologies without actually building the network. CUDA is a parallel computing platform and programming model introduced by NVIDIA, which allows developers to use GPUs for high-performance computing. Measuring the time of key operators using CUDA can more accurately evaluate the performance differences of different topologies during the training process.

[0034] For example, the Taskgraph in Flexflow is as Figure 2 shown: Figure 2 (a) shows an example of the parallelization strategy of a standard 3-layer recurrent neural network, which consists of an embedding layer, a recurrent layer, and a linear layer. Config, i.e., the parallel strategy, indicates how to allocate the operation operations in each layer to the designated GPUs. For example, batch = 2 means that the current operation is allocated to two GPUs in the Batch dimension.

[0035] Figure 2 (b) shows the corresponding task flow chart. Each square and hexagon represent an ordinary task and a communication task respectively, and each directed edge represents the dependency relationship between tasks. The exeTime (task execution time) and device (task execution device number) of the tasks are shown at the top of each column. t 1:2 : represents the 2nd parallel task of operation operation1. For each task t i:ki and t j:kj , if the two tasks are assigned to the same device, an edge (t i:ki -> t j:kj ) is added in the taskhgraph, indicating that there is a data dependency relationship between the two tasks. Otherwise, a communication task t c as shown in the figure needs to be added between them.

[0036] In Figure 2 (c) and Figure 2 (d), the words "r" and "s" represent the readyTime (ready time) and startTime (start execution time) of each task respectively, and the dashed edges represent the nextTask (the next task associated with the current task).

[0037] It proposes a method for optimizing the network topology connection method among internal nodes of a cluster to find a way to reduce the training time for large models, not just limited to simple deep learning models. It proposes and verifies the hypothesis that different types of large models have different tendencies for cluster topology connection due to their underlying structures or the sparsity of their structures. The experiments based on the above hypothesis are verified.

[0038] This application extends TopoOpt, adds the Embedding operator and MOE model-related operators in large models, and expands the dimensions of the operators according to the dimensions of the input data, with a wider applicability. It refines the traffic pattern, not only allowing the DP policy and only including the weight update Allreduce in DP, but also adding the traffic patterns of TP and PP, considering the model traffic more comprehensively, and determining a more perfect distributed policy. It can provide a cluster topology selection scheme for minimizing the training time for deep learning models, especially large models, based on their structural characteristics.

[0039] In order to verify the feasibility of this application through experiments. All these experiments are implemented using CUDA and Cpp languages and tested using Shell scripts. CUDA is used to measure the forward and backward propagation times of the model operators; a Simulator class is designed to construct the Taskgraph of the operators based on the definition of the model structure, add the dependency relationships of the Taskgraph for data correlation, and for the Allreduce operation, a new data path is added by constructing the routing between the source node and the target node, thereby adding new communication-related Tasks, and finally simulating the training time. Specifically, the experimental steps of this application are as follows: In the first step, according to the input model type, such as GPT2 or Llama2 or MOE, measure the forward and backward propagation times of its operators. According to possible parallel strategies, such as DP, TP, etc., measure the propagation time under this parallel strategy as a candidate for the optimal parallel strategy of this operator. After measuring all operators, generate the measure.json file of the results.

[0040] In the second step, after the Simulator class sets the topology type of the cluster, calculate the operator calculation times by importing the measure.json file of the model in this class. Randomly select a parallel scheme for each operator, calculate according to the data flow direction, and finally obtain the current simulated training time. Then randomly select an operator to change its parallel strategy and recalculate the new training time. If the new time is better than the simulated time of the previous step, update the parallel strategy of this operator, otherwise continue to search until the search times are reached or the result cannot be optimized further. Define the training time at this time as the optimal training time of the current topology.

[0041] In the third step, perform steps 1 and 2 for each large model and organize the data. Some of the implementation results are as follows: The experimental results for GPT2 simulation are shown in Figure 3(a); the experimental results for Llama2 simulation are shown in Figure 3(b); the experimental results for MOE simulation are shown in Figure 3(c).

[0042] The following conclusions can be drawn from the above experiments: 1) Deep learning models with relatively complex model structures (specifically large language models in this experiment) have a certain tendency in the selection of (logical array center) network topologies.

[0043] 2) Different types of large models have different tendencies in the selection of (logical data center) network topologies.

[0044] 3) In the model training scenario, For large models with a dense structure such as GPT and Llama models, the running time of simulation under the Fattree topology is relatively short, showing a tendency to select the Fattree data center network topology; while for large models with a sparse structure such as the MOE model, the running time of simulation under the TopoOpt topology is relatively short, showing a tendency to select the TopoOpt data center network topology. network topology selection tendency. Figure 4

[0045] As Figure 5 shown, the third objective of this application is to provide an optimal network topology selection system for different types of large models, including: An expansion module for expanding the search object to large models, selecting a large model, expanding the Embedding operator for each selected large model, and increasing the data processing dimension; A simulation module for determining several candidate cluster topologies and performing training time simulation on each candidate cluster topology at a coarse-grained level; using CUDA to measure the time of key operators, and the time measurement will be used as an input parameter in the simulation process; A selection module for comparing the simulation training times under different cluster topologies, determining whether there is an obvious topology selection tendency according to the simulation results, and selecting the cluster topology with the shortest training time as the optimal solution.

[0046] The simulation module is specifically used for: Constructing a Taskgraph for operator Task to reflect the data dependency relationship and calculation order between operators; adding a parallel strategy to the Taskgraph and adding traffic communication according to the parallel strategy. The simulation includes forward propagation, backward propagation, and weight update.

[0047] The parallel strategy includes data parallelism and model parallelism; traffic communication is for weight update and synchronization between DPs and the forward and backward data transfer between TPs, MPs, and PPs.

[0048] The selection module is specifically used for: After setting the topology type of the cluster, calculate the operator computing time in the imported model file, randomly select a parallelization scheme for each operator, calculate according to the data flow direction, and finally obtain the current simulation training time. Then randomly select an operator to change its parallelization strategy and recalculate the new training time. If the new time is better than the previous simulation time, update the parallelization strategy of this operator; otherwise, continue the search until the search times are reached or the result cannot be optimized further. Define the training time at this time as the optimal training time of the current topology.

[0049] The optimal network topology structure selection system for different types of large models in this application is based on the above-mentioned optimal network topology structure selection method for different types of large models.

[0050] As Figure 1 shown, the fourth objective of the embodiment of this application is to provide an electronic device, including a memory 701, a processor 702, and a computer program stored in the memory 701 and executable on the processor. When the processor executes the computer program, it implements the above-mentioned optimal network topology structure selection method for different types of large models. It also includes a communication interface 703 and a bus 704.

[0051] The fifth objective of the embodiment of this application is to provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned optimal network topology structure selection method for different types of large models.

[0052] The sixth objective of the embodiment of this application is to provide a computer program product. The computer program product includes computer instructions, and the computer instructions direct the computer to execute the above-mentioned optimal network topology structure selection method for different types of large models.

[0053] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the functions specified in the process Figure 1 one process or multiple processes and / or boxes Figure 1 specified in one box or multiple boxes.

[0054] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or boxes Figure 1 specified in one box or multiple boxes.

[0055] This application may be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, readable storage media, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks ​ one block or multiple blocks.

[0057] Obviously, the described embodiments are only partial embodiments of this application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the specific implementation manners of this application or make equivalent replacements. Any modification or equivalent replacement that does not depart from the spirit and scope of this application shall be covered by the protection scope of the claims of this application.

Claims

1. A method for selecting optimal network topology structures for different types of large models, characterized in that: include: Expand the search object to large models, select large models, expand the Embedding operator and increase the data processing dimension for each selected large model; Determine several candidate cluster topologies, and perform training time simulation on each candidate cluster topology at a coarse-grained level; Use CUDA to measure the time of key operators, and the time measurement will be used as an input parameter in the simulation process; Compare the simulation training time under different cluster topologies and determine whether there is an obvious topology selection tendency based on the simulation results; The cluster topology with the shortest training time is selected as the optimal solution.

2. The method for selecting optimal network topology structures for different types of large models according to claim 1, characterized in that: The training time simulation is performed on each candidate cluster topology structure at a coarse-grained level, including: Build the Taskgraph of the operator Task to reflect the data dependency and calculation order between operators; add parallel strategies in the Taskgraph, and add traffic communication based on the parallel strategies.

3. The method for selecting optimal network topology structures of different types of large models according to claim 2 is characterized in that: The simulation includes forward propagation, backward propagation and weight updating.

4. The method for selecting optimal network topology structures for different types of large models according to claim 2, characterized in that: The parallel strategies include data parallelism and model parallelism.

5. The method for selecting optimal network topology structures of different types of large models according to claim 2, characterized in that: The traffic communication is used for updating and synchronizing weights between DPs, and for forward and backward transmission of data between TPs, MPs, and PPs.

6. The method for selecting optimal network topology structures of different types of large models according to claim 2, characterized in that: The simulation training time under different cluster topologies is compared, and according to the simulation results, whether there is an obvious topology selection tendency; The cluster topology with the shortest training time is selected as the optimal solution, including: After setting the topology type of the cluster, the operator calculation time in the imported model file is used to randomly select a parallel scheme for each operator, and the calculation is performed according to the data flow direction. Finally, the current simulation training time is obtained. Then, an operator is randomly selected to change its parallel strategy and the new training time is recalculated. If the new time is better than the simulation time in the previous step, the parallel strategy of the operator is updated. Otherwise, the search continues until the search number is reached or the result cannot be optimized any more. The training time at this time is defined as the optimal training time for the current topology.

7. A system for selecting optimal network topology structures for different types of large models, characterized in that: include: The expansion module is used to expand the search object to the large model, select the large model, expand the Embedding operator for each selected large model, and increase the data processing dimension; A simulation module, used to determine several candidate cluster topologies, and to simulate the training time of each candidate cluster topology at a coarse-grained level; Use CUDA to measure the time of key operators, and the time measurement will be used as an input parameter in the simulation process; The selection module is used to compare the simulation training time under different cluster topologies and determine whether there is an obvious topology selection tendency based on the simulation results; The cluster topology with the shortest training time is selected as the optimal solution.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for selecting optimal network topology structures of different types of large models as described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for selecting optimal network topology structures for different types of large models as described in any one of claims 1-6 is implemented.

10. A computer program product, comprising computer instructions, characterized in that: The computer instructions instruct the computer to execute the method for selecting optimal network topology structures for different types of large models as described in any one of claims 1-6.

Images