Ai model super-network optimization method and apparatus, and computer-readable storage medium

Abstract

Disclosed in the present invention are an AI model super-network optimization method and apparatus, and a computer-readable storage medium. The method comprises: segmenting a first candidate model to obtain a plurality of first candidate model sub-structures, and segmenting a second candidate model to obtain a plurality of second candidate model sub-structures; then, uniformly distributing the plurality of first candidate model sub-structures and the plurality of second candidate model sub-structures into a plurality of computing nodes; and performing iterative optimization on the plurality of first candidate model sub-structures and the plurality of second candidate model sub-structures, and during optimization, using a weight sharing method to enable the plurality of first candidate model sub-structures and the plurality of second candidate model sub-structures to form a super-network, wherein the first candidate model and the second candidate model are candidate models of different structures that are obtained by performing structural transformation on the basis of the same basic model. The method can solve the problem of computing nodes being unable to support a large-model super-network and optimize the large-model super-network in the current super-network optimization algorithms.

Description

AI model super network optimization method, device and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on October 11, 2023, with application number 202311319318.1, and priority to the Chinese patent application entitled “AI model super network optimization method, device and computer-readable storage medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to an AI model super-network optimization method, device, and computer-readable storage medium. Background Art

[0003] With the development of AI technology, neural networks (also called models or AI models) have been widely used in fields such as computer vision (CV), natural language processing (NLP), graph learning, and recommendation systems. Because different application scenarios have different requirements for models, it is necessary to select an appropriate model based on the needs of the application scenario. In the past, models were all manually designed. In recent years, the super network optimization algorithm has been developed. This algorithm can automatically iteratively optimize the super network formed by a large number of candidate models (also called candidate networks) and determine the most suitable candidate model from the optimized super network.

[0004] Currently, the most commonly used hypernetwork optimization algorithm in the industry is the neural architecture search (NAS) algorithm. The NAS algorithm works as follows: Structural changes are made to the base model to obtain multiple candidate models with different structures. Assuming there are M candidate models, all of them are deployed to N computing nodes (such as graphics processing units (GPUs), tensor processing units (TPUs), and neural network processing units (NPUs)). Then, the N*M candidate models on the N computing nodes are iteratively optimized. Specifically, during the optimization process, weight sharing is first used to form a hypernetwork from the M candidate models on each computing node, resulting in N hypernetworks corresponding to the N computing nodes. These N hypernetworks on the N computing nodes are then iteratively optimized.

[0005] In the above NAS algorithm, since each computing node needs to carry the super-network formed by multiple candidate models, and the computing node's carrying capacity is limited, if the parameter scale of the super-network is very large (such as billions or hundreds of billions), the computing node cannot bear it and cannot optimize the large-model super-network.

[0006] Summary of the Invention

[0007] The present application provides an AI model supernetwork optimization method, device and computer-readable storage medium, which can solve the problem that the computing nodes in the current supernetwork optimization algorithm cannot carry the AI ​​model supernetwork and cannot optimize large-scale AI model supernetworks (also called large model supernetworks).

[0008] In a first aspect, a method for super-network optimization of an AI model is provided, which may include the following steps: dividing a first candidate model to obtain multiple first candidate model substructures, and dividing a second candidate model to obtain multiple second candidate model substructures, wherein the multiple first candidate model substructures are distributed in multiple computing nodes, and the multiple second candidate model substructures are also distributed in multiple computing nodes, and the first candidate model and the second candidate model are candidate models of different structures obtained by structural changes based on the same basic model, and then the multiple first candidate model substructures and the multiple second candidate model substructures are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super-network.

[0009] In the above scheme, after the basic model is structurally changed to obtain multiple candidate models with different structures (the above scheme takes the multiple candidate models as the first candidate model and the second candidate model as an example), each candidate model is segmented to obtain multiple substructures of each candidate model, and the multiple substructures of each candidate model are distributed among multiple computing nodes. Then, the substructures of the multiple candidate models are iteratively optimized, and during the optimization process, a weight sharing method is adopted to form a super network of the substructures of the multiple candidate models on the multiple computing nodes. It can be seen that each computing node only needs to carry one substructure of each of the multiple candidate models that make up the super network, and does not need to carry the entire super network. The complete super network is carried by multiple computing nodes together, and subsequently the multiple computing nodes together optimize the super network, thereby solving the problem in the NAS algorithm that the computing nodes cannot carry the large model super network and cannot optimize the large model super network.

[0010] In one possible implementation, the first candidate model and the second candidate model both belong to a candidate model set, which is a set of multiple candidate models with different structures obtained by performing structural changes based on the same basic model. The multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed across multiple computing nodes.

[0011] The iterative optimization of the multiple first candidate model substructures and the multiple second candidate model substructures is performed. During the optimization process, a weight sharing method is adopted so that the multiple first candidate model substructures and the multiple second candidate model substructures form a super network, including:

[0012] The substructures of all candidate models in the candidate model set are iteratively optimized. During the optimization process, a weight sharing method is used to make the substructures of all candidate models in the candidate model set form a super network.

[0013] In one possible implementation, the iterative optimization of the substructures of all candidate models in the candidate model set includes:

[0014] The Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

[0015] In one possible implementation, the above-mentioned Bayesian optimization algorithm is used to perform the i-th optimization on the substructure of all candidate models in the candidate model set, including: using the Bayesian optimization algorithm, based on the performance of the 1st candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructure of all candidate models in the candidate model set is iteratively optimized, and K is an integer greater than 1; inputting the training data set into multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; and updating the weight of the hypernetwork based on the gradient corresponding to the i-th candidate model.

[0016] It can be understood that the above-mentioned first candidate model is the candidate model selected for training during the first optimization process of the substructure of all candidate models in the candidate model set, the above-mentioned second candidate model is the candidate model selected for training during the second optimization process of the substructure of all candidate models in the candidate model set,…, the above-mentioned i-1th candidate model is the candidate model selected for training during the i-1th optimization process of the substructure of all candidate models in the candidate model set.

[0017] In the above scheme, in each optimization process, the Bayesian optimization algorithm is used to predict from the super network a candidate model whose performance is better than the candidate model trained in the historical iteration. The candidate model is trained to obtain the corresponding gradient, and the weight of the super network is updated based on the gradient. Compared with the NAS algorithm, in each optimization process of the super network, N candidate models need to be trained to obtain the gradients corresponding to the N candidate models, and then the weight of the super network is updated based on the gradients corresponding to the N candidate models. This can improve the optimization efficiency of the super network.

[0018] In one possible implementation, the Bayesian optimization algorithm is used to predict, based on the performance of the first candidate model to the performance of the (i-1)th candidate model, the i-th candidate model having better performance than the performance of the first candidate model to the performance of the (i-1)th candidate model from the hypernetwork, including:

[0019] The 1st candidate model to the i-1th candidate model are divided according to performance to obtain a first set and a second set, the first set includes candidate models whose performance is better than the first performance among the 1st candidate model to the i-1th candidate model, and the second set includes candidate models whose performance is lower than or equal to the first performance among the 1st candidate model to the i-1th candidate model; multiple first numbers and multiple second numbers are counted, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, the i-th candidate model with performance better than the performance of the 1st candidate model to the i-1th candidate model is predicted from the hypernetwork.

[0020] By implementing the above implementation, it is possible to predict from the hypernetwork an i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0021] In one possible implementation, the method of predicting, based on the multiple first coefficients and the multiple second coefficients, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, includes:

[0022] Based on multiple first numbers, multiple first frequencies are obtained, wherein the multiple first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set; based on multiple second numbers, multiple second frequencies are obtained, wherein the multiple second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set; based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

[0023] In one possible implementation, the method of predicting, based on the multiple first frequencies and the multiple second frequencies, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model includes:

[0024] Calculate the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, where the quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0025] In one possible implementation, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0026] In a possible implementation, the performance includes at least one of running time, accuracy, and parameter quantity.

[0027] In a possible implementation, the basic model is a Transformer network, or the basic model is a convolutional neural network (CNN).

[0028] In a possible implementation, the plurality of computing nodes include any one or more of a GPU, a TPU, and an NPU.

[0029] In one possible implementation, after completing the iterative optimization of the substructures of all candidate models in the candidate model set, the method further includes:

[0030] The Kth candidate model is sent to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

[0031] In a second aspect, a model super network optimization device is provided, the device comprising:

[0032] a segmentation module, configured to segment the first candidate model to obtain a plurality of first candidate model substructures, and segment the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by structurally changing the same base model;

[0033] a distribution module, configured to distribute the plurality of first candidate model substructures among a plurality of computing nodes, and to distribute the plurality of second candidate model substructures among the plurality of computing nodes;

[0034] An optimization module is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to enable the multiple first candidate model substructures and the multiple second candidate model substructures to form a super network.

[0035] In one possible implementation, the first candidate model and the second candidate model both belong to a candidate model set, where the candidate model set is a set of multiple candidate models with different structures obtained by performing structural changes based on the same basic model, and multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed among the multiple computing nodes.

[0036] The optimization module is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is adopted to enable the substructures of all candidate models in the candidate model set to form a super network.

[0037] In a possible implementation, the optimization module is specifically configured to:

[0038] A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

[0039] In a possible implementation, the optimization module is specifically configured to:

[0040] Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the (i-1)th candidate model, predict from the hypernetwork an i-th candidate model whose performance is better than the performance of the first candidate model to the performance of the (i-1)th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1;

[0041] Inputting a training data set into the plurality of computing nodes to train the i-th candidate model, and obtaining a gradient corresponding to the i-th candidate model;

[0042] Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

[0043] In a possible implementation, the optimization module is specifically configured to:

[0044] The 1st candidate model to the i-1th candidate model are divided according to performance to obtain a first set and a second set, the first set includes candidate models whose performance is better than the first performance among the 1st candidate model to the i-1th candidate model, and the second set includes candidate models whose performance is lower than or equal to the first performance among the 1st candidate model to the i-1th candidate model; multiple first numbers and multiple second numbers are counted, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, the i-th candidate model with performance better than the performance of the 1st candidate model to the i-1th candidate model is predicted from the hypernetwork.

[0045] In a possible implementation, the optimization module is specifically configured to:

[0046] Based on multiple first numbers, multiple first frequencies are obtained, wherein the multiple first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set; based on multiple second numbers, multiple second frequencies are obtained, wherein the multiple second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set; based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

[0047] In a possible implementation, the optimization module is specifically configured to:

[0048] Calculate the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, where the quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0049] In one possible implementation, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0050] In a possible implementation, the performance includes at least one of running time, accuracy, and parameter quantity.

[0051] In a possible implementation, the above basic model is a Transformer network, or the basic model is a CNN.

[0052] In a possible implementation, the plurality of computing nodes include any one or more of a GPU, a TPU, and an NPU.

[0053] In a possible implementation, the apparatus further includes a sending module;

[0054] After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

[0055] In a third aspect, a management node is provided, comprising a processor and a memory; the processor is used to execute instructions stored in the memory, so that the management node implements the method provided in the first aspect or any possible implementation of the first aspect.

[0056] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and the instructions are used to implement the method provided in the first aspect or any possible implementation of the first aspect.

[0057] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when read and executed by a computing device, enables the computing device to perform the method provided in the first aspect or any possible implementation of the first aspect.

[0058] In the sixth aspect, a computing system is provided, which includes a management node and multiple computing nodes, the management node is used to implement the method provided in the above-mentioned first aspect or any possible implementation of the first aspect, and the multiple computing nodes are used to undertake the training operation of the candidate model in the method provided in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] FIG1 is a schematic diagram of a standard structure of a Transformerblock provided in an embodiment of the present application;

[0060] FIG2 is a schematic diagram of different models weight sharing and merging into one model provided in an embodiment of the present application;

[0061] FIG3 is a schematic diagram of the structure of an artificial intelligence system provided in an embodiment of the present application;

[0062] FIG4 is a schematic diagram of the NAS algorithm provided in an embodiment of the present application performing the i-th optimization on a super network;

[0063] FIG5 is a flow chart of an AI model super-network optimization method provided in an embodiment of the present application;

[0064] FIG6A is a schematic diagram of synthesizing a hypernetwork of multiple candidate models provided by an embodiment of the present application;

[0065] FIG6B is a schematic diagram of performing the i-th optimization on a supernetwork according to an embodiment of the present application;

[0066] FIG7 is a schematic diagram of a process for performing the i-th optimization on a supernetwork according to an embodiment of the present application;

[0067] FIG8 is a schematic diagram of a process for predicting an i-th candidate model using a Bayesian optimization algorithm according to an embodiment of the present application;

[0068] FIG9 is a schematic structural diagram of an AI model super network optimization device provided in an embodiment of the present application;

[0069] FIG10 is a schematic structural diagram of a management node provided in an embodiment of the present application;

[0070] FIG11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0071] FIG12 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application;

[0072] FIG13 is a schematic diagram of the structure of a computing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0074] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0075] In order to facilitate a clear understanding of the technical solution provided by this application, some nouns and terms involved in the technical solution provided by this application are first explained.

[0076] (1) Neural Network

[0077] It can be composed of neural units (also called neurons), which can refer to the variables x s and intercept b as inputs, the output of the operation unit can be:

[0078] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s is x s The weight of the neural unit, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer. The activation function can be a sigmoid function and other functions, which are not limited here. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0079] (2) Convolutional neural network (CNN)

[0080] A CNN is a deep neural network with a convolutional architecture. A convolutional neural network consists of a feature extractor consisting of convolutional layers and subsampling layers, which can be considered a filter. A convolutional layer is a layer of neurons that performs convolution processing on the input signal. In a convolutional layer of a convolutional neural network, a neuron can only connect to some of the neurons in adjacent layers. A convolutional layer typically contains several feature planes, each of which can be composed of a rectangular arrangement of neurons. Neurons in the same feature plane share weights, which are called convolution kernels. Shared weights can be understood as a position-independent method for extracting image information.

[0081] Convolution kernels can be initialized as matrices of random size, and during the training process of the convolutional neural network, the convolution kernels can be learned to obtain reasonable weights. In addition, the direct benefit of shared weights is that they reduce the number of connections between the layers of the convolutional neural network, while also reducing the risk of overfitting.

[0082] (3) Transformer Network

[0083] The Transformer network, also known as the Transformer model, is a multi-layer neural network based on self-attention modules. Currently primarily used for natural language processing tasks, the Transformer network consists of one or more layers of Transformer modules (blocks). The structure of a standard Transformer block is shown in Figure 1, where the encoder is on the left and the decoder is on the right. Each encoder can include any number of encoding submodules, each of which includes a multi-head self-attention module and a feedforward neural network. Similarly, each decoder can include any number of decoding submodules, each of which includes two multi-head self-attention modules and a feedforward neural network. The number of encoding submodules and decoding submodules can be different. FFN can also be called a multilayer perceptron (MLP).

[0084] (4) Loss function

[0085] In the process of training a neural network, because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the desired target value, and then update the weight matrix of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network). For example, if the network's predicted value is too high, the weight matrix is ​​adjusted to make it predict a lower value, and this adjustment is continued until the neural network can predict the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function or objective function, which is an important equation used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.

[0086] (5) Backpropagation algorithm

[0087] During neural network training, the back propagation (BP) algorithm can be used to correct the size of the initial model parameters, reducing the model's reconstruction error loss. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the initial model parameters, thereby converging the error loss. The BP algorithm is a backpropagation movement driven by error loss, aiming to obtain the optimal model parameters, such as the weight matrix.

[0088] (6) Candidate model refers to a model obtained by structurally changing the basic model, wherein the basic model can be any model such as CNN, deep neural network, Transformer model, etc. Taking the basic model as the Transformer block shown in Figure 1 as an example, for example, the number of heads in the encoding submodule or the multi-head self-attention module in the decoding submodule in the Transformer block can be changed to obtain candidate models with different structures.

[0089] (7) Model supernetwork, also known as supernetwork or supermodel, is a model that is formed by merging multiple candidate models with different structures after structural changes are made to the basic model. This model is called a supernetwork, and the merging method can be weight sharing. Taking model A and model B shown in Figure 2 as an example, when model A and model B are combined into a supernetwork C, the same structure in model A and model B (i.e., the structure circled by the dotted line in Figure 2) is merged. The merging here is weight sharing. By weight sharing, the number of parameters that need to be optimized can be reduced when optimizing the supernetwork.

[0090] The following introduces the application scenarios of the embodiments of the present application.

[0091] As shown in FIG3 , embodiments of the present application can be applied to the artificial intelligence system shown in FIG3 . The artificial intelligence system includes a local device 100 and a server 200 . The local device 100 and the server 200 can be connected via a communication network using any communication mechanism or communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0092] The user can operate the local device 100 and submit the super network optimization task to the server 200 through the communication network.

[0093] Among them, the local device 100, which can also be called the terminal device 100, can be any computing device, such as a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a virtual reality (VR) device, a smart car or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0094] The server 200 can be a computing device or a computing device cluster composed of multiple computing devices. The computing device may include but is not limited to physical servers, virtual machines, containers, chips, etc., among which physical servers may include but are not limited to ARM servers, X86 servers, etc.; a virtual machine refers to a complete computer system with complete hardware system functions implemented by network function virtualization (NFV) technology and simulated by software, running in a completely isolated environment; a container refers to a group of processes that are subject to resource constraints and isolated from each other, and is not specifically limited here.

[0095] Server 200 is responsible for allocating hypernetwork optimization tasks to a resource pool consisting of computing devices or clusters of computing devices, such as CPUs, GPUs, TPUs, NPUs, etc., providing them with corresponding computing power, storage space, or various software services. When server 200 is a cluster of computing devices, it can be a cloud service provider's central cloud data center or an edge data center provided by the cloud service provider to users. After completing the hypernetwork optimization, server 200 returns the most suitable candidate model determined from the hypernetwork to the user.

[0096] The user can deploy the most suitable candidate model on the local device 100. The local device 100 with the most suitable candidate model deployed can be used in a variety of scenarios. For example, the local device 100 can be used for identity authentication using facial recognition technology; as another example, the local device 100 can be used for natural language understanding; as another example, the local device 100 can be used for augmented reality in games.

[0097] However, different application scenarios have different requirements for neural networks. For example, when the local device 100 is used for identity authentication through facial recognition technology, a more accurate neural network is required, that is, a more accurate candidate model is required; when the local device 100 is used for augmented reality in games, a more responsive neural network is required, that is, a more responsive candidate model is required.

[0098] Furthermore, different types of local devices 100 may have different requirements for neural networks. For example, different local devices 100 may have different computational latencies for the same neural network due to differences in underlying hardware. This difference in type can refer to different types of local devices 100, such as a smartphone versus a tablet, or different models of the same type of local device 100, such as different models of mobile phones.

[0099] Therefore, a suitable candidate model may be selected for a specific local device 100 based on the type of the local device 100 , the application scenario, and other possible restrictions.

[0100] In the artificial intelligence system shown in FIG3 , the current server 200 typically uses a super network optimization algorithm such as the NAS algorithm to determine a suitable candidate model from a large number of candidate models. The specific process of the server 200 using the NAS algorithm to optimize the super network is as follows: the server 200 receives the basic model (CNN, deep neural network, Transformer network, etc.), the data set (including the training data set and the test data set), and the variation range of the basic model (including the structural parameters to be optimized in the basic model and the value variation range of the structural parameters to be optimized) uploaded by the user operating the local device 100. After receiving the above data, the server 200 performs structural changes on the basic model according to the variation range of the above basic model to obtain multiple candidate models with different structures, assuming there are M candidate models. After that, all M candidate models are deployed to N computing nodes (such as N GPUs). Finally, the N*M candidate models on the N computing nodes are iteratively optimized. Specifically, during the optimization process, a weight sharing method is first used to form a super network for the M candidate models on each computing node, obtaining N super networks corresponding to the N computing nodes, and then the N super networks on the N computing nodes are iteratively optimized. Wherein, M and N are both integers greater than 1.

[0101] Next, with reference to FIG4 , the specific process of iteratively optimizing N supernetworks on N computing nodes using the NAS algorithm is described.

[0102] Taking the i-th optimization as an example, as shown in Figure 4, where the computing nodes are GPUs, the reinforcement learning / evolutionary algorithm is used to select N candidate models from the supernetwork and activate these N candidate models in the N supernetworks on the N computing nodes (there is a one-to-one correspondence between the N computing nodes and the N candidate models). Then, the test dataset is input into each computing node to test the performance of the candidate model activated on each computing node (such as running time, accuracy, or number of parameters, etc. In Figure 4, the performance is accuracy, represented by Acc). The training dataset is also input into each computing node to train the candidate model activated on each computing node to obtain gradients. Finally, the performance of the N candidate models is fed back to the reinforcement learning / evolutionary algorithm. Based on the performance of these N candidate models, the reinforcement learning / evolutionary algorithm selects N candidate models from the supernetwork for the i+1-th optimization, fuses the N gradients, and updates the weights of the N supernetworks on the N computing nodes according to the fused gradients. Wherein, 1≤i≤K, K is the number of times the N supernetworks are iteratively optimized, and both K and N are integers greater than 1.

[0103] After completing K iterative optimizations of N*M candidate models on N computing nodes, based on the performance of the N candidate models tested in the K-th optimization process, the candidate model with the best performance among the N candidate models is provided to the user as the most suitable candidate model.

[0104] In response to the problem in the above-mentioned supernetwork optimization method that each computing node needs to carry the complete supernetwork and the carrying capacity of the computing node is limited, the computing node cannot optimize the large model supernetwork. This application provides an AI model supernetwork optimization method.

[0105] In the AI ​​model super network optimization method provided in the present application, after structural changes are made based on the same basic model to obtain multiple candidate models of different structures, each of the multiple candidate models is divided into multiple substructures, and the multiple substructures corresponding to each candidate model are distributed in multiple computing nodes. The multiple substructures of each candidate model are in a one-to-one correspondence with the multiple computing nodes. Then, the substructures of the multiple candidate models are iteratively optimized. Specifically, in the optimization process, a weight sharing method is first adopted to make the substructures of all candidate models on multiple computing nodes form a super network, and then the super network is iteratively optimized. It can be seen that in the AI ​​model super network optimization method provided in the present application, each computing node only needs to carry one substructure of each candidate model in the multiple candidate models, and does not need to carry the complete super network. The complete super network is carried by multiple computing nodes together, and the super network is subsequently optimized by multiple computing nodes together, thereby solving the problems existing in the above-mentioned NAS algorithm.

[0106] In order to facilitate a clearer understanding of the AI ​​model super network optimization method provided in this application, a detailed description is given below in conjunction with the flow chart shown in Figure 5 and Figure 6A.

[0107] As shown in FIG5 , the method includes the following steps:

[0108] S501: Based on the same basic model, structural changes are performed to obtain M candidate models with different structures, as shown in the left part of FIG6A .

[0109] S502: Split each of the M candidate models to obtain multiple substructures of each candidate model, and distribute the multiple substructures of each candidate model to multiple computing nodes, as shown in the middle part of FIG6A .

[0110] Specifically, the multiple substructures of each candidate model correspond one to one with the multiple computing nodes. That is, each computing node bears one substructure of each candidate model, and a total of M substructures corresponding to M candidate models need to be borne.

[0111] S503: Iteratively optimize all substructures of the M candidate models on multiple computing nodes. During the optimization process, a weight sharing method is used to form all substructures of the M candidate models into a super network, as shown in the right part of FIG6A .

[0112] The following is a detailed introduction to the various steps shown in FIG5 .

[0113] In S501, the basic model can be any neural network such as CNN, deep neural network, Transformer network, etc., and this application does not make any specific limitations.

[0114] Specifically, the base model can be structurally modified based on the range of changes in the base model submitted by the user (including the structural parameters to be optimized in the base model and the range of changes in the values ​​of the structural parameters to be optimized), thereby obtaining M candidate models with different structures. Taking any two candidate models from the M candidate models as an example, the candidate models with different structures generally have some identical structures and some different structures.

[0115] Taking the basic model as Transformer network A including m layers of Transformer blocks (as shown in Figure 1) as an example, assuming that the structural parameters to be optimized in Transformer network A include the hidden dimension of each layer of Transformer block in Transformer network A, the multi-layer perceptron ratio (MLP ratio) and the head number of the multi-head self-attention module, it can be seen that there are 3*m structural parameters to be optimized in Transformer network A. Assuming that the value range of the hidden dimension of each layer of Transformer block is {9216, 12288, 15360}, the value range of the MLP ratio is {3, 3.5, 4}, and the value range of the head number is {64, 96, 128}, then we can get 3*9 m candidate models with different structures.

[0116] In S502, the multiple computing nodes can be multiple GPUs, multiple CPUs, multiple NPUs, multiple TPUs, or a part of them can be GPUs and the rest can be NPUs, a part of them can be NPUs and the rest can be TPUs, or a part of them can be GPUs, a part of them can be NPUs, and the rest can be TPUs, etc. This application does not specifically limit the types and quantities of the multiple computing nodes.

[0117] The manner of dividing the M candidate models into substructures may include at least one of tensor parallelism (TP) and pipeline parallelism (PP).

[0118] Tensor parallelism is a parallel computing method primarily used in fields like machine learning. During model training, tensor parallelism focuses on breaking down tensors into smaller parts and performing parallel computations on multiple compute nodes. This distributes the computational load across multiple nodes, improving computational efficiency and accelerating model training.

[0119] Pipeline parallelism is a parallel computing method primarily used in fields such as machine learning. Pipeline parallelism focuses on breaking down computational tasks (such as model training) into multiple stages (for example, for convolutional neural networks in machine learning, the computational task can be divided into stages such as forward propagation, backpropagation, and parameter update), and executing these stages in parallel across multiple compute nodes, thereby improving computational efficiency. Unlike tensor parallelism, pipeline parallelism focuses on the temporal distribution of computational tasks rather than the spatial distribution of tensors.

[0120] First, take the candidate model's splitting method as pipeline parallelism, and the candidate model as a Transformer network A including m layers of Transformer blocks as an example. Then, the candidate model A can be split (usually evenly split) based on the pipeline parallel number N to obtain N substructures, and then the N substructures are distributed on the N computing nodes. Specifically, the N substructures are in a one-to-one correspondence with the N computing nodes. In a possible embodiment, if m is not an integer multiple of N, then when splitting, the number of layers of the first N-1 substructures can be the same, and the number of layers of the Nth substructure can be less / more. For example, assuming m is 125 and N is 20, the number of layers of the first 19 substructures is 6, and the number of layers of the last substructure is 5. In a specific implementation, the above-mentioned pipeline parallel number N can be provided by the user or pre-set.

[0121] Next, taking the candidate model's splitting method as tensor parallelism and the candidate model as a Transformer network A including m layers of Transformer blocks as an example, the multi-head self-attention module in each layer of the Transformer block of the candidate model A can be split according to the number of heads, and the feedforward neural network in each layer of the Transformer block of the candidate model A can be split according to the product of the hidden dimension and the MLP ratio (usually evenly) to obtain Z substructures, and then the Z substructures are distributed on the Z computing nodes. Specifically, the Z substructures are in a one-to-one correspondence with the Z computing nodes. In a possible embodiment, if the number of heads and / or the product of the hidden dimension and the MLP ratio is not an integer multiple of Z, then when splitting, the number of heads and the product of the hidden dimension and the MLP ratio of the first N-1 substructures can be the same, and the number of heads and the product of the hidden dimension and the MLP ratio of the Nth substructure can be less / more, or the candidate model can be deleted from the M candidate models and not split. In a specific implementation, the pipeline parallel number Z can be provided by the user or preset.

[0122] Taking the candidate model's splitting method as tensor parallelism, and the candidate model as a CNN including m convolutional layers (a convolutional layer may include many convolution kernels) as an example, for each convolutional layer, the multiple convolution kernels included in the convolutional layer can be split according to the number of convolution kernels based on the tensor parallel number Z (usually evenly split) to obtain Z substructures, and then the Z substructures are distributed on the Z computing nodes. Specifically, the Z substructures are in a one-to-one correspondence with the Z computing nodes. In a possible embodiment, if the number of convolution kernels included in the convolutional layer is not an integer multiple of Z, then when splitting, the number of convolution kernels of the first N-1 substructures can be the same, and the number of convolution kernels of the Nth substructure can be less / more.

[0123] Taking the candidate model splitting method as a combination of pipeline parallelism and tensor parallelism, and the candidate model being a Transformer network A including m-layer Transformer blocks as an example, the candidate model A can be split based on the pipeline parallel number N and the tensor parallel number Z. Specifically, the candidate model A can be split based on the pipeline parallel number N to obtain N substructures (the splitting method is the same as above), and then the candidate model A can be split based on the tensor parallel number Z to obtain Z substructures (the splitting method is the same as above). Then, the first substructure in the N substructures and the first substructure in the Z substructures are distributed on the first computing node, and the second substructure in the N substructures and the second substructure in the Z substructures are distributed on the second computing node, and so on.

[0124] In S503 , specifically, a Bayesian optimization algorithm or the like may be used to iteratively optimize all substructures of the M candidate models on multiple computing nodes.

[0125] Taking the i-th optimization of all substructures of M candidate models using the Bayesian optimization algorithm as an example, please refer to Figures 6B and 7.

[0126] S701: Using the Bayesian optimization algorithm, the performance of the first candidate model P1 to the performance of the i-1 candidate model P i-1 , the performance predicted from the hypernetwork is better than the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 The i-th candidate model is shown in Figure 6B.

[0127] Wherein, 1≤i≤K, K is the number of times all substructures in the M candidate models are iteratively optimized, and K is an integer greater than 1.

[0128] The performance of the first candidate model P1 to the performance of the i-1 candidate model P i-1 The performance may be obtained by testing using the same test data set, and may include at least one of accuracy, running time, and number of parameters.

[0129] It should be noted that the super network here is the super network after the first i-1 optimizations.

[0130] For the specific implementation process of S701, please refer to the relevant description of Figure 8 below.

[0131] S702: Input the training data set into multiple computing nodes to train the i-th candidate model and obtain the gradient G corresponding to the i-th candidate model. i , as shown in Figure 6B.

[0132] S703: Based on the gradient G corresponding to the i-th candidate model iThe weights of the hypernetwork are updated, as shown in FIG6B .

[0133] It can be understood that since the i-th candidate model is based on the performance P1 of the 1st candidate model to the performance P of the i-1th candidate model i-1 , the performance predicted from the hypernetwork is better than the performance of the first candidate model P1 to the performance of the i-1 candidate model P i-1 candidate model, then the Kth candidate model is the candidate model with the best performance obtained after the iterative optimization of all substructures of the M candidate models. Therefore, in a specific embodiment of the present application, after completing the iterative optimization of all substructures of the M candidate models, the Kth candidate model can be sent to the user's local device 100 as a model that can be deployed on the user's local device 100, so that the user can deploy the Kth candidate model on the local device 100.

[0134] It can be understood that after the user deploys the K-th candidate model on the local device 100, the K-th candidate model can be used to process user data. For example, if the K-th candidate model is a Transformer network for natural language understanding, then the K-th candidate model can be used to recognize the text data input by the user and output the recognized text data. For another example, if the K-th candidate model is a CNN for face recognition, then the K-th candidate model can be used to recognize the face in the face image input by the user. For another example, if the K-th candidate model is a deep neural network for object classification, then the K-th candidate model can be used to classify the items in the object image input by the user.

[0135] Referring to FIG8 , FIG8 is a schematic diagram of a process for predicting the i-th candidate model using a Bayesian optimization algorithm according to an embodiment of the present application. As shown in FIG8 , the process includes the following steps:

[0136] S801: Based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 , the 1st candidate model to the i-1th candidate model are divided to obtain a first set and a second set, wherein the first set includes candidate models from the 1st candidate model to the i-1th candidate model whose performance is better than the first performance, and the second set includes candidate models from the 1st candidate model to the i-1th candidate model whose performance is lower than or equal to the first performance.

[0137] The first performance can be based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model. i-1 Calculated, such as the first performance is the performance of the first candidate model P1 to the performance of the i-1 candidate model P i-1Optionally, the first performance can also be customized according to the actual scenario, which is not specifically limited in this application.

[0138] It should be noted that when i is equal to 1, since there is no historical performance of candidate models, the first candidate model can be randomly selected from the hypernetwork. When i is equal to 2, the second candidate model can also be randomly selected from the hypernetwork.

[0139] S802: Count multiple first numbers and multiple second numbers, where the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters that need to be optimized in the basic model.

[0140] The structural parameters in the structural parameter set, such as the hidden dimension, MLP ratio, and head number of each Transformer block in the Transformer network A in the example above.

[0141] S803: Based on the multiple first numbers and the multiple second numbers, predict from the hypernetwork the performance that is better than the performance P1 of the first candidate model to the performance P of the i-1th candidate model. i-1 The i-th candidate model.

[0142] Specifically, S803 may include steps S8031 - S8033 shown in FIG8 .

[0143] S8031: Obtain a plurality of first frequencies based on the plurality of first order numbers, wherein the plurality of first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set.

[0144] S8032: Obtain a plurality of second frequencies based on the plurality of second numbers, wherein the plurality of second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set.

[0145] S8033: Based on the multiple first frequencies and the multiple second frequencies, predict from the hypernetwork the performance that is better than the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 The i-th candidate model.

[0146] Furthermore, S8033 can be implemented by referring to the following method (1) or method (2):

[0147] Method (1) calculates the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, and then determines the value with the largest quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model. The quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to the value.

[0148] Method (2) calculates the difference corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, and then determines the value with the largest corresponding difference among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model. The difference corresponding to each value is the difference between the first frequency corresponding to each value and the second frequency corresponding to the value.

[0149] The following takes the implementation of S8033 through the above-mentioned method (1) as an example to describe the implementation process of the embodiment shown in FIG8 in combination with a specific example.

[0150] In S801, assuming that i is 5, the performance is accuracy, the performance P1 of candidate model 1 (referring to the first candidate model) is 0.52, the performance P2 of candidate model 2 (referring to the second candidate model) is 0.55, the performance P3 of candidate model 3 (referring to the third candidate model) is 0.65, and the performance P4 of candidate model 4 (referring to the fourth candidate model) is 0.68. The first performance is the average value of P1 to P4, which is 0.6. Then, based on the performance of these four candidate models, the first set obtained by dividing these four candidate models includes candidate model 3 and candidate model 4, and the second set obtained includes candidate model 1 and candidate model 2.

[0151] In S802, the value of the jth structural parameter in the structural parameter set is V j1 、V j2 and V j3 There are 3 values ​​in total. Take the first and second times of these 3 values ​​as an example. Assume that the value of the j-th structural parameter in candidate model 1 is V j1 , the value of the j-th structural parameter in candidate model 2 is also V j1 , the value of the j-th structural parameter in candidate model 3 is V j3 , the value of the j-th structural parameter in candidate model 4 is V j2 , then we can get V j1 The number of times it appears in the first set (including candidate model 3 and candidate model 4) is 0, and the number of times it appears in the second set (including candidate model 1 and candidate model 2) is 2, V j2 The number of times it appears in the first set is 1, and the number of times it appears in the second set is 0. j3The number of occurrences in the first set is 1, and the number of occurrences in the second set is 0.

[0152] To avoid the following calculation of V j1 、V j2 and V j3 The corresponding quotient is infinite (that is, the dividend is not 0 and the divisor is 0), which can be calculated in the statistical V j1 、V j2 and V j3 Before the number of times each value appears in the first set and the number of times each value appears in the second set are initialized to a fixed value, such as 1 or 2. Taking initialization as 1 as an example, the V j1 The number of times it appears in the first set is 1, and the number of times it appears in the second set is 3. j2 The number of times it appears in the first set is 2, and the number of times it appears in the second set is 1. j3 The number of times it appears in the first set is 2, and the number of times it appears in the second set is 1. That is, j1 corresponds to the first number 1 and the second number 3, j2 corresponds to the first number 2 and the second number 1, V j3 The corresponding number is 2 for the first time and 1 for the second time.

[0153] In S803, based on V j1 The corresponding first number 1 can be calculated to get V j1 The corresponding first frequency = 1 / (1+2+2) = 1 / 5;

[0154] Based on V j1 The corresponding second number 3 can be calculated to get V j1 The corresponding second frequency = 3 / (3+1+1) = 3 / 5;

[0155] Based on V j2 The corresponding first number 2 can be calculated to get V j2 The corresponding first frequency = 2 / (1+2+2) = 2 / 5;

[0156] Based on V j2 The corresponding second number 1 can be calculated to get V j2 The corresponding second frequency = 1 / (3+1+1) = 1 / 5;

[0157] Based on V j3 The corresponding first number 2 can be calculated to get V j3 The corresponding first frequency = 2 / (1+2+2) = 2 / 5;

[0158] Based on V j3 The corresponding second number 1 can be calculated to get Vj3 The corresponding second frequency=1 / (3+1+1)=1 / 5.

[0159] Next, based on the above V j1 The corresponding first frequency and second frequency can be calculated to get V j1 The corresponding quotient = (1 / 5) / (3 / 5) = 1 / 3;

[0160] Based on the above V j2 The corresponding first frequency and second frequency can be calculated to get V j2 The corresponding quotient = (2 / 5) / (1 / 5) = 2;

[0161] Based on the above V j3 The corresponding first frequency and second frequency can be calculated to get V j3 The corresponding quotient = (2 / 5) / (1 / 5) = 2.

[0162] Since V j2 and V j3 The corresponding quotient is the same and the largest, so V j2 or V j3 Determine the value of the j-th structural parameter in the 5th candidate model.

[0163] It can be understood that if the process shown in the above example is also performed on other structural parameters in the structural parameter set, the value of the structural parameter set in the 5th candidate model can be determined. Once the value of the structural parameter set in the 5th candidate model is determined, the 5th candidate model can be determined.

[0164] It should be noted that the implementation process of S803 described above is merely an example. In a specific implementation, the implementation process of S803 may be other, and this application does not impose any specific limitations thereon. For example, the quotient of the first and second quotients corresponding to multiple values ​​of the j-th structural parameter in the structural parameter set may be compared, and the value with the largest quotient among the multiple values ​​may be determined as the value of the j-th structural parameter in the i-th candidate model.

[0165] It should also be noted that the embodiment shown in FIG8 is only based on the performance P1 of the first candidate model to the performance P1 of the i-1th candidate model. i-1 A possible embodiment of predicting the i-th candidate model. In another possible embodiment, a regression model (such as linear regression, ridge regression, or support vector regression) or a method based on domain knowledge rules can be used to predict the i-th candidate model based on the performance of the first candidate model to the performance of the i-1-th candidate model.

[0166] After the i-th candidate model is determined, the i-th candidate model may be activated in multiple computing nodes, that is, multiple substructures of the i-th candidate model distributed in multiple computing nodes may be activated, and S702 - S703 may be executed.

[0167] It should be noted that the embodiment of FIG5 is described as an example in which the way of dividing the M candidate models includes at least one of tensor parallelism and pipeline parallelism. In one possible embodiment, when the number of computing nodes is large enough, the way of dividing the M candidate models may also include data parallelism (DP). Data parallelism is a parallel computing method mainly used in fields such as machine learning. Data parallelism focuses on decomposing the training data set into multiple batches for parallel computing, thereby improving computing efficiency.

[0168] Taking the candidate model's segmentation method as pipeline parallelism and data parallelism, and the candidate model as a Transformer network A including m layers of Transformer blocks (as shown in Figure 1) as an example, the candidate model A can be segmented (usually evenly segmented) based on the pipeline parallel number N to obtain N substructures, and then the N substructures are distributed on N computing nodes. Specifically, the N substructures are in a one-to-one correspondence with the N computing nodes. In one possible embodiment, if m is not an integer multiple of N, then when segmenting, the first N-1 substructures can have the same number of layers, and the Nth substructure can have fewer layers. For example, assuming m is 125 and N is 20, the first 19 substructures have 6 layers, and the last substructure has 5 layers. Taking the data parallel number T (T is an integer greater than 1) as an example, assuming that N computing nodes are numbered 1 to N, each numbered computing node also has T-1 replicated computing nodes with the same number, that is, the computing node numbered 1 has T-1 replicated computing nodes with the same number, the computing node numbered 2 has T-1 replicated computing nodes with the same number, ..., the computing node numbered N has T-1 replicated computing nodes with the same number, and the T-1 replicated computing nodes with the same number are used to replicate the candidate model substructure on the computing node with the same number. In a specific implementation, the pipeline parallel number N and data parallel number T can be provided by the user or pre-set, and T is an integer greater than 1.

[0169] Then, a data parallel method can be used to iteratively optimize all candidate model substructures on T groups of T*N computing nodes labeled 1 to N. Specifically, during the optimization process, a weight sharing method can be used to form a super network of all candidate model substructures on each group of computing nodes in the T groups of computing nodes, thereby forming T identical super networks, and then the data parallel method can be used to iteratively optimize the T identical super networks.

[0170] The specific process of iteratively optimizing T identical hypernetworks using the data parallel method is as follows: the training dataset is divided into T batches of training data, and then the T batches of training data are used to perform parallel iterative optimization on the T hypernetworks, where the T hypernetworks and the T batches of training data are in a one-to-one correspondence.

[0171] Taking T supernetworks as supernetwork 1, supernetwork 2, and supernetwork 3, and performing the i-th optimization on these three supernetworks as an example, the implementation method can be the following method 1 or method 2.

[0172] In method 1, the i-th optimization of supernetwork 1, the i-th optimization of supernetwork 2, and the i-th optimization of supernetwork 3 are not independent, as follows:

[0173] The first step is to input the test data set into the N computing nodes where the hypernetwork 1 is located to test the 1.i candidate model and obtain the performance P of the 1.i candidate model. 1.i , input the test data set into the N computing nodes where the hypernetwork 2 is located to test the 2.i candidate model, and obtain the performance P of the 2.i candidate model 2.i , and input the test data set into the N computing nodes where the hypernetwork 3 is located to test the 3.i candidate model, and obtain the performance P of the 3.i candidate model 3.i , where the 1.i candidate model is based on the Bayesian optimization algorithm, based on the performance P of the 1.1 candidate model 1.1 The performance P of the 1.i-1 candidate model 1.i-1 , the performance of the 2.i candidate model P 2.i And the performance P of the 3.i candidate model 3.i , the performance predicted from the hypernetwork is better than the performance of the 1.1 candidate model P 1.1 The performance P of the 1.i-1 candidate model 1.i-1 , the performance of the 2.i candidate model P 2.i And the performance P of the 3.i candidate model 3.i candidate models.

[0174] Here, "1st." represents the first group of computing nodes, i.e., the N computing nodes where supernetwork 1 is located; "2nd." represents the first group of computing nodes, i.e., the N computing nodes where supernetwork 1 is located; and "3rd." represents the third group of computing nodes, i.e., the N computing nodes where supernetwork 3 is located. The 2.ith and 3.ith candidate models may be randomly selected from the supernetwork. Optionally, the 2.ith and 3.ith candidate models may be the same candidate model as the 1.ith candidate model, or one of the 2.ith and 3.ith candidate models may be the same candidate model as the 1.ith candidate model, and the other may be randomly selected from the supernetwork.

[0175] Using the Bayesian optimization algorithm, the method for predicting the 1.i-th candidate model is similar to the method for predicting the i-th candidate model shown in Figure 8. Please refer to the relevant description of Figure 8. For the sake of brevity, it will not be elaborated here.

[0176] The second step is to input the first batch of training data into the N computing nodes where the hypernetwork 1 is located to train the 1.i candidate model and obtain the gradient G corresponding to the 1.i candidate model. 1.i , input the second batch of training data into the N computing nodes where the hypernetwork 2 is located to train the 2.i candidate model, and obtain the gradient G corresponding to the 2.i candidate model 2.i , input the third batch of training data into the N computing nodes where the hypernetwork 3 is located to train the 3.i candidate model, and obtain the gradient G corresponding to the 3.i candidate model 3.i .

[0177] Step 3: Based on G 1.i , G 2.i and G 3.i The weights of hypernetwork 1, hypernetwork 2, and hypernetwork 3 are updated.

[0178] Specifically, if any two candidate models among the above-mentioned 1.i candidate models, 2.i candidate models and 3.i candidate models, or all candidate models, have the same structure that shares the same weight, the gradients corresponding to the weights shared by the same structure can be averaged, and then the weights shared by the above-mentioned same structures in supernetwork 1, supernetwork 2 and supernetwork 3 can be synchronously updated based on the gradient average.

[0179] After the iterative optimization of supernetwork 1, supernetwork 2 and supernetwork 3 is completed through the above-mentioned method 1, the 1.Kth candidate model can be sent to the user's local device 100 as a model that can be deployed on the user's local device 100, so that the user can deploy the 1.Kth candidate model on the local device 100 and then use the 1.Kth candidate model to process user data.

[0180] In Method 2, the i-th optimization of hypernetwork 1, hypernetwork 2, and hypernetwork 3 is independent. That is, the i-th optimization of these three hypernetworks does not affect each other. The specific process for performing the i-th optimization of hypernetwork 1, hypernetwork 2, and hypernetwork 3 is the same as the i-th optimization of a hypernetwork described in S701-S703 above. For the sake of brevity, please refer to the description of S701-S703 above and will not be further elaborated here.

[0181] After the iterative optimization of supernetwork 1, supernetwork 2 and supernetwork 3 is completed through method 2, the performance of the three K-th candidate models can be compared, and the K-th candidate model with the best performance can be used as a model that can be deployed on the user's local device 100 and sent to the user's local device 100, so that the user can deploy it on the local device 100 and use the K-th candidate model with the best performance to process user data.

[0182] It should be understood that the above-mentioned method 1 or method 2 is merely an example and should not be regarded as a specific limitation on the implementation method of iterative optimization of supernetwork 1, supernetwork 2 and supernetwork 3.

[0183] The AI ​​model super network optimization method provided by the present application is that after the basic model is structurally changed to obtain multiple candidate models of different structures, each candidate model is divided into multiple substructures of each candidate model, and the multiple substructures of each candidate model are distributed in multiple computing nodes (such as GPU, NPU, CPU, TPU, etc.). The multiple substructures of each candidate model are in a one-to-one correspondence with the multiple computing nodes. Then, the substructures of the multiple candidate models are iteratively optimized. Specifically, in the optimization process, a weight sharing method is first adopted to make the substructures of all candidate models on multiple computing nodes form a super network, and then the super network is iteratively optimized. It can be seen that each computing node only needs to carry one substructure of each candidate model in the multiple candidate models that make up the super network, and there is no need to carry the complete super network. The complete super network is carried by multiple computing nodes together, and subsequently the super network is optimized by multiple computing nodes together. Therefore, it can solve the problem that the computing nodes in the NAS algorithm cannot carry the large model super network and cannot optimize the large model super network.

[0184] Secondly, the AI ​​model super network optimization method provided in this application uses the Bayesian optimization algorithm to predict from the super network a candidate model with better performance than the candidate model trained in the historical iteration in each optimization process, trains the candidate model to obtain the corresponding gradient, and updates the weight of the super network based on the gradient. Compared with the NAS algorithm, in each optimization process of the super network, N candidate models need to be trained to obtain the gradients corresponding to the N candidate models, and then the weights of the super network are updated based on the gradients corresponding to the N candidate models, which can improve the optimization efficiency of the super network.

[0185] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, correspondingly, relevant devices and equipment for cooperating in implementing the above scheme are also provided below.

[0186] 9 , which is a schematic diagram of the structure of an AI model super network optimization device 900 provided in an embodiment of the present application. As shown in FIG9 , the device 900 includes: a segmentation module 901 , an allocation module 902 , and an optimization module 903 .

[0187] a segmentation module 901 configured to segment the first candidate model to obtain a plurality of first candidate model substructures, and segment the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by structurally changing the same base model;

[0188] a distribution module 902 for distributing the plurality of first candidate model substructures among a plurality of computing nodes, and distributing the plurality of second candidate model substructures among the plurality of computing nodes;

[0189] The optimization module 903 is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to enable the multiple first candidate model substructures and the multiple second candidate model substructures to form a super network.

[0190] In some possible implementations, the first candidate model and the second candidate model both belong to a candidate model set, which is a set of candidate models with different structures obtained by performing structural changes based on the same basic model. The multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed across multiple computing nodes.

[0191] The optimization module 903 is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is adopted to form a super network of the substructures of all candidate models in the candidate model set.

[0192] In some possible implementations, the optimization module 903 is specifically configured to iteratively optimize the substructures of all candidate models in the candidate model set using a Bayesian optimization algorithm.

[0193] In some possible implementations, the optimization module 903 is specifically used to: use a Bayesian optimization algorithm to predict, based on the performance of the 1st candidate model to the i-1th candidate model, an i-th candidate model whose performance is better than the performance of the 1st candidate model to the i-1th candidate model from the hypernetwork, 1≤i≤K, K is the number of times the substructure of all candidate models in the candidate model set is iteratively optimized, and K is an integer greater than 1; input the training data set into multiple computing nodes to train the i-th candidate model to obtain the gradient corresponding to the i-th candidate model; and update the weight of the hypernetwork based on the gradient corresponding to the i-th candidate model.

[0194] In some possible implementations, the optimization module 903 is specifically used to: divide the 1st candidate model to the i-1th candidate model according to performance to obtain a first set and a second set, the first set including candidate models from the 1st candidate model to the i-1th candidate model whose performance is better than the first performance, and the second set including candidate models from the 1st candidate model to the i-1th candidate model whose performance is lower than or equal to the first performance; counting multiple first numbers and multiple second numbers, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structural parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, predicting from the hypernetwork the i-th candidate model with performance better than the performance of the 1st candidate model to the i-1th candidate model.

[0195] In some possible implementations, the optimization module 903 is specifically configured to: obtain multiple first frequencies based on multiple first numbers, wherein the multiple first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set; obtain multiple second frequencies based on multiple second numbers, wherein the multiple second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set; and predict, based on the multiple first frequencies and the multiple second frequencies, obtain from the hypernetwork an i-th candidate model having performance better than the performance of the 1st candidate model to the i-1th candidate model.

[0196] In some possible implementations, the above-mentioned optimization module 903 is specifically used to: calculate the quotient corresponding to each value among multiple values ​​of the j-th structural parameter in the structural parameter set, where the quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0197] In some possible implementations, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0198] In some possible implementations, the aforementioned performance includes at least one of running time, accuracy, and parameter quantity.

[0199] In some possible implementations, the above-mentioned basic model is a Transformer network, or the basic model is a CNN, or the basic model is a deep neural network.

[0200] In some possible implementations, the multiple computing nodes include any one or more of GPU, TPU, NPU, and CPU.

[0201] In some possible implementations, as shown in FIG9 , the apparatus 900 further includes a sending module 904 ;

[0202] After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device. The Kth candidate model is used to process the user's text data and output the processed text data.

[0203] In all of the above-described possible implementations, the segmentation module 901, allocation module 902, optimization module 903, and sending module 904 can be implemented via software or hardware. For example, the implementation of the segmentation module 901 will be described below using the segmentation module 901 as an example. Similarly, the implementations of the allocation module 902, optimization module 903, and sending module 904 can refer to the implementation of the segmentation module 901.

[0204] As an example of a software functional unit, the splitting module 901 may include a program running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the splitting module 901 may include a program running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the program may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the program may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0205] Similarly, multiple hosts / virtual machines / containers used to run the program can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0206] As an example of a hardware functional unit, the segmentation module 901 may include at least one computing device, such as a server. Alternatively, the segmentation module 901 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0207] The multiple computing devices included in the slicing module 901 can be distributed in the same region or in different regions. The multiple computing devices included in the slicing module 901 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the slicing module 901 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0208] It should be noted that the steps that the splitting module 901, the allocation module 902, the optimization module 903 and the sending module 904 are responsible for implementing can be specified as needed. The splitting module 901, the allocation module 902, the optimization module 903 and the sending module 904 respectively implement different steps in the AI ​​model super network optimization method to realize all the functions of the device 900.

[0209] See Figure 10, which is a schematic diagram of the structure of a management node provided in an embodiment of the present application. The management node can be applied to the server 200 shown in Figure 3, or used to deploy the AI ​​model super network optimization device 900 shown in Figure 9 to implement the AI ​​model super network optimization method described in the above method embodiment.

[0210] In Figure 10, the management node provided by the present application is represented by a computing device 1000. As shown in Figure 10, the computing device 1000 includes: a bus 1001, a processor 1002, a memory 1003, a network interface 1004, and an output interface 1005. The processor 1002, the memory 1003, the network interface 1004, and the output interface 1005 communicate with each other via the bus 1001. The computing device 1000 can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device 1000 can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.

[0211] Bus 1001 may be a standard peripheral component interconnect bus or an extended industry standard architecture bus. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG10 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 1001 may include a path for transmitting information between various components of computing device 1000 (e.g., processor 1002, memory 1003, network interface 1004, and output interface 1005).

[0212] Processor 1002 is composed of one or more general-purpose processors, such as a CPU, an NPU, or a combination of a CPU and a hardware chip. The hardware chip is an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a system on chip (SoC), or any combination thereof. Processor 1002 executes various types of digital storage instructions, such as software or firmware programs stored in memory 1003, which enables computing device 1000 to provide a wide variety of services.

[0213] The memory 1003 may include a volatile memory, such as a random access memory, or a non-volatile memory, such as a read-only memory, a flash memory, a mechanical hard disk, or a solid-state drive.

[0214] Memory 1003 stores an executable program, and processor 1002 executes the executable program to implement the functions of the aforementioned segmentation module 901, allocation module 902, optimization module 903, and sending module 904, thereby implementing the AI ​​model super-network optimization method described in the above method embodiment. That is, memory 1003 stores instructions for executing the AI ​​model super-network optimization method described in the above method embodiment.

[0215] The network interface 1004 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to obtain files or network data.

[0216] The output interface 1005 uses, for example but not limited to, a wired interface to enable communication between the computing device 1000 and other devices or a communication network.

[0217] Referring to Figure 11, Figure 11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. The computing device cluster provided in the present application includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0218] As shown in Figure 11, the computing device cluster includes at least one computing device 1000. The memory 1003 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the AI ​​model super network optimization method.

[0219] In some possible implementations, the memory 1003 of one or more computing devices 1000 in the computing device cluster may also store some instructions for executing the AI ​​model super-network optimization method. In other words, the combination of one or more computing devices 1000 can jointly execute instructions for executing the AI ​​model super-network optimization method.

[0220] It should be noted that the memory 1003 in different computing devices 1000 in the computing device cluster can store different instructions, each used to execute part of the functions of the AI ​​model super network optimization device 900. In other words, the instructions stored in the memory 1003 in different computing devices 1000 can implement the functions of one or more of the aforementioned splitting module 901, allocation module 902, optimization module 903, and sending module 904.

[0221] Referring to Figure 12, Figure 12 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. In some possible implementations, one or more computing devices in the computing device cluster can be connected via a network. The network can be a wide area network or a local area network, etc. As shown in Figure 12, two computing devices 1000A and 1000B are connected via a network. Specifically, the network is connected to the network through the network interface and / or output interface in each computing device. In this type of possible implementation, the memory 1003 in the computing device 1000A stores instructions for executing the functions of the segmentation module 901 and the allocation module 902. At the same time, the memory 1003 in the computing device 1000B stores instructions for executing the functions of the optimization module 903 and the sending module 904.

[0222] The connection method between the computing device clusters shown in Figure 12 can be considered to be that the AI ​​model super network optimization method provided in this application requires a large number of candidate models to be split and distributed, so it is considered to entrust the functions implemented by the splitting module 901 and the distribution module 902 to the computing device 1000A for execution.

[0223] It should be understood that the functions of the computing device 1000A shown in Figure 12 may also be completed by multiple computing devices. Similarly, the functions of the computing device 1000B may also be completed by multiple computing devices.

[0224] An embodiment of the present application also provides a computing system, which includes the aforementioned management node (i.e., the computing device 1000 shown in Figure 10) and multiple computing nodes, wherein the multiple computing nodes are used to undertake the training operation of the candidate model in the above-mentioned AI model super network optimization method. Referring to the computing system shown in Figure 13, it includes GPUs 1101 to 1104 connected by a bus in a management node and a computing device 1100. The computing device 1100 can be a server, such as a central server, an edge server, or a local server in a local data center.

[0225] It should be understood that Figure 13 is only an example of the computing system provided by this application. For example, in a specific implementation, the multiple computing nodes are multiple NPUs / TPUs in a single computing device, or multiple CPUs / GPUs / NPUs / TPUs in multiple computing devices, or any combination of CPUs, GPUs, NPUs, and TPUs. Optionally, the computing system is a single computing device, one CPU / GPU / NPU / TPU in the single computing device is a management node, and the other multiple CPUs / GPUs / NPUs / TPUs in the single computing device are or are combined into multiple computing nodes. This application does not specifically limit the type and number of the computing system and multiple computing nodes.

[0226] The present application also provides a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the AI ​​model super-network optimization method.

[0227] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive). The computer-readable storage medium includes instructions for instructing a computing device to execute the AI ​​model super-network optimization method.

[0228] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for optimizing an artificial intelligence (AI) model super network, characterized in that: The method comprises: Splitting the first candidate model to obtain a plurality of first candidate model substructures, wherein the plurality of first candidate model substructures are distributed in a plurality of computing nodes; Splitting the second candidate model to obtain a plurality of second candidate model substructures, wherein the plurality of second candidate model substructures are distributed in the plurality of computing nodes, and the first candidate model and the second candidate model are candidate models of different structures obtained by performing structural changes based on the same basic model; The multiple first candidate model substructures and the multiple second candidate model substructures are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

2. The method according to claim 1, characterized in that The first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed in the multiple computing nodes; Iteratively optimizing the multiple first candidate model substructures and the multiple second candidate model substructures, during the optimization process, using a weight sharing method so that the multiple first candidate model substructures and the multiple second candidate model substructures form a super network, including: The substructures of all candidate models in the candidate model set are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the substructures of all candidate models in the candidate model set form a super network.

3. The method according to claim 2, characterized in that The iterative optimization of the substructures of all candidate models in the candidate model set includes: A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

4. The method according to claim 3, characterized in that The Bayesian optimization algorithm is used to optimize the substructures of all candidate models in the candidate model set for the i-th time, including: Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork an i-th candidate model whose performance is better than the performance of the first candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1; Inputting the training data set into the multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

5. The method according to claim 4, characterized in that The Bayesian optimization algorithm is used to predict, based on the performance of the first candidate model to the performance of the i-1th candidate model, the i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model from the hypernetwork, including: Dividing the first candidate model to the i-1th candidate model according to performance to obtain a first set and a second set, the first set including candidate models whose performance is better than the first performance among the first candidate model to the i-1th candidate model, and the second set including candidate models whose performance is lower than or equal to the first performance among the first candidate model to the i-1th candidate model; Counting a plurality of first numbers and a plurality of second numbers, wherein the plurality of first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the plurality of second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; Based on the multiple first numbers and the multiple second numbers, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

6. The method according to claim 5, characterized in that The step of predicting, based on the plurality of first numbers and the plurality of second numbers, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, comprises: Based on the multiple first numbers, a plurality of first frequencies are obtained, wherein the multiple first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set; Obtaining a plurality of second frequencies based on the plurality of second numbers, wherein the plurality of second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set; Based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model is predicted from the hypernetwork.

7. The method according to claim 6, characterized in that The method of predicting, based on the multiple first frequencies and the multiple second frequencies, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, comprises: Calculating a quotient corresponding to each value among multiple values ​​of the j-th structural parameter in the structural parameter set, wherein the quotient corresponding to each value is a quotient of a first frequency corresponding to each value and a second frequency corresponding to each value; The value with the largest corresponding quotient among multiple values ​​of the j-th structural parameter is determined as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the first candidate model to the performance of the i-1-th candidate model.

8. The method according to any one of claims 1 to 7, characterized in that: The method of splitting the first candidate model to obtain the multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain the multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain the multiple first candidate model substructures.

9. The method according to any one of claims 1 to 8, characterized in that: The performance includes at least one of running time, accuracy, and parameter quantity.

10. The method according to any one of claims 1 to 9, characterized in that: The basic model is a translator Transformer network, or the basic model is a convolutional neural network CNN.

11. The method according to any one of claims 1 to 10, characterized in that: The multiple computing nodes include any one or more of a graphics processing unit GPU, a tensor processing unit TPU and a neural network processor NPU.

12. The method according to any one of claims 4 to 11, characterized in that After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the method further includes: The Kth candidate model is sent to a terminal device of the user, where the Kth candidate model is used to process the text data of the user and output the processed text data.

13. An AI model super network optimization device, characterized in that: The device comprises: A segmentation module, used for segmenting the first candidate model to obtain a plurality of first candidate model substructures, and segmenting the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by structurally changing the same basic model; a distribution module, configured to distribute the plurality of first candidate model substructures among a plurality of computing nodes, and to distribute the plurality of second candidate model substructures among the plurality of computing nodes; The optimization module is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

14. The device according to claim 13, characterized in that The first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed in the multiple computing nodes; The optimization module is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is used to make the substructures of all candidate models in the candidate model set form a super network.

15. The device according to claim 14, characterized in that The optimization module is specifically used for: A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

16. The device according to claim 15, characterized in that The optimization module is specifically used for: Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1; Inputting the training data set into the multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

17. The device according to claim 16, characterized in that The device also includes a sending module; After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

18. A management node, characterized in that: The management node includes a processor and a memory, the memory stores codes, and the processor executes the codes to implement the method according to any one of claims 1 to 12.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 12.

20. A computing system, characterized in that: The computing system includes a management node and multiple computing nodes, the management node is used to execute the method as described in any one of claims 1 to 12, and the multiple computing nodes are used to undertake the training operation of the candidate model in the method as described in any one of claims 1 to 12.

Images