AI model super network optimization method and device and computer readable storage medium

Abstract

The invention discloses an AI model super network optimization method and device and a computer readable storage medium, and the method comprises the steps: segmenting a first candidate model to obtain a plurality of first candidate model substructures, and segmenting a second candidate model to obtain a plurality of second candidate model substructures, and then the plurality of first candidate model substructures and the plurality of second candidate model substructures are distributed in a plurality of computational nodes, and iterative optimization is performed on the plurality of first candidate model substructures and the plurality of second candidate model substructures, and enabling the plurality of first candidate model substructures and the plurality of second candidate model substructures to form a super network by adopting a weight sharing mode, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by performing structure change based on the same basic model. According to the method, the problems that in a current super network optimization algorithm, a computing node cannot bear a large-model super network, and the large-model super network cannot be optimized can be solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to an AI model super network optimization method, device and computer-readable storage medium. Background Art

[0002] With the development of AI technology, neural networks (also called models or AI models) have been widely used in fields such as computer vision (CV), natural language processing (NLP), graph learning, and recommendation systems. Since different application scenarios have different requirements for models, it is necessary to select a suitable model according to the requirements of the application scenario. In the past, models were all designed manually. In recent years, a hypernetwork optimization algorithm has been developed that can automatically iteratively optimize a hypernetwork formed by a large number of candidate models (also called candidate networks), and determine the most suitable candidate model from the optimized hypernetwork.

[0003] At present, the commonly used super network optimization algorithm in the industry is the neural architecture search (NAS) algorithm. The implementation process of the NAS algorithm is: multiple candidate models with different structures are obtained by structural changes based on the basic model. Assuming that there are M candidate models, all of them are deployed to N computing nodes (such as graphics processing unit (GPU), tensor processing unit (TPU), neural network processing unit (NPU)), and then the N*M candidate models on the N computing nodes are iteratively optimized. Specifically, in the optimization process, the weight sharing method is first used to make the M candidate models on each computing node form a super network, and the N super networks corresponding to the N computing nodes are obtained, and then the N super networks on the N computing nodes are iteratively optimized.

[0004] In the above NAS algorithm, since each computing node needs to carry the super network formed by multiple candidate models, and the carrying capacity of the computing nodes is limited, if the parameter scale of the super network is very large (such as billions or hundreds of billions), the computing nodes cannot bear it and the optimization of the large model super network cannot be performed. Summary of the invention

[0005] The present application provides an AI model supernetwork optimization method, device and computer-readable storage medium, which can solve the problem that the computing nodes in the current supernetwork optimization algorithm cannot carry the AI ​​model supernetwork and cannot optimize the larger-scale AI model supernetwork (also called large model supernetwork).

[0006] In a first aspect, a method for super network optimization of an AI model is provided, which may include the following steps: dividing a first candidate model to obtain multiple first candidate model substructures, and dividing a second candidate model to obtain multiple second candidate model substructures, wherein the multiple first candidate model substructures are distributed in multiple computing nodes, and the multiple second candidate model substructures are also distributed in multiple computing nodes, and the first candidate model and the second candidate model are candidate models of different structures obtained by structural changes based on the same basic model, and then the multiple first candidate model substructures and the multiple second candidate model substructures are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

[0007] In the above scheme, after the basic model is structurally changed to obtain multiple candidate models with different structures (the above scheme takes the multiple candidate models as the first candidate model and the second candidate model as an example), each candidate model is segmented to obtain multiple substructures of each candidate model, and the multiple substructures of each candidate model are distributed in multiple computing nodes. Then, the substructures of the multiple candidate models are iteratively optimized, and in the optimization process, a weight sharing method is adopted so that the substructures of the multiple candidate models on the multiple computing nodes form a super network. It can be seen that each computing node only needs to carry one substructure of each candidate model in the multiple candidate models that constitute the super network, and does not need to carry the complete super network. The complete super network is carried by multiple computing nodes together, and then the super network is optimized by multiple computing nodes together, thereby solving the problem that the computing nodes in the NAS algorithm cannot carry the large model super network and cannot optimize the large model super network.

[0008] In a possible implementation, the first candidate model and the second candidate model both belong to a candidate model set, which is a set of candidate models with different structures obtained by performing structural changes on the same basic model, and the multiple substructures obtained by segmenting each candidate model except the first candidate model and the second candidate model in the candidate model set are also distributed in multiple computing nodes;

[0009] The iterative optimization of the multiple first candidate model substructures and the multiple second candidate model substructures is performed. During the optimization process, a weight sharing method is adopted so that the multiple first candidate model substructures and the multiple second candidate model substructures form a super network, including:

[0010] The substructures of all candidate models in the candidate model set are iteratively optimized. During the optimization process, a weight sharing method is used to make the substructures of all candidate models in the candidate model set form a super network.

[0011] In a possible implementation, the iterative optimization of the substructures of all candidate models in the candidate model set includes:

[0012] The Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

[0013] In a possible implementation, the above-mentioned Bayesian optimization algorithm is used to perform the i-th optimization on the substructure of all candidate models in the candidate model set, including: using the Bayesian optimization algorithm, based on the performance of the 1st candidate model to the i-1th candidate model, predicting from the hypernetwork the i-th candidate model whose performance is better than the performance of the 1st candidate model to the i-1th candidate model, 1≤i≤K, K is the number of iterative optimizations of the substructure of all candidate models in the candidate model set, and K is an integer greater than 1; inputting the training data set into multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; and updating the weight of the hypernetwork based on the gradient corresponding to the i-th candidate model.

[0014] It can be understood that the above-mentioned first candidate model is the candidate model selected for training in the first optimization process of the substructure of all candidate models in the candidate model set, the above-mentioned second candidate model is the candidate model selected for training in the second optimization process of the substructure of all candidate models in the candidate model set,…, the above-mentioned i-1th candidate model is the candidate model selected for training in the i-1th optimization process of the substructure of all candidate models in the candidate model set.

[0015] In the above scheme, in each optimization process, the Bayesian optimization algorithm is used to predict a candidate model from the super network whose performance is better than the candidate model trained in the historical iteration, and the candidate model is trained to obtain the corresponding gradient, and the weight of the super network is updated based on the gradient. Compared with the NAS algorithm, in each optimization process of the super network, N candidate models need to be trained to obtain the gradients corresponding to the N candidate models, and then the weight of the super network is updated based on the gradients corresponding to the N candidate models, which can improve the optimization efficiency of the super network.

[0016] In a possible implementation, the above-mentioned Bayesian optimization algorithm is used to predict, based on the performance of the first candidate model to the performance of the i-1th candidate model, the i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model from the hypernetwork, including:

[0017] The 1st to i-1th candidate models are divided according to performance to obtain a first set and a second set, the first set includes candidate models whose performance is better than the first performance among the 1st to i-1th candidate models, and the second set includes candidate models whose performance is lower than or equal to the first performance among the 1st to i-1th candidate models; multiple first numbers and multiple second numbers are counted, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, the i-th candidate model with performance better than the performance of the 1st to i-1th candidate models is predicted from the hypernetwork.

[0018] By implementing the above implementation, it is possible to predict from the hypernetwork the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0019] In a possible implementation, the method of predicting, based on the multiple first numbers and the multiple second numbers, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, includes:

[0020] Based on multiple first numbers, multiple first frequencies are obtained, wherein the multiple first frequencies are the frequencies at which the values ​​of the structure parameter set appear in the first set; based on multiple second numbers, multiple second frequencies are obtained, wherein the multiple second frequencies are the frequencies at which the values ​​of the structure parameter set appear in the second set; based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

[0021] In a possible implementation, the method of predicting, based on the multiple first frequencies and the multiple second frequencies, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, includes:

[0022] Calculate the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, where the quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0023] In one possible implementation, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0024] In a possible implementation, the above performance includes at least one of running time, accuracy, and parameter quantity.

[0025] In a possible implementation, the basic model is a Transformer network, or the basic model is a convolutional neural network (CNN).

[0026] In a possible implementation, the plurality of computing nodes include any one or more of a GPU, a TPU, and an NPU.

[0027] In a possible implementation, after completing the iterative optimization of the substructures of all candidate models in the candidate model set, the method further includes:

[0028] The Kth candidate model is sent to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

[0029] In a second aspect, a model super network optimization device is provided, the device comprising:

[0030] A segmentation module, used for segmenting the first candidate model to obtain a plurality of first candidate model substructures, and segmenting the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by structurally changing the same basic model;

[0031] a distribution module, configured to distribute the plurality of first candidate model substructures among a plurality of computing nodes, and to distribute the plurality of second candidate model substructures among the plurality of computing nodes;

[0032] The optimization module is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

[0033] In a possible implementation, the first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed in the multiple computing nodes;

[0034] The optimization module is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is used to make the substructures of all candidate models in the candidate model set form a super network.

[0035] In a possible implementation, the optimization module is specifically used to:

[0036] A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

[0037] In a possible implementation, the optimization module is specifically used to:

[0038] Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1;

[0039] Inputting the training data set into the multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model;

[0040] Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

[0041] In a possible implementation, the optimization module is specifically used to:

[0042] The 1st to i-1th candidate models are divided according to performance to obtain a first set and a second set, the first set includes candidate models whose performance is better than the first performance among the 1st to i-1th candidate models, and the second set includes candidate models whose performance is lower than or equal to the first performance among the 1st to i-1th candidate models; multiple first numbers and multiple second numbers are counted, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, the i-th candidate model with performance better than the performance of the 1st to i-1th candidate models is predicted from the hypernetwork.

[0043] In a possible implementation, the optimization module is specifically used to:

[0044] Based on multiple first numbers, multiple first frequencies are obtained, wherein the multiple first frequencies are the frequencies at which the values ​​of the structure parameter set appear in the first set; based on multiple second numbers, multiple second frequencies are obtained, wherein the multiple second frequencies are the frequencies at which the values ​​of the structure parameter set appear in the second set; based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

[0045] In a possible implementation, the optimization module is specifically used to:

[0046] Calculate the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, where the quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0047] In one possible implementation, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0048] In a possible implementation, the above performance includes at least one of running time, accuracy, and parameter quantity.

[0049] In a possible implementation, the above basic model is a Transformer network, or the basic model is a CNN.

[0050] In a possible implementation, the plurality of computing nodes include any one or more of a GPU, a TPU, and an NPU.

[0051] In a possible implementation manner, the device further includes a sending module;

[0052] After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

[0053] According to a third aspect, a management node is provided, comprising a processor and a memory; the processor is used to execute instructions stored in the memory, so that the management node implements the method provided by the first aspect or any possible implementation of the first aspect.

[0054] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions for implementing the method provided in the first aspect or any possible implementation of the first aspect.

[0055] According to a fifth aspect, a computer program product is provided, comprising a computer program. When the computer program is read and executed by a computing device, the computing device executes the method provided in the first aspect or any possible implementation of the first aspect.

[0056] In a sixth aspect, a computing system is provided, comprising a management node and multiple computing nodes, wherein the management node is used to implement the method provided in the first aspect or any possible implementation of the first aspect, and the multiple computing nodes are used to undertake the training operations on the candidate models in the method provided in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of the standard structure of the Transformerblock provided in the embodiment of the present application;

[0058] Figure 2 It is a schematic diagram of different model weights shared and merged into one model provided in an embodiment of the present application;

[0059] Figure 3 It is a structural diagram of an artificial intelligence system provided in an embodiment of the present application;

[0060] Figure 4is a schematic diagram of the NAS algorithm provided in the embodiment of the present application performing the i-th optimization on the super network;

[0061] Figure 5 It is a flowchart of an AI model super network optimization method provided in an embodiment of the present application;

[0062] Fig. 6A It is a schematic diagram of synthesizing a hypernetwork of multiple candidate models provided in an embodiment of the present application;

[0063] Figure 6B is a schematic diagram of performing the i-th optimization on a super network provided in an embodiment of the present application;

[0064] Figure 7 is a schematic diagram of a process for performing the i-th optimization on a super network provided in an embodiment of the present application;

[0065] Figure 8 It is a schematic diagram of a process of using a Bayesian optimization algorithm to predict an i-th candidate model provided in an embodiment of the present application;

[0066] Fig. 9 It is a structural diagram of an AI model super network optimization device provided in an embodiment of the present application;

[0067] Fig.10 It is a structural diagram of a management node provided in an embodiment of the present application;

[0068] Fig.11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0069] Fig.12 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application;

[0070] Fig.13 It is a structural diagram of a computing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention. The terms used in the implementation mode of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0072] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0073] In order to facilitate a clear understanding of the technical solution provided by this application, some nouns and terms involved in the technical solution provided by this application are first explained.

[0074] (1) Neural Network

[0075] It can be composed of neural units (also called neurons), which can refer to variables x. s and intercept b as inputs, the output of the operation unit can be:

[0076]

[0077] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s For x s The weight of the neural unit, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer. The activation function can be a sigmoid function and other functions, which are not limited here. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0078] (2) Convolutional neural network (CNN)

[0079] CNN is a deep neural network with a convolutional structure. The convolutional neural network contains a feature extractor consisting of a convolutional layer and a subsampling layer, which can be regarded as a filter. The convolutional layer refers to the neuron layer in the convolutional neural network that performs convolution processing on the input signal. In the convolutional layer of the convolutional neural network, a neuron can only be connected to some neurons in the adjacent layers. A convolutional layer usually contains several feature planes, and each feature plane can be composed of some neural units arranged in a rectangular shape. The neural units in the same feature plane share weights, and the shared weights here are the convolution kernels. Shared weights can be understood as the way of extracting image information is independent of position.

[0080] The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0081] (3) Transformer Network

[0082] The Transformer network, also known as the Transformer model, is a multi-layer neural network based on the self-attention module. Currently, it is mainly used to process natural language tasks. The Transformer network consists of one or more layers of Transformer modules (blocks). The structure of a standard Transformer block is as follows: Figure 1 As shown, the left side is the encoder and the right side is the decoder. Each encoder may include any number of encoding submodules, each encoding submodule includes a multi-head self-attention module and a feedforward neural network; similarly, each decoder may include any number of decoding submodules, each decoding submodule includes two multi-head self-attention modules and a feedforward neural network. The number of encoding submodules may be different from the number of decoding submodules. Among them, FFN can also be called multilayer perceptron (MLP).

[0083] (4) Loss function

[0084] In the process of training a neural network, because we want the output of the neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight matrix of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network). For example, if the network's predicted value is high, adjust the weight matrix to make it predict lower, and keep adjusting until the neural network can predict the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function (objective function), which is an important equation used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.

[0085] (5) Back propagation algorithm

[0086] In the process of training a neural network, the error back propagation (BP) algorithm can be used to correct the size of the parameters in the initial model, so that the reconstruction error loss of the model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the parameters in the initial model are updated by back propagating the error loss information, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal model parameters, such as the weight matrix.

[0087] (6) Candidate model refers to the model obtained by changing the structure of the basic model. The basic model can be any model such as CNN, deep neural network, Transformer model, etc. Figure 1 Taking the Transformer block shown in the figure as an example, for example, the number of heads of the multi-head self-attention module in the encoding submodule or the decoding submodule in the Transformer block can be changed to obtain candidate models with different structures.

[0088] (7) Model super network, also known as super network or super model, is a model that is obtained by merging multiple candidate models with different structures after changing the structure of the basic model. The model is called super network, and the merging method can be weight sharing. Figure 2 As an example, when model A and model B are combined into a hypernetwork C, the same structure in model A and model B (i.e. Figure 2The structure in the dotted box in the figure is merged, and the merger here is weight sharing. By weight sharing, the number of parameters that need to be optimized can be reduced when optimizing the hypernetwork.

[0089] The application scenarios of the embodiments of the present application are introduced below.

[0090] like Figure 3 As shown, the embodiment of the present application can be applied to Figure 3 The artificial intelligence system shown. The artificial intelligence system includes a local device 100 and a server 200, and the local device 100 and the server 200 can be connected through a communication network with any communication mechanism / communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0091] The user can operate the local device 100 and submit the super network optimization task to the server 200 through the communication network.

[0092] Among them, the local device 100, which may also be referred to as the terminal device 100, may be any computing device, such as a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a virtual reality (VR) device, a smart car or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0093] The server 200 can be a computing device or a computing device cluster composed of multiple computing devices. The computing device may include but is not limited to physical servers, virtual machines, containers, chips, etc., among which physical servers may include but are not limited to ARM servers, X86 servers, etc.; a virtual machine refers to a complete computer system with complete hardware system functions implemented by network function virtualization (NFV) technology and simulated by software, running in a completely isolated environment; a container refers to a group of processes that are subject to resource constraints and isolated from each other, and is not specifically limited here.

[0094] The server 200 is responsible for allocating the super network optimization tasks to the resource pool composed of computing devices or computing device clusters, such as CPU, GPU, TPU, NPU, etc., and providing them with corresponding computing power, storage space or various software services. When the server 200 is a computing device cluster, the server 200 can be the central cloud data center of the cloud service provider, or it can be the edge data center provided by the cloud service provider to the user. After the server 200 completes the optimization of the super network, it will return the most suitable candidate model determined from the super network to the user.

[0095] The user can deploy the most suitable candidate model on the local device 100, and the local device 100 with the most suitable candidate model can be applied to various scenarios. As an example, the local device 100 can be used for identity authentication through face recognition technology; as another example, the local device 100 can be used for natural language understanding; as another example, the local device 100 can also be used for augmented reality in games.

[0096] However, different application scenarios have different requirements for neural networks. For example, when the local device 100 is used for identity authentication through face recognition technology, a neural network with higher accuracy is required, that is, a candidate model with higher accuracy is required; when the local device 100 is used for augmented reality in games, a neural network with faster response is required, that is, a candidate model with faster response is required.

[0097] In addition, different types of local devices 100 have different requirements for neural networks. For example, different local devices 100 may have different computational delays for the same neural network due to differences in underlying hardware. The different types may refer to different types of local devices 100, such as local devices 100 being smartphones and tablets; or different types may refer to different models of local devices 100 of the same type, such as local devices 100 being different models of mobile phones.

[0098] Therefore, a suitable candidate model may be selected for a specific local device 100 based on the type of the local device 100 , the application scenario, and other possible restrictions.

[0099] exist Figure 3In the artificial intelligence system shown, the current server 200 usually uses a super network optimization algorithm such as a NAS algorithm to determine a suitable candidate model from a large number of candidate models. The specific process of the server 200 using the NAS algorithm to optimize the super network is as follows: the server 200 receives the basic model (CNN, deep neural network, Transformer network, etc.), the data set (including the training data set and the test data set) uploaded by the user operating the local device 100, and the variation range of the basic model (including the structural parameters to be optimized in the basic model and the value variation range of the structural parameters to be optimized). After receiving the above data, the server 200 performs structural changes on the basic model according to the variation range of the above basic model to obtain multiple candidate models with different structures, assuming that there are M candidate models, and then all the M candidate models are deployed to N computing nodes (such as N GPUs). Finally, the N*M candidate models on the N computing nodes are iteratively optimized. Specifically, in the optimization process, the weight sharing method is first used to form a super network for the M candidate models on each computing node, and N super networks corresponding to the N computing nodes are obtained, and then the N super networks on the N computing nodes are iteratively optimized. Wherein, M and N are both integers greater than 1.

[0100] Next, combine Figure 4 This paper introduces the specific process of iterative optimization of N super networks on N computing nodes using the NAS algorithm.

[0101] Take the i-th optimization as an example, Figure 4 As shown, in Figure 4 In the example, the computing node is GPU, and N candidate models are selected from the super network by using reinforcement learning / evolutionary algorithm, and these N candidate models are activated in N super networks on N computing nodes (N computing nodes and N candidate models are in a one-to-one correspondence). Then, the test data set is input into each computing node to test the performance of the candidate model activated on each computing node (such as running time, accuracy or number of parameters, etc.). Figure 4 In the above example, the performance is the accuracy, represented by Acc), and the training data set is input into each computing node to train the candidate model activated on each computing node to obtain the gradient. Finally, the performance of the obtained N candidate models is fed back to the reinforcement learning / evolutionary algorithm, and the reinforcement learning / evolutionary algorithm selects N candidate models for the i+1th optimization from the super network based on the performance of the N candidate models, and after fusing the N gradients, the weights of the N super networks on the N computing nodes are updated according to the fused gradients. Among them, 1≤i≤K, K is the number of iterative optimizations of the N super networks, and K and N are both integers greater than 1.

[0102] After completing K iterative optimizations of N*M candidate models on N computing nodes, based on the performance of the N candidate models tested in the Kth optimization process, the candidate model with the best performance among the N candidate models is provided to the user as the most suitable candidate model.

[0103] In view of the problem in the above-mentioned supernetwork optimization method that each computing node needs to carry a complete supernetwork and the carrying capacity of the computing node is limited, the computing node cannot optimize the large model supernetwork. This application provides an AI model supernetwork optimization method.

[0104] In the AI ​​model super network optimization method provided in the present application, after the structural changes are made based on the same basic model to obtain multiple candidate models with different structures, each of the multiple candidate models is divided into multiple substructures, and the multiple substructures corresponding to each candidate model are distributed in multiple computing nodes. The multiple substructures of each candidate model are in a one-to-one correspondence with the multiple computing nodes. Then, the substructures of the multiple candidate models are iteratively optimized. Specifically, in the optimization process, a weight sharing method is first adopted to make the substructures of all candidate models on multiple computing nodes form a super network, and then the super network is iteratively optimized. It can be seen that in the AI ​​model super network optimization method provided in the present application, each computing node only needs to carry one substructure of each candidate model in the multiple candidate models, and does not need to carry the complete super network. The complete super network is carried by multiple computing nodes together, and then the super network is optimized by multiple computing nodes together, thereby solving the problems existing in the above-mentioned NAS algorithm.

[0105] In order to more clearly understand the AI ​​model super network optimization method provided by this application, Figure 5 The process diagram shown and Fig. 6A Describe in detail.

[0106] like Figure 5 As shown, the method comprises the following steps:

[0107] S501: Based on the same basic model, the structure is changed to obtain M candidate models with different structures, such as Fig. 6A Shown in the left part.

[0108] S502: Segment each candidate model in the M candidate models to obtain multiple substructures of each candidate model, and distribute the multiple substructures of each candidate model to multiple computing nodes, such as Fig. 6A The middle part is shown.

[0109] Specifically, the multiple substructures of each candidate model correspond to the multiple computing nodes in a one-to-one relationship. That is, each computing node undertakes a substructure of each candidate model, and a total of M substructures corresponding to M candidate models need to be undertaken.

[0110] S503: Iteratively optimize all substructures of the M candidate models on multiple computing nodes. During the optimization process, a weight sharing method is used to make all substructures of the M candidate models form a super network, such as Fig. 6A Shown on the right.

[0111] Below Figure 5 Each step shown is described in detail.

[0112] In S501, the basic model can be any neural network such as CNN, deep neural network, Transformer network, etc., and this application does not make any specific limitation.

[0113] Specifically, the basic model can be structurally changed based on the variation range of the basic model submitted by the user (including the structural parameters to be optimized in the basic model and the variation range of the values ​​of the structural parameters to be optimized), to obtain M candidate models with different structures. Taking any two candidate models among the M candidate models as an example, the any two candidate models with different structures usually have partially the same structure and partially different structures.

[0114] The basic model includes m layers of Transformer blocks (such as Figure 1 Assume that the structural parameters to be optimized in Transformer network A include the hidden dimension of each layer of Transformer block in Transformer network A, the MLP ratio, and the head number of the multi-head self-attention module. It can be seen that there are 3*m structural parameters to be optimized in Transformer network A. Assuming that the value range of the hidden dimension of each layer of Transformer block is {9216, 12288, 15360}, the value range of the MLP ratio is {3, 3.5, 4}, and the value range of the head number is {64, 96, 128}, then we can get 3*9 m Candidate models with different structures.

[0115] In S502, the multiple computing nodes may be multiple GPUs, multiple CPUs, multiple NPUs, multiple TPUs, or a part may be GPUs and the rest may be NPUs, a part may be NPUs and the rest may be TPUs, or a part may be GPUs, a part may be NPUs, and the rest may be TPUs, etc. This application does not specifically limit the types and quantities of the multiple computing nodes.

[0116] The method of dividing the M candidate models into substructures may include at least one of tensor parallelism (TP) and pipeline parallelism (PP).

[0117] Among them, tensor parallelism is a parallel computing method mainly used in fields such as machine learning. Tensor parallelism mainly focuses on decomposing tensors into smaller parts during model training and performing parallel computing on multiple computing nodes. Through tensor parallelism, the computing load can be distributed on multiple computing nodes, improving computing efficiency and thus speeding up model training.

[0118] Pipeline parallelism is a parallel computing method mainly used in fields such as machine learning. Pipeline parallelism focuses on decomposing computing tasks (such as model training tasks) into multiple stages (for example, for convolutional neural networks in machine learning, the computing tasks can be divided into forward propagation, backpropagation, and parameter update stages), and executing these stages in parallel on multiple computing nodes to improve computing efficiency. Unlike tensor parallelism, pipeline parallelism focuses on the time distribution of computing tasks rather than the spatial distribution of tensors.

[0119] First, take the candidate model's segmentation method as pipeline parallelism, and the candidate model is a Transformer network A including m layers of Transformer blocks as an example. Then, the candidate model A can be segmented (usually evenly segmented) based on the pipeline parallel number N to obtain N substructures, and then the N substructures are distributed on N computing nodes. Specifically, the N substructures are in a one-to-one correspondence with the N computing nodes. In a possible embodiment, if m is not an integer multiple of N, then when segmenting, the first N-1 substructures can have the same number of layers, and the Nth substructure can have fewer / more layers. For example, assuming m is 125 and N is 20, the first 19 substructures have 6 layers, and the last substructure has 5 layers. In a specific implementation, the above pipeline parallel number N can be provided by the user or pre-set.

[0120] Next, taking the candidate model segmentation method as tensor parallelism, and the candidate model as a Transformer network A including m layers of Transformerblock as an example, the multi-head self-attention module in each layer of Transformer block of the candidate model A can be segmented (usually evenly segmented) according to the head number based on the tensor parallel number Z, and the feedforward neural network in each layer of Transformer block of the candidate model A can be segmented according to the product of hidden dimension and MLP ratio to obtain Z substructures, and then the Z substructures are distributed on Z computing nodes. Specifically, the Z substructures are in a one-to-one correspondence with the Z computing nodes. In a possible embodiment, if the number of heads and / or the product of hidden dimension and MLP ratio is not an integer multiple of Z, then when segmenting, the number of heads and the product of hidden dimension and MLP ratio of the first N-1 substructures can be the same, and the number of heads and the product of hidden dimension and MLP ratio of the Nth substructure can be less / more, or the candidate model can be deleted from the M candidate models and not segmented. In a specific implementation, the pipeline parallel number Z can be provided by the user or preset.

[0121] Taking the candidate model segmentation method as tensor parallel, and the candidate model as a CNN including m layers of convolutional layers (the convolutional layer may include many convolutional kernels) as an example, for each convolutional layer, the multiple convolutional kernels included in the convolutional layer can be segmented according to the number of convolutional kernels based on the tensor parallel number Z (usually evenly segmented) to obtain Z substructures, and then the Z substructures are distributed on the Z computing nodes. Specifically, the Z substructures correspond to the Z computing nodes in a one-to-one relationship. In a possible embodiment, if the number of convolutional kernels included in the convolutional layer is not an integer multiple of Z, then when segmenting, the number of convolutional kernels of the first N-1 substructures can be the same, and the number of convolutional kernels of the Nth substructure can be less / more.

[0122] Taking the candidate model segmentation method as a combination of pipeline parallelism and tensor parallelism, and the candidate model being a Transformer network A including m-layer Transformer blocks as an example, the candidate model A can be segmented based on the pipeline parallel number N and the tensor parallel number Z. Specifically, the candidate model A can be segmented based on the pipeline parallel number N to obtain N substructures (the segmentation method is the same as above), and then the candidate model A can be segmented based on the tensor parallel number Z to obtain Z substructures (the segmentation method is the same as above), and then the first substructure of the N substructures and the first substructure of the Z substructures are distributed on the first computing node, and the second substructure of the N substructures and the second substructure of the Z substructures are distributed on the second computing node, and so on.

[0123] In S503, specifically, a Bayesian optimization algorithm or the like may be used to iteratively optimize all substructures of the M candidate models on multiple computing nodes.

[0124] Take the Bayesian optimization algorithm for the i-th optimization of all substructures of M candidate models as an example, see Figure 6B and Figure 7 .

[0125] S701: Using the Bayesian optimization algorithm, based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 , the performance predicted from the hypernetwork is better than the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 The i-th candidate model of Figure 6B shown.

[0126] Wherein, 1≤i≤K, K is the number of times all substructures in the M candidate models are iteratively optimized, and K is an integer greater than 1.

[0127] The performance of the first candidate model P1 to the performance of the i-1th candidate model P i-1 The performance may be obtained by testing using the same test data set, and may include at least one of accuracy, running time, and number of parameters.

[0128] It should be noted that the super network here is the super network after the first i-1 optimizations.

[0129] For the specific implementation process of S701, please see below Figure 8 Related description.

[0130] S702: Input the training data set into multiple computing nodes to train the i-th candidate model, and obtain the gradient G corresponding to the i-th candidate model. i ,like Figure 6B shown.

[0131] S703: Based on the gradient G corresponding to the i-th candidate model i Update the weights of the hypernetwork, such as Figure 6B shown.

[0132] It can be understood that since the i-th candidate model is based on the performance P1 of the 1st candidate model to the performance P of the i-1th candidate model i-1 , the performance predicted from the hypernetwork is better than the performance of the first candidate model P1 to the performance of the i-1th candidate model P i-1 candidate model, then the Kth candidate model is the candidate model with the best performance obtained after iterative optimization of all substructures of the M candidate models. Therefore, in a specific embodiment of the present application, after completing the iterative optimization of all substructures of the M candidate models, the Kth candidate model can be sent to the user's local device 100 as a model that can be deployed on the user's local device 100, so that the user can deploy the Kth candidate model on the local device 100.

[0133] It can be understood that after the user deploys the Kth candidate model on the local device 100, the Kth candidate model can be used to process user data. For example, if the Kth candidate model is a Transformer network for natural language understanding, then the Kth candidate model can be used to recognize the text data input by the user and output the recognized text data. For another example, if the Kth candidate model is a CNN for face recognition, then the Kth candidate model can be used to recognize the face in the face image input by the user. For another example, if the Kth candidate model is a deep neural network for object classification, then the Kth candidate model can be used to classify the objects in the object image input by the user.

[0134] See also Figure 8 , Figure 8 A schematic diagram of a process for predicting the i-th candidate model using a Bayesian optimization algorithm is provided in an embodiment of the present application, such as Figure 8 As shown, the following steps are included:

[0135] S801: Based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 , the 1st candidate model to the i-1th candidate model are divided to obtain a first set and a second set, wherein the first set includes candidate models from the 1st candidate model to the i-1th candidate model whose performance is better than the first performance, and the second set includes candidate models from the 1st candidate model to the i-1th candidate model whose performance is lower than or equal to the first performance.

[0136] The first performance can be based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 Calculated, if the first performance is the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 The average value or median value of the first performance can also be customized according to the actual scenario, which is not specifically limited in this application.

[0137] It should be noted that when i is equal to 1, since there is no historical candidate model performance, the first candidate model can be randomly selected from the hypernetwork. When i is equal to 2, the second candidate model can also be randomly selected from the hypernetwork.

[0138] S802: Count multiple first numbers and multiple second numbers, where the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters that need to be optimized in the basic model.

[0139] The structural parameters in the structural parameter set include the hidden dimension, MLP ratio, and head number of each layer of the Transformer block in the Transformer network A in the example above.

[0140] S803: Based on the multiple first numbers and the multiple second numbers, predict from the hypernetwork the performances of the candidate models P1 to P2 of the first candidate model to obtain the performances of the candidate models P3 to P4 of the i-1th candidate model. i-1 The i-th candidate model.

[0141] Specifically, S803 may include Figure 8 Steps S8031-S8033 shown.

[0142] S8031: Obtain a plurality of first frequencies based on a plurality of first order numbers, wherein the plurality of first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set.

[0143] S8032: Obtain a plurality of second frequencies based on the plurality of second numbers, wherein the plurality of second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set.

[0144] S8033: Based on the multiple first frequencies and the multiple second frequencies, predict from the hypernetwork the performance that is better than the performance P1 of the first candidate model to the performance P of the i-1th candidate model i-1 The i-th candidate model.

[0145] Furthermore, S8033 can be implemented by referring to the following method (1) or method (2):

[0146] Method (1), calculating the quotient corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, and then determining the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model. The quotient corresponding to each value is the quotient of the first frequency corresponding to each value and the second frequency corresponding to the value.

[0147] Method (2), calculating the difference corresponding to each of the multiple values ​​of the j-th structural parameter in the structural parameter set, and then determining the value with the largest corresponding difference among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model. The difference corresponding to each value is the difference between the first frequency corresponding to each value and the second frequency corresponding to the value.

[0148] The following takes S8033 implemented by the above method (1) as an example and describes it in combination with a specific example Figure 8 The implementation process of the illustrated embodiment.

[0149] In S801, assuming that i is 5, the performance is accuracy, the performance P1 of candidate model 1 (referring to the first candidate model) is 0.52, the performance P2 of candidate model 2 (referring to the second candidate model) is 0.55, the performance P3 of candidate model 3 (referring to the third candidate model) is 0.65, and the performance P4 of candidate model 4 (referring to the fourth candidate model) is 0.68. The first performance is the average value of P1 to P4, which is 0.6. Based on the performance of these four candidate models, the first set obtained by dividing these four candidate models includes candidate model 3 and candidate model 4, and the second set obtained includes candidate model 1 and candidate model 2.

[0150] In S802, the value of the jth structural parameter in the structural parameter set is V j1 、V j2 and V j3 There are 3 values ​​in total. The first and second times corresponding to these 3 values ​​are counted as an example. Assume that the value of the j-th structural parameter in candidate model 1 is V j1 , the value of the jth structural parameter in candidate model 2 is also V j1 , the value of the jth structural parameter in candidate model 3 is V j3 , the value of the j-th structural parameter in candidate model 4 is V j2 , then we can get V j1 The number of times it appears in the first set (including candidate model 3 and candidate model 4) is 0, and the number of times it appears in the second set (including candidate model 1 and candidate model 2) is 2. j2 The number of times it appears in the first set is 1, and the number of times it appears in the second set is 0.j3 The number of occurrences in the first set is 1, and the number of occurrences in the second set is 0.

[0151] To avoid the following calculation of V j1 、V j2 and V j3 The corresponding quotient is infinite (that is, the dividend is not 0 and the divisor is 0), which can be calculated in the statistical V j1 、V j2 and V j3 Before calculating the number of times each value appears in the first set and the number of times each value appears in the second set, the number of times each value appears in the first set and the number of times each value appears in the second set are initialized to fixed values, such as 1 or 2. Taking initialization as 1 as an example, the V j1 The number of times it appears in the first set is 1, and the number of times it appears in the second set is 3. j2 The number of times it appears in the first set is 2, and the number of times it appears in the second set is 1. j3 The number of times it appears in the first set is 2, and the number of times it appears in the second set is 1. That is, j1 corresponds to the first number 1 and the second number 3, j2 corresponds to the first number 2 and the second number 1, V j3 The corresponding number is 2 for the first time and 1 for the second time.

[0152] In S803, based on V j1 The corresponding first number 1 can be calculated to get V j1 The corresponding first frequency = 1 / (1+2+2) = 1 / 5;

[0153] Based on V j1 The corresponding second number 3 can be calculated to get V j1 The corresponding second frequency = 3 / (3+1+1) = 3 / 5;

[0154] Based on V j2 The corresponding first number 2 can be calculated to get V j2 The corresponding first frequency = 2 / (1+2+2) = 2 / 5;

[0155] Based on V j2 The corresponding second number 1 can be calculated to get V j2 The corresponding second frequency = 1 / (3+1+1) = 1 / 5;

[0156] Based on V j3 The corresponding first number 2 can be calculated to get V j3 The corresponding first frequency = 2 / (1+2+2) = 2 / 5;

[0157] Based on V j3The corresponding second number 1 can be calculated to get V j3 The corresponding second frequency=1 / (3+1+1)=1 / 5.

[0158] Next, based on the above V j1 The corresponding first frequency and second frequency can be calculated to get V j1 The corresponding quotient = (1 / 5) / (3 / 5) = 1 / 3;

[0159] Based on the above V j2 The corresponding first frequency and second frequency can be calculated to get V j2 The corresponding quotient = (2 / 5) / (1 / 5) = 2;

[0160] Based on the above V j3 The corresponding first frequency and second frequency can be calculated to get V j3 The corresponding quotient = (2 / 5) / (1 / 5) = 2.

[0161] Since V j2 and V j3 The corresponding quotient is the same and the largest, so V j2 or V j3 Determine the value of the jth structural parameter in the 5th candidate model.

[0162] It can be understood that if the process shown in the above example is also performed on other structural parameters in the structural parameter set, the value of the structural parameter set in the 5th candidate model can be determined. Once the value of the structural parameter set in the 5th candidate model is determined, the 5th candidate model can be determined.

[0163] It should be noted that the implementation process of S803 is only used as an example. In a specific implementation, the implementation process of S803 may be other, and this application does not specifically limit this. For example, the quotient of the first number and the second number corresponding to multiple values ​​of the j-th structural parameter in the structural parameter set may be compared, and the value with the largest quotient among the multiple values ​​is determined as the value of the j-th structural parameter in the i-th candidate model.

[0164] It should also be noted that the above Figure 8 The embodiment shown is only based on the performance P1 of the first candidate model to the performance P of the i-1th candidate model. i-1 A possible implementation of predicting the i-th candidate model. In another possible implementation, a regression model (such as linear regression, ridge regression, or support vector regression) or a method based on domain knowledge rules can also be used to predict the i-th candidate model based on the performance of the first candidate model to the performance of the i-1-th candidate model.

[0165] After the i-th candidate model is determined, the i-th candidate model may be activated in multiple computing nodes, that is, multiple substructures of the i-th candidate model distributed in multiple computing nodes may be activated, and S702 - S703 may be executed.

[0166] It should be noted that the above Figure 5 The embodiment is described by taking the method of dividing the M candidate models as an example, including at least one of tensor parallelism and pipeline parallelism. In a possible embodiment, when the number of computing nodes is large enough, the method of dividing the M candidate models may also include data parallelism (DP). Data parallelism is a parallel computing method mainly used in fields such as machine learning. Data parallelism focuses on decomposing the training data set into multiple batches for parallel computing, thereby improving computing efficiency.

[0167] The candidate model is split into pipeline parallelism and data parallelism, and the candidate model includes m layers of Transformerblock ( Figure 1 Taking the Transformer network A shown in FIG. 1 as an example, the candidate model A can be segmented (usually evenly segmented) based on the pipeline parallel number N to obtain N substructures, and then the N substructures are distributed on the N computing nodes. Specifically, the N substructures correspond to the N computing nodes in a one-to-one relationship. In a possible embodiment, if m is not an integer multiple of N, the first N-1 substructures can be segmented with the same number of layers, and the Nth substructure can have fewer layers. For example, assuming that m is 125 and N is 20, the first 19 substructures have 6 layers, and the last substructure has 5 layers. Taking the data parallel number T (T is an integer greater than 1) as an example, assuming that N computing nodes are numbered from 1 to N, each numbered computing node also has T-1 replicated computing nodes with the same number, that is, the computing node numbered 1 has T-1 replicated computing nodes numbered 1, the computing node numbered 2 has T-1 computing nodes numbered 2, ..., the computing node numbered N has T-1 computing nodes numbered N, and the T-1 computing nodes with the same number are used to replicate the candidate model substructure on the computing node with the same number. In a specific implementation, the above pipeline parallel number N and data parallel number T can be provided by the user or preset, and T is an integer greater than 1.

[0168] Then, a data parallel method can be used to iteratively optimize all candidate model substructures on T groups of T*N computing nodes labeled 1 to N. Specifically, during the optimization process, a weight sharing method can be used to make all candidate model substructures on each group of computing nodes in the T groups of computing nodes form a super network, thereby forming T identical super networks, and then a data parallel method can be used to iteratively optimize the T identical super networks.

[0169] The specific process of iteratively optimizing T identical hypernetworks using the data parallel method is as follows: dividing the training data set into T batches of training data, and then using the T batches of training data to perform parallel iterative optimization on the T hypernetworks, wherein the T hypernetworks and the T batches of training data are in a one-to-one correspondence.

[0170] Taking T supernetworks as supernetwork 1, supernetwork 2 and supernetwork 3, and performing the i-th optimization on these three supernetworks as an example, the implementation method may be the following method 1 or method 2.

[0171] In method 1, the i-th optimization of hypernetwork 1, the i-th optimization of hypernetwork 2, and the i-th optimization of hypernetwork 3 are not independent, as follows:

[0172] The first step is to input the test data set into the N computing nodes where the hypernetwork 1 is located to test the 1.i candidate model and obtain the performance P of the 1.i candidate model. 1.i , input the test data set into the N computing nodes where the hypernetwork 2 is located to test the 2.ith candidate model, and obtain the performance P of the 2.ith candidate model 2.i , and input the test data set into the N computing nodes where the hypernetwork 3 is located to test the 3.ith candidate model, and obtain the performance P of the 3.ith candidate model 3.i , where the 1.i candidate model is based on the performance P of the 1.1 candidate model using the Bayesian optimization algorithm. 1.1 The performance P of the 1.i-1th candidate model 1.i-1 , the performance of the 2.i candidate model P 2.i And the performance P of the 3.i candidate model 3.i , the performance predicted from the hypernetwork is better than the performance of the 1.1 candidate model P 1.1 The performance P of the 1.i-1th candidate model 1.i-1 , the performance of the 2.i candidate model P 2.i And the performance P of the 3.i candidate model 3.i Candidate models.

[0173] Wherein, "1st." represents the 1st group of computing nodes, i.e., the N computing nodes where supernetwork 1 is located, "2nd." represents the 1st group of computing nodes, i.e., the N computing nodes where supernetwork 1 is located, and "3rd." represents the 3rd group of computing nodes, i.e., the N computing nodes where supernetwork 3 is located. The 2.ith candidate model and the 3.ith candidate model may be randomly selected from the hypernetwork, and optionally, the 2.ith candidate model and the 3.ith candidate model may also be the same candidate model as the 1.ith candidate model, or one of the 2.ith candidate model and the 3.ith candidate model may be the same candidate model as the 1.ith candidate model, and the other may be randomly selected from the hypernetwork.

[0174] Using the Bayesian optimization algorithm, the method of predicting the 1.i-th candidate model is the same as Figure 8 The way to predict the i-th candidate model is similar to that shown in Figure 8 For the sake of brevity, the related description will not be elaborated here.

[0175] The second step is to input the first batch of training data into the N computing nodes where the hypernetwork 1 is located to train the 1.i-th candidate model and obtain the gradient G corresponding to the 1.i-th candidate model. 1.i , input the second batch of training data into the N computing nodes where the hypernetwork 2 is located to train the 2.ith candidate model, and obtain the gradient G corresponding to the 2.ith candidate model 2.i , input the third batch of training data into the N computing nodes where the hypernetwork 3 is located to train the 3.ith candidate model, and obtain the gradient G corresponding to the 3.ith candidate model 3.i .

[0176] Step 3: Based on G 1.i , G 2.i and G 3.i The weights of hypernetwork 1, hypernetwork 2, and hypernetwork 3 are updated.

[0177] Specifically, if any two candidate models among the above-mentioned 1.i candidate models, 2.i candidate models and 3.i candidate models or all candidate models have the same structure that shares the same weight, the gradients corresponding to the weights shared by the same structure can be averaged, and then the weights shared by the above-mentioned same structures in hypernetwork 1, hypernetwork 2 and hypernetwork 3 can be synchronously updated based on the gradient average.

[0178] After iterative optimization of supernetwork 1, supernetwork 2 and supernetwork 3 is completed through the above-mentioned method 1, the 1.Kth candidate model can be sent to the user's local device 100 as a model that can be deployed on the user's local device 100, so that the user can deploy the 1.Kth candidate model on the local device 100, and then use the 1.Kth candidate model to process user data.

[0179] In mode 2, the i-th optimization of hypernetwork 1, the i-th optimization of hypernetwork 2, and the i-th optimization of hypernetwork 3 are independent, that is, the i-th optimizations of the three hypernetworks do not affect each other. The specific process of performing the i-th optimization on hypernetwork 1, hypernetwork 2, and hypernetwork 3, respectively, is the same as the process of performing the i-th optimization on the hypernetwork described in S701-S703 above. Please refer to the relevant description of S701-S703 above. For the sake of brevity of the specification, it will not be elaborated here.

[0180] After iterative optimization of supernetwork 1, supernetwork 2 and supernetwork 3 is completed through method 2, the performance of the three Kth candidate models can be compared, and the Kth candidate model with the best performance can be used as a model that can be deployed on the user's local device 100 and sent to the user's local device 100, so that the user can deploy it on the local device 100 and use the Kth candidate model with the best performance to process user data.

[0181] It should be understood that the above-mentioned method 1 or method 2 is only used as an example and should not be regarded as a specific limitation on the implementation method of iterative optimization of super network 1, super network 2 and super network 3.

[0182] The AI ​​model super network optimization method provided by the present application above, after the basic model is structurally changed to obtain multiple candidate models with different structures, each candidate model is segmented to obtain multiple substructures of each candidate model, and the multiple substructures of each candidate model are distributed in multiple computing nodes (such as GPU, NPU, CPU, TPU, etc.), and the multiple substructures of each candidate model correspond to the multiple computing nodes one by one. Then, the substructures of the multiple candidate models are iteratively optimized. Specifically, in the optimization process, the weight sharing method is first adopted to make the substructures of all candidate models on multiple computing nodes form a super network, and then the super network is iteratively optimized. It can be seen that each computing node only needs to carry a substructure of each candidate model in the multiple candidate models that constitute the super network, and there is no need to carry the complete super network. The complete super network is carried by multiple computing nodes together, and then the super network is optimized by multiple computing nodes together, so it can solve the problem that the computing nodes in the NAS algorithm cannot carry the large model super network and cannot optimize the large model super network.

[0183] Secondly, the AI ​​model super network optimization method provided in the present application uses a Bayesian optimization algorithm to predict from the super network a candidate model with performance better than the candidate model trained in the historical iteration in each optimization process, trains the candidate model to obtain the corresponding gradient, and updates the weight of the super network based on the gradient. Compared with the NAS algorithm, in each optimization process of the super network, it is necessary to train N candidate models to obtain the gradients corresponding to the N candidate models, and then update the weight of the super network based on the gradients corresponding to the N candidate models, which can improve the optimization efficiency of the super network.

[0184] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, correspondingly, relevant devices and equipment for cooperating in implementing the above scheme are also provided below.

[0185] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of an AI model super network optimization device 900 provided in an embodiment of the present application. Fig. 9 As shown, the device 900 includes: a segmentation module 901 , an allocation module 902 and an optimization module 903 .

[0186] A segmentation module 901 is used to segment the first candidate model to obtain a plurality of first candidate model substructures, and to segment the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by performing structural changes on the same basic model;

[0187] An allocating module 902, configured to distribute the plurality of first candidate model substructures to a plurality of computing nodes, and to distribute the plurality of second candidate model substructures to the plurality of computing nodes;

[0188] The optimization module 903 is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

[0189] In some possible implementations, the first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model except the first candidate model and the second candidate model in the candidate model set are also distributed in multiple computing nodes;

[0190] The above-mentioned optimization module 903 is specifically used for iteratively optimizing the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is adopted to make the substructures of all candidate models in the candidate model set form a super network.

[0191] In some possible implementations, the optimization module 903 is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set using a Bayesian optimization algorithm.

[0192] In some possible implementations, the above-mentioned optimization module 903 is specifically used to: adopt a Bayesian optimization algorithm, based on the performance of the 1st candidate model to the performance of the i-1th candidate model, predict from the hypernetwork the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructure of all candidate models in the candidate model set is iteratively optimized, and K is an integer greater than 1; input the training data set into multiple computing nodes to train the i-th candidate model, and obtain the gradient corresponding to the i-th candidate model; based on the gradient corresponding to the i-th candidate model, update the weight of the hypernetwork.

[0193] In some possible implementations, the above-mentioned optimization module 903 is specifically used to: divide the 1st candidate model to the i-1th candidate model according to performance to obtain a first set and a second set, the first set including candidate models whose performance is better than the first performance among the 1st candidate model to the i-1th candidate model, and the second set including candidate models whose performance is lower than or equal to the first performance among the 1st candidate model to the i-1th candidate model; count multiple first numbers and multiple second numbers, wherein the multiple first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the multiple second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structural parameters to be optimized in the basic model; based on the multiple first numbers and the multiple second numbers, predict from the hypernetwork the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0194] In some possible implementations, the above-mentioned optimization module 903 is specifically used to: obtain multiple first frequencies based on multiple first numbers, wherein the multiple first frequencies are the frequencies at which the values ​​of the structure parameter set appear in the first set; obtain multiple second frequencies based on multiple second numbers, wherein the multiple second frequencies are the frequencies at which the values ​​of the structure parameter set appear in the second set; based on the multiple first frequencies and the multiple second frequencies, predict from the hypernetwork an i-th candidate model whose performance is better than the performance of the 1st candidate model to the i-1th candidate model.

[0195] In some possible implementations, the above-mentioned optimization module 903 is specifically used to: calculate the quotient corresponding to each value among multiple values ​​of the j-th structural parameter in the structural parameter set, the quotient corresponding to each value being the quotient of the first frequency corresponding to each value and the second frequency corresponding to each value; determine the value with the largest corresponding quotient among the multiple values ​​of the j-th structural parameter as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the 1st candidate model to the performance of the i-1th candidate model.

[0196] In some possible implementations, the above-mentioned method of splitting the first candidate model to obtain multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain multiple first candidate model substructures.

[0197] In some possible implementations, the above performance includes at least one of running time, accuracy, and parameter quantity.

[0198] In some possible implementations, the above-mentioned basic model is a Transformer network, or the basic model is a CNN, or the basic model is a deep neural network.

[0199] In some possible implementations, the above-mentioned multiple computing nodes include any one or more of GPU, TPU, NPU, and CPU.

[0200] In some possible implementations, such as Fig. 9 As shown, the apparatus 900 further includes a sending module 904;

[0201] After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device. The Kth candidate model is used to process the user's text data and output the processed text data.

[0202] The segmentation module 901, the allocation module 902, the optimization module 903, and the sending module 904 in all possible implementations described above can be implemented by software or by hardware. Exemplarily, the segmentation module 901 is taken as an example to introduce the implementation of the segmentation module 901. Similarly, the implementation of the allocation module 902, the optimization module 903, and the sending module 904 can refer to the implementation of the segmentation module 901.

[0203] As an example of a software functional unit, the split module 901 may include a program running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the split module 901 may include a program running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the program may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the program may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0204] Similarly, multiple hosts / virtual machines / containers used to run the program can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0205] As an example of a hardware functional unit, the segmentation module 901 may include at least one computing device, such as a server, etc. Alternatively, the segmentation module 901 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0206] The multiple computing devices included in the split module 901 can be distributed in the same region or in different regions. The multiple computing devices included in the split module 901 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the split module 901 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0207] It should be noted that the steps that the splitting module 901, the allocation module 902, the optimization module 903 and the sending module 904 are responsible for implementing can be specified as needed. The splitting module 901, the allocation module 902, the optimization module 903 and the sending module 904 respectively implement different steps in the AI ​​model super network optimization method to realize all the functions of the device 900.

[0208] See also Fig.10 , Fig.10 is a schematic diagram of a management node structure provided in an embodiment of the present application. The management node can be applied to Figure 3 The server 200 shown, or for deploying Fig. 9 The AI ​​model super network optimization device 900 shown is used to implement the AI ​​model super network optimization method described in the above method embodiment.

[0209] exist Fig.10 In the example, the computing device 1000 represents the management node provided by the present application. Fig.10 As shown, the computing device 1000 includes: a bus 1001, a processor 1002, a memory 1003, a network interface 1004 and an output interface 1005. The processor 1002, the memory 1003, the network interface 1004 and the output interface 1005 communicate through the bus 1001. The computing device 1000 can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device 1000 can also be a terminal device such as a desktop, a laptop or a smart phone. It should be understood that the present application does not limit the number of processors and memories in the computing device 1000.

[0210] The bus 1001 may be a peripheral component interconnection standard bus or an extended industry standard structure bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10The bus 1001 may include a path for transmitting information between various components of the computing device 1000 (eg, the processor 1002, the memory 1003, the network interface 1004, and the output interface 1005).

[0211] The processor 1002 is composed of one or more general-purpose processors, such as a CPU, an NPU, or a combination of a CPU and a hardware chip. The above-mentioned hardware chip is an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a system on chip (SoC), or any combination thereof. The processor 1002 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 1003, which enables the computing device 1000 to provide a wide variety of services.

[0212] The memory 1003 may include a volatile memory, such as a random access memory, or a non-volatile memory, such as a read-only memory, a flash memory, a mechanical hard disk, or a solid state drive.

[0213] The memory 1003 stores an executable program, and the processor 1002 executes the executable program to respectively implement the functions of the aforementioned segmentation module 901, the allocation module 902, the optimization module 903, and the sending module 904, thereby implementing the AI ​​model super network optimization method described in the above method embodiment. That is, the memory 1003 stores instructions for executing the AI ​​model super network optimization method described in the above method embodiment.

[0214] The network interface 1004 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to obtain files or network data.

[0215] The output interface 1005 uses, for example but not limited to, a wired interface to implement communication between the computing device 1000 and other devices or a communication network.

[0216] See also Fig.11 , Fig.11 1 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. The computing device cluster provided in the present application includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0217] like Fig.11 As shown, the computing device cluster includes at least one computing device 1000. The memory 1003 in one or more computing devices 1000 in the computing device cluster may store the same instructions for executing the AI ​​model super network optimization method.

[0218] In some possible implementations, the memory 1003 of one or more computing devices 1000 in the computing device cluster may also respectively store some instructions for executing the AI ​​model super network optimization method. In other words, the combination of one or more computing devices 1000 can jointly execute instructions for executing the AI ​​model super network optimization method.

[0219] It should be noted that the memory 1003 in different computing devices 1000 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the AI ​​model super network optimization device 900. That is, the instructions stored in the memory 1003 in different computing devices 1000 can implement the functions of one or more modules of the aforementioned segmentation module 901, allocation module 902, optimization module 903 and sending module 904.

[0220] See also Fig.12 , Fig.12 FIG. 1 is a schematic diagram of another computing device cluster provided in an embodiment of the present application. In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. Fig.12 As shown, two computing devices 1000A and 1000B are connected via a network. Specifically, the network is connected via a network interface and / or an output interface in each computing device. In this type of possible implementation, the memory 1003 in the computing device 1000A stores instructions for executing the functions of the segmentation module 901 and the allocation module 902. At the same time, the memory 1003 in the computing device 1000B stores instructions for executing the functions of the optimization module 903 and the sending module 904.

[0221] Fig.12The connection method between the computing device clusters shown can be based on the fact that the AI ​​model super network optimization method provided in this application requires a large number of candidate models to be segmented and allocated, so it is considered that the functions implemented by the segmentation module 901 and the allocation module 902 are handed over to the computing device 1000A for execution.

[0222] It should be understood that Fig.12 The functions of the computing device 1000A shown in FIG. 1000A may also be completed by multiple computing devices. Similarly, the functions of the computing device 1000B may also be completed by multiple computing devices.

[0223] The present application also provides a computing system, which includes the aforementioned management node (i.e. Fig.10 The computing device 1000 shown in the figure) and multiple computing nodes, wherein the multiple computing nodes are used to undertake the training operation of the candidate model in the above-mentioned AI model super network optimization method. Fig.13 The computing system shown includes a management node and GPUs 1101 to 1104 connected via a bus in a computing device 1100. The computing device 1100 may be a server, such as a central server, an edge server, or a local server in a local data center.

[0224] It should be understood that Fig.13 This application only provides an example of a computing system. For example, in a specific implementation, multiple computing nodes are multiple NPUs / TPUs in a single computing device, or multiple CPUs / GPUs / NPUs / TPUs in multiple computing devices, or any combination of CPUs, GPUs, NPUs, and TPUs. Optionally, the computing system is a single computing device, one CPU / GPU / NPU / TPU in the single computing device is a management node, and the other multiple CPUs / GPUs / NPUs / TPUs in the single computing device are or are combined into multiple computing nodes. This application does not specifically limit the type and number of the computing system or multiple computing nodes.

[0225] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the AI ​​model super network optimization method.

[0226] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive), etc. The computer-readable storage medium includes instructions for instructing the computing device to execute the AI ​​model super network optimization method.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An artificial intelligence AI model super network optimization method, characterized in that: The method comprises: Splitting the first candidate model to obtain a plurality of first candidate model substructures, wherein the plurality of first candidate model substructures are distributed in a plurality of computing nodes; Splitting the second candidate model to obtain a plurality of second candidate model substructures, wherein the plurality of second candidate model substructures are distributed in the plurality of computing nodes, and the first candidate model and the second candidate model are candidate models of different structures obtained by performing structural changes based on the same basic model; The multiple first candidate model substructures and the multiple second candidate model substructures are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

2. The method according to claim 1, characterized in that: The first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed in the multiple computing nodes; Iteratively optimizing the multiple first candidate model substructures and the multiple second candidate model substructures, during the optimization process, using a weight sharing method so that the multiple first candidate model substructures and the multiple second candidate model substructures form a super network, including: The substructures of all candidate models in the candidate model set are iteratively optimized. During the optimization process, a weight sharing method is adopted to make the substructures of all candidate models in the candidate model set form a super network.

3. The method according to claim 2, characterized in that The iterative optimization of the substructures of all candidate models in the candidate model set includes: A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

4. The method according to claim 3, characterized in that The Bayesian optimization algorithm is used to optimize the substructures of all candidate models in the candidate model set for the i-th time, including: Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork an i-th candidate model whose performance is better than the performance of the first candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1; Inputting the training data set into the multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

5. The method according to claim 4, characterized in that The Bayesian optimization algorithm is used to predict, based on the performance of the first candidate model to the performance of the i-1th candidate model, the i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model from the hypernetwork, including: Dividing the first candidate model to the i-1th candidate model according to performance to obtain a first set and a second set, the first set including candidate models whose performance is better than the first performance among the first candidate model to the i-1th candidate model, and the second set including candidate models whose performance is lower than or equal to the first performance among the first candidate model to the i-1th candidate model; Counting a plurality of first numbers and a plurality of second numbers, wherein the plurality of first numbers are the number of times the values ​​of the structure parameter set appear in the first set, and the plurality of second numbers are the number of times the values ​​of the structure parameter set appear in the second set, and the structure parameter set includes the structure parameters to be optimized in the basic model; Based on the multiple first numbers and the multiple second numbers, an i-th candidate model having performance better than the performance of the first candidate model to the i-1-th candidate model is predicted from the hypernetwork.

6. The method according to claim 5, characterized in that The step of predicting, based on the plurality of first numbers and the plurality of second numbers, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, comprises: Based on the multiple first numbers, a plurality of first frequencies are obtained, wherein the multiple first frequencies are frequencies at which the values ​​of the structure parameter set appear in the first set; Obtaining a plurality of second frequencies based on the plurality of second numbers, wherein the plurality of second frequencies are frequencies at which the values ​​of the structure parameter set appear in the second set; Based on the multiple first frequencies and the multiple second frequencies, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model is predicted from the hypernetwork.

7. The method according to claim 6, characterized in that The method of predicting, based on the multiple first frequencies and the multiple second frequencies, from the hypernetwork, an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1-th candidate model, comprises: Calculating a quotient corresponding to each value among multiple values ​​of the j-th structural parameter in the structural parameter set, wherein the quotient corresponding to each value is a quotient of a first frequency corresponding to each value and a second frequency corresponding to each value; The value with the largest corresponding quotient among multiple values ​​of the j-th structural parameter is determined as the value of the j-th structural parameter in the i-th candidate model whose performance is better than the performance of the first candidate model to the performance of the i-1-th candidate model.

8. The method according to any one of claims 1 to 7, characterized in that: The method of splitting the first candidate model to obtain the multiple first candidate model substructures includes at least one of tensor parallelism and pipeline parallelism, and the method of splitting the second candidate model to obtain the multiple second candidate model substructures is the same as the method of splitting the first candidate model to obtain the multiple first candidate model substructures.

9. The method according to any one of claims 1 to 8, characterized in that: The performance includes at least one of running time, accuracy, and parameter quantity.

10. The method according to any one of claims 1 to 9, characterized in that: The basic model is a translator Transformer network, or the basic model is a convolutional neural network CNN.

11. The method according to any one of claims 1 to 10, characterized in that: The multiple computing nodes include any one or more of a graphics processing unit GPU, a tensor processing unit TPU and a neural network processor NPU.

12. The method according to any one of claims 4 to 11, characterized in that After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the method further includes: The Kth candidate model is sent to a terminal device of the user, where the Kth candidate model is used to process the text data of the user and output the processed text data.

13. An AI model super network optimization device, characterized in that: The device comprises: A segmentation module, used for segmenting the first candidate model to obtain a plurality of first candidate model substructures, and segmenting the second candidate model to obtain a plurality of second candidate model substructures, wherein the first candidate model and the second candidate model are candidate models of different structures obtained by structurally changing the same basic model; a distribution module, configured to distribute the plurality of first candidate model substructures among a plurality of computing nodes, and to distribute the plurality of second candidate model substructures among the plurality of computing nodes; The optimization module is used to iteratively optimize the multiple first candidate model substructures and the multiple second candidate model substructures. During the optimization process, a weight sharing method is adopted to make the multiple first candidate model substructures and the multiple second candidate model substructures form a super network.

14. The device according to claim 13, characterized in that The first candidate model and the second candidate model both belong to a candidate model set, the candidate model set is a set of candidate models with different structures obtained by performing structural changes based on the same basic model, and the multiple substructures obtained by segmenting each candidate model in the candidate model set except the first candidate model and the second candidate model are also distributed in the multiple computing nodes; The optimization module is specifically used to iteratively optimize the substructures of all candidate models in the candidate model set. During the optimization process, a weight sharing method is used to make the substructures of all candidate models in the candidate model set form a super network.

15. The device according to claim 14, characterized in that The optimization module is specifically used for: A Bayesian optimization algorithm is used to iteratively optimize the substructures of all candidate models in the candidate model set.

16. The device according to claim 15, characterized in that The optimization module is specifically used for: Using a Bayesian optimization algorithm, based on the performance of the first candidate model to the performance of the i-1th candidate model, predicting from the hypernetwork an i-th candidate model having performance better than the performance of the first candidate model to the performance of the i-1th candidate model, 1≤i≤K, K is the number of times the substructures of all candidate models in the candidate model set are iteratively optimized, and K is an integer greater than 1; Inputting the training data set into the multiple computing nodes to train the i-th candidate model, and obtaining the gradient corresponding to the i-th candidate model; Based on the gradient corresponding to the i-th candidate model, the weight of the hypernetwork is updated.

17. The device according to claim 16, characterized in that The device also includes a sending module; After completing the iterative optimization of the substructures of all candidate models in the candidate model set, the sending module is used to send the Kth candidate model to the user's terminal device, and the Kth candidate model is used to process the user's text data and output the processed text data.

18. A management node, characterized in that: The management node includes a processor and a memory, the memory stores codes, and the processor executes the codes to implement the method according to any one of claims 1 to 12.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device, the computing device performs the method according to any one of claims 1 to 12.

20. A computing system, characterized in that: The computing system includes a management node and multiple computing nodes, the management node is used to execute the method as described in any one of claims 1 to 12, and the multiple computing nodes are used to undertake the training operation of the candidate model in the method as described in any one of claims 1 to 12.

Images