Weight updating method and device for AI model

Abstract

The weight updating method for the AI model comprises the steps that a target mapping relation is obtained, the target mapping relation indicates a mapping relation between a first weight parameter and a second weight parameter, the first weight parameter is a weight parameter of a training model, the training model is an AI model obtained after a target AI model is subjected to initial training in a training stage, and the second weight parameter is a weight parameter of a training model; the second weight parameter is a weight parameter of the reasoning model; the inference model is an AI model of the target AI model in an inference stage, and the inference model is obtained based on training model processing; when the target AI model is retrained or fine-tuned, determining an updated first weight parameter of the target AI model; and updating a second weight parameter of the reasoning model based on the updated first weight parameter and the target mapping relation. The mapping relation of the weight parameters before and after reasoning processing is recorded. When the weight is updated, the weight of the inference model is updated only through the target mapping relation. The work of re-compiling the model and the like is avoided, and the model development efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for updating the weights of an AI model. Background Art

[0002] With the development of artificial intelligence (AI), the application of AI models is becoming more and more common. In the reasoning stage of AI models, scenarios where model weights need to be updated are often encountered. In the current technical solutions, the updated trained model is often recompiled and processed to obtain a model for performing reasoning, which is very inefficient. Especially when updating the weights of large models, there is also the problem of re-modeling, model compilation, model deployment and other processing, which causes the AI ​​model weight update process to take a long time. Summary of the invention

[0003] In order to solve the above-mentioned problems, a model weight updating method and device are provided in the embodiments of the present application. When updating the weight of the inference model, the step of processing the training model is omitted, the time required for weight updating is shortened, and the efficiency of weight updating is improved.

[0004] To this end, the following technical solutions are adopted in the embodiments of the present application:

[0005] In a first aspect, an embodiment of the present application provides a method for updating the weight of an AI model, including: obtaining a target mapping relationship, the target mapping relationship indicating a mapping relationship between a first weight parameter and a second weight parameter, the first weight parameter being a weight parameter of a training model, the training model being an AI model of a target AI model after initial training in a training phase, and the second weight parameter being a weight parameter of an inference model; the inference model being an AI model of a target AI model in an inference phase, and the inference model being obtained based on the training model; when the target AI model is retrained or fine-tuned, determining the first weight parameter of the target AI model after retraining or fine-tuning in the training phase; and, based on the updated first weight parameter and the target mapping relationship, updating the second weight parameter of the inference model corresponding to the target AI model.

[0006] That is to say, in the application scenario of the AI ​​model, it usually includes two stages: training and reasoning. After the first training of the model is completed, in the reasoning stage, the trained model needs to be processed to obtain a model for performing reasoning. In the method of the embodiment of the present application, the model structure changes of the processing process are recorded to obtain the correspondence between the weight parameters before and after the processing, that is, the target mapping relationship. In the embodiment of the present application, when the model needs to update the weight parameters again, such as retraining or fine-tuning the model, it is only necessary to find the corresponding weight parameters in the reasoning model through the target mapping relationship for update. Therefore, the work of re-model compilation during weight update is avoided, the efficiency of weight update of the reasoning model is improved, and the efficiency of AI model development and debugging is improved.

[0007] In one possible implementation, the inference model is deployed on a single card, and the processing of the training model includes model compilation; obtaining the target mapping relationship, including: recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameter; recording the weight parameters of the training model after model compilation to obtain the second weight parameter; based on the first weight parameter and the second weight parameter, determining the target mapping relationship.

[0008] In this implementation, the embodiment of the present application provides an application scenario of single-card reasoning or small-parameter model reasoning. A small-parameter model refers to an AI model with fewer parameters, which can be deployed on a single card for reasoning. In this application scenario, the trained model needs to be compiled and then deployed on a single card in the reasoning stage. In the method of the embodiment of the present application, the model structure changes before and after the model compilation process are recorded to obtain the corresponding relationship between the weight parameters before and after the model compilation process, that is, the target mapping relationship. In the embodiment of the present application, when the AI ​​model needs to update the weight parameters again, such as retraining or fine-tuning the AI ​​model, it is only necessary to find the corresponding required weight parameters in the reasoning model through the target mapping relationship for update. Therefore, the processing work of re-model compilation during weight update in the single-card application scenario is avoided, the efficiency of weight update of the reasoning model is improved, and the efficiency of AI model development and debugging is improved.

[0009] In another possible implementation, the inference model can be deployed on multiple cards, and the processing of the training model includes model segmentation and model compilation; obtaining the target mapping relationship includes: recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; recording the weight parameters of the training model after the model segmentation processing to obtain the third weight parameters; determining the first mapping relationship based on the first weight parameters and the third weight parameters; recording the weight parameters of the training model after the model compilation processing to obtain the second weight parameters; determining the second mapping relationship based on the third weight parameters and the third weight parameters; determining the target mapping relationship based on the first mapping relationship and the second mapping relationship.

[0010] In this implementation, the embodiment of the present application provides an application scenario of multi-card reasoning or large parameter model reasoning. A large parameter model refers to an AI model with many parameters, which needs to be deployed on multiple cards for reasoning, such as some common large AI models. In this application scenario, it is necessary to divide the trained model, compile the model, and finally deploy the model on multiple cards for reasoning. In the method of the embodiment of the present application, the model structure changes of each model processing stage are recorded to obtain the corresponding relationship of the weight parameters before and after each model processing stage. The set of corresponding relationships of the weight parameters before and after the processing of each processing stage is the target mapping relationship; or the corresponding relationship of the weight parameters before and after the processing of the entire reasoning stage is derived to obtain the target mapping relationship. In the embodiment of the present application, when the AI ​​model needs to update the weight parameters again, such as retraining or fine-tuning the AI ​​model, it is only necessary to find the corresponding weight parameters that need to be updated in the reasoning model through the target mapping relationship. Therefore, in the multi-card application scenario, the processing work such as re-model division and re-model compilation during weight update is avoided, the efficiency of weight update of the reasoning model is improved, and the efficiency of AI model development and debugging is improved.

[0011] In another possible implementation, the model segmentation includes operator segmentation. Based on the first weight parameter and the third weight parameter, determining the first mapping relationship includes: segmenting the first operator into at least the second operator and the third operator; deploying the second operator and the third operator on different cards; the parameters of the first operator include the first weight parameter, and the parameters of the second operator and the third operator include the third weight parameter; determining the first mapping relationship based on the mapping relationship between the parameters of the first operator and the parameters of the second operator and the third operator.

[0012] In this implementation, an embodiment of the present application provides an example of model segmentation for multi-card reasoning, namely operator segmentation. Taking the multiplication operator as an example, assume that a large matrix multiplication is segmented by the operator during the segmentation process. Then the matrix of the sample data will be segmented, and the corresponding original weights will also be segmented. Before and after the operator segmentation process, the mapping relationship of the weights is recorded, and when the weights before the operator segmentation are updated, the corresponding weights after the operator segmentation can be directly updated based on the recorded mapping relationship. This avoids re-segmenting the operator, saves time for weight updates, and improves the work efficiency of model development.

[0013] In another possible implementation, model compilation includes operator fusion. Obtaining a target mapping relationship includes: fusing at least a fourth operator and a fifth operator to obtain a sixth operator; the parameters of the fourth operator and the fifth operator include a first weight parameter, and the parameters of the sixth operator include a second weight parameter; and determining a target mapping relationship based on a mapping relationship between the parameters of the fourth operator, the fifth operator, and the parameters of the sixth operator.

[0014] In this implementation, an embodiment of the present application provides an example of model compilation, namely operator fusion. Operator fusion is an example of graph optimization and graph merging. During the processing of operator fusion, the corresponding mapping relationship of the weight parameters before and after the processing is recorded. When the weight parameters before the operator fusion change, the corresponding weight parameters after the operator fusion can be directly updated based on the mapping relationship without the need to re-perform the operator fusion. Therefore, the time for weight update is saved and the work efficiency of model development is improved.

[0015] In another possible implementation, the target AI model is retrained or fine-tuned, including: adding training data to incrementally train the training model process to retrain the target AI model; or fine-tuning the target AI model by incrementally training the network layer of the training model; the network layer includes a fully connected layer.

[0016] In this implementation, the scenario of retraining or fine-tuning is illustrated. In practical applications, the method of the embodiment of the present application can be adapted to all scenarios where weights are updated in the inference phase.

[0017] In the second aspect, an embodiment of the present application provides a weight updating device for an AI model, comprising: an acquisition module, used to acquire a target mapping relationship, the target mapping relationship indicates a mapping relationship between a first weight parameter and a second weight parameter, the first weight parameter is a weight parameter of a training model, the training model is an AI model of the target AI model after initial training in the training phase, and the second weight parameter is a weight parameter of an inference model; the inference model is an AI model of the target AI model in the inference phase, and the inference model is obtained based on the training model; a weight updating module, used to determine the first weight parameter of the target AI model after retraining or fine-tuning in the training phase when the target AI model is retrained or fine-tuned; and, used to update the second weight parameter of the inference model corresponding to the target AI model based on the updated first weight parameter and the target mapping relationship.

[0018] In one possible implementation, the inference model is deployed on a single card, and the processing of the training model includes model compilation; an acquisition module is specifically used to: record the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; record the weight parameters of the training model after model compilation to obtain the second weight parameters; based on the first weight parameters and the second weight parameters, determine the target mapping relationship.

[0019] In another possible implementation, the inference model is used to be deployed on multiple cards, and the processing of the training model includes model segmentation and model compilation; the acquisition module is specifically used to: record the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; record the weight parameters of the training model after the model segmentation processing to obtain the third weight parameters; determine the first mapping relationship based on the first weight parameters and the third weight parameters; record the weight parameters of the training model after the model compilation processing to obtain the second weight parameters; determine the second mapping relationship based on the third weight parameters and the third weight parameters; determine the target mapping relationship based on the first mapping relationship and the second mapping relationship.

[0020] In another possible implementation, model segmentation includes operator segmentation; an acquisition module is specifically used to: segment the first operator into at least a second operator and a third operator; the second operator and the third operator are deployed on different cards; the parameters of the first operator include a first weight parameter, and the parameters of the second operator and the third operator include a third weight parameter; based on the mapping relationship between the parameters of the first operator and the parameters of the second operator and the third operator, determine the first mapping relationship.

[0021] In another possible implementation, model compilation includes operator fusion; an acquisition module is specifically used to: fuse at least a fourth operator and a fifth operator to obtain a sixth operator; the parameters of the fourth operator and the fifth operator include a first weight parameter, and the parameters of the sixth operator include a second weight parameter; based on the mapping relationship between the parameters of the fourth operator, the fifth operator and the parameters of the sixth operator, determine the target mapping relationship.

[0022] In another possible implementation, the weight update module is specifically used to: increase the training data to incrementally train the training model process to retrain the target AI model; or to fine-tune the target AI model by incrementally training the network layer of the training model; the network layer includes a fully connected layer.

[0023] In a third aspect, the present application provides a cloud management platform, comprising: at least one computing node for performing the reasoning phase of the AI ​​model; in the reasoning phase, based on the above method, the second weight parameter of the AI ​​model is updated.

[0024] In a fourth aspect, the present application provides a computing device, including a memory and a processor. The memory stores instructions, and when the instructions are executed by the processor, the above method is implemented.

[0025] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above method is implemented.

[0026] In a sixth aspect, the present application provides a computer program product, which includes program instructions, and when the program instructions are executed by a computer, the computer executes a method. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The following is a brief introduction to the drawings required for use in the embodiments or technical descriptions.

[0028] Figure 1 A schematic diagram of the single-card reasoning process provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the multi-card reasoning process provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of a weight updating method of an AI model provided in an embodiment of the present application;

[0031] Figure 4 A flowchart of a weight updating method for single-card reasoning provided in an embodiment of the present application;

[0032] Figure 5 A flowchart of a weight updating method for multi-card reasoning provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of a flow chart of operator segmentation provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of a flow chart of operator fusion provided in an embodiment of the present application;

[0035] Figure 8 A flowchart of a weight updating process provided in an embodiment of the present application;

[0036] Fig. 9 A schematic diagram of the composition of a weight updating device for an AI model provided in an embodiment of the present application;

[0037] Fig.10 A schematic diagram of the composition of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0039] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an "or" relationship, for example, A / B means A or B.

[0040] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0041] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0042] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0043] In order to facilitate understanding of the solution provided by the embodiment of the present application, some terms involved in the solution are first briefly introduced.

[0044] Computing power: The ability of a computer device or computing / data center to process information.

[0045] AI model: In the embodiments of this application, it is also referred to as a model, neural network model, etc. AI model refers to a computer program built through techniques such as machine learning and deep learning, which can analyze, infer and predict data. Based on a large amount of training data and algorithms, AI models can automatically learn and extract features from input data, and make decisions and predictions based on the learned knowledge.

[0046] Model compilation: refers to the process of deploying models trained by various training frameworks to different terminal hardware. This process can be achieved through a deep learning neural network compiler. The basic idea of ​​machine learning compilation is transformation: perform equivalent transformations on the same computing logic to make memory access more friendly and adapt to the special instruction set of the underlying hardware as much as possible. Model compilation includes graph optimization, operator optimization, etc.

[0047] Neural network compiler: Its input is the model definition file trained by the deep learning training framework, and its output is code that can be efficiently executed on different hardware.

[0048] Distributed reasoning or multi-card reasoning: When there is a large amount of training data or large-scale models that cannot be stored and calculated by a single machine, the data or model needs to be divided and distributed to each working node or chip.

[0049] Model partitioning: For the inference stage of a large model, it is necessary to divide the large model into several sub-models, and then place each sub-model on a working node for calculation. Model parallelism, that is, the way of model partitioning, is divided into two types: pipeline parallelism and tensor parallelism. Pipeline model parallelism is to place different layers of the model on different devices, such as the first few layers on one device, the middle layers on another device, and the last few layers on a third device. Tensor parallelism is intra-layer partitioning, which divides a certain layer and places it on different devices. It can also be understood as allocating matrix operations to different devices, such as dividing a matrix multiplication into multiple matrix multiplications and placing them on different devices.

[0050] Weight parameters: Model parameters including weights. In a neural network model, weight parameters include weights, biases, scale factors, etc. During training, weight parameters include parameters that are updated.

[0051] Taking deep learning as an example, the model parameters of deep learning have exceeded one trillion. With the development of artificial intelligence, the number of parameters of neural network models will become larger and larger. The huge number of parameters has brought great challenges to the training and reasoning stages of neural network models. The training and reasoning of large models are very complex projects. Among them, how to efficiently update the weights in the trained large model is a key technology in the reasoning stage. A large model refers to a neural network model with a large number of parameters.

[0052] According to theoretical calculations of current hardware, a single card may not be able to complete the reasoning task for models with huge parameter scales, so the common practice is multi-card reasoning, that is, splitting the model and then reasoning on the split modules in blocks.

[0053] Figure 1 Schematic diagram of the single card reasoning process provided in the embodiment of this application. Figure 1 As shown in the figure, the general process of single-card reasoning in the training phase includes: training the model based on the training data to obtain a trained model, which includes the model structure and model parameters, and the model parameters include weights. The general process of single-card reasoning in the reasoning phase includes: compiling the model structure and model parameters including graph optimization and operator optimization, and using the compiled model to perform reasoning in the business reasoning program.

[0054] In the single-card inference solution, when the model on the training side is slightly adjusted, the inference side needs to recompile the model. Although there is a technical solution that does not require recompilation and directly updates the weights, this solution is only applicable to models with smaller parameter rules, such as models below 5B, where B is a unit of billion.

[0055] For example, taking 80G video memory as an example, the model size is usually 4 times the number of parameters. According to the Float calculation amount, the number of parameters is 80 / 4=20B, so the theoretical limit of single-card model reasoning is a 20B model. According to the Float calculation amount, the number of parameters is 80 / (4*4)=5B, so the theoretical limit of single-card model training is a 5B model. Therefore, for large models with tens of billions or trillions of parameters, multiple cards are required for training and reasoning. The training and reasoning of large models are both very complex projects. How to efficiently update weights during the reasoning process of trained large models is a key technology.

[0056] Figure 2 1 is a flow chart of multi-card reasoning provided in the embodiment of the present application. Multi-card reasoning is also called distributed reasoning. Figure 2 As shown in the figure, the general process in the training phase includes: distributing the model on multiple chips for model training, and merging the trained models. The general process in the inference phase includes: segmenting the merged model; dividing the graph structure of the neural network to obtain multiple subgraphs; compiling the multiple subgraphs respectively; deploying the compiled subgraphs to the corresponding models respectively; and using the deployed model in the business inference program to perform inference work.

[0057] In the Duoka inference solution, when the model on the training side is slightly adjusted, the inference side needs to re-divide and recompile the model. Although there is a technical solution that does not require recompilation and directly updates the weights, this solution is only for models with smaller parameter rules, such as models below 5B; this solution also cannot update the weights of subgraphs after the neural network graph structure is divided.

[0058] Therefore, each time the weight of a large model is updated, the process of cutting the graph, compiling, and deploying is required. Therefore, for example, each weight update takes about 5 hours, and the efficiency of development and debugging is relatively low.

[0059] In other words, the basic characteristics of the inference technology solution for large models include: the deployment of large models requires card division or distributed reasoning; large models need to be split and the model compiled after splitting.

[0060] Therefore, the segmentation of large models and the compilation of the models after segmentation are time-consuming. For example, for a model with 33B parameters, the model compilation takes more than 5 hours. For example, if the parameters are updated after model training, each update requires re-segmentation and model compilation, which is a very time-consuming process for the development, debugging and deployment of a model. For example, when debugging the accuracy and performance of some models, developers need to update parameters repeatedly, so they need to perform the model segmentation and model compilation process multiple times.

[0061] Therefore, a technical solution is needed to solve the problem of time-consuming model weight update in the inference phase.

[0062] In order to solve the problem of long time consumption when updating model weights in the inference phase, such as Figure 3 As shown, a model weight update method is provided in the embodiment of the present application. The model weight update method saves the time-consuming recompilation each time, improves the development and debugging efficiency of large models, and accelerates the weight update process of large models. Figure 3 As shown, the embodiment of the present application provides a weight updating method of an AI model, which mainly includes the following steps:

[0063] Step S301, obtain the target mapping relationship, the target mapping relationship indicates the mapping relationship between the first weight parameter and the second weight parameter, the first weight parameter is the weight parameter of the training model, the training model is the AI ​​model of the target AI model after the initial training is completed in the training phase, and the second weight parameter is the weight parameter of the inference model; the inference model is the AI ​​model of the target AI model in the inference phase, and the inference model is obtained based on the training model.

[0064] Step S302, when the target AI model is retrained or fine-tuned, determine the first weight parameter of the target AI model after retraining or fine-tuning during the training phase; and, based on the updated first weight parameter and the target mapping relationship, update the second weight parameter of the inference model corresponding to the target AI model.

[0065] It can be further understood that, in the process from the completion of the first training to the completion of the inference deployment, the mapping relationship of the weight parameters, that is, the target mapping relationship, is recorded. After obtaining the target mapping relationship, whenever the first weight parameter is updated, the corresponding second weight parameter can be updated based on the obtained target mapping relationship. Or whenever the first weight parameter is updated, steps S301 and S302 are executed, that is, the target mapping relationship is first obtained, and then the corresponding second weight parameter is updated based on the obtained target mapping relationship.

[0066] Figure 4This is a flow chart of a model weight update method in a single card reasoning scenario provided in an embodiment of the present application. In a single card reasoning scenario, in the business reasoning program, the trained model needs to be compiled, and the model that has been optimized by graph optimization, operator optimization, etc. can be more friendly to the hardware.

[0067] like Figure 4 As shown, in the model weight updating method of the embodiment of the present application, the mapping relationship between the weights before and after compilation is recorded. For example, the weights before compilation are weight a and weight b. The compilation process is optimized through the operator fusion. The weights a and weight b are changed to weight c. Then, the mapping relationship between the weights before and after compilation is the mapping relationship between weight a, weight b and weight c with operator fusion.

[0068] It can be further understood that, assuming that the model input in the inference stage is the training model, and the model used for the inference process in the inference stage is the inference model, the mapping relationship between the weights before and after compilation can be understood as the mapping relationship between the training model and the inference model. Exemplarily, assuming that the multiple weights of the training model are vector x, and the multiple weights of the inference model are vector y, in the inference stage, the compilation process of processing the training model to obtain the inference model is optimized through an operator fusion, and weights a and b are changed to weight c. Then the mapping relationship between the weights before and after compilation is the mapping relationship between weights a, weight b and weight c with operator fusion. By recording the mapping relationship between the weights before and after compilation,

[0069] Figure 5 A flow chart of a model weight update method in a multi-card reasoning scenario provided in an embodiment of the present application. In a multi-card reasoning scenario, in the business reasoning program, the trained model needs to go through the process of graph slicing, model compilation, and model deployment. Graph slicing refers to dividing the graph structure of the neural network to obtain the graph structure for reasoning on each chip. After graph slicing, model compilation, and model deployment, each subgraph of the compilation number can be deployed on each chip. After graph slicing, model compilation, and model deployment, the model can be more friendly to the hardware.

[0070] like Figure 5 As shown, in the model weight updating method of the embodiment of the present application, in the training stage, the trained model is a merged model of the models trained on multiple chips. In the reasoning stage, the merged model needs to be processed by cutting the graph, compiling the model, deploying the model, etc. to obtain a model for reasoning. In the embodiment of the present application, the change relationship of the weights during the processing of the reasoning stage is recorded to obtain the mapping relationship between the weights before and after the processing. Based on the mapping relationship of the weights, when new weight parameters are obtained through training, the corresponding weight values ​​in the inference model can be directly updated, saving the processing processes of re-cutting the graph, compiling the model, and deploying the model.

[0071] In the embodiments of the present application, for example, during the first subgraph splitting, subgraph optimization compilation, subgraph merging and other links, the mapping relationship and mapping formula of the model structure of each link are recorded. In the subsequent stage, if there is a need to update the weights of the training model, the previous mapping relationship and mapping formula can be reused to directly assign values ​​to the weights of the inference model to achieve the operation of updating the inference model. In other words, the purpose of quickly updating the weights each time is achieved without having to go through several hours of subgraph splitting, subgraph optimization / compilation, subgraph merging and other links.

[0072] Figure 6 Schematic diagram of an operator segmentation process in the model segmentation stage provided in the embodiment of the present application. Figure 6 As shown in the figure, after a large model is trained, it is necessary to split the model in tensor parallel or pipeline parallel, and split a model into M sub-models, which are placed on M cards for deployment and inference. Assume that a large matrix multiplication is performed during the splitting process. Then the original weight W will also be split into W0 to WM-1. Among them, M is a positive integer.

[0073] For example, Figure 6 As shown, a segmentation algorithm based on operator segmentation, such as a tensor parallel or pipeline parallel algorithm, is used to segment operator a into operator 1a, operator 2a, operator 3a, etc., and then the segmented operators are deployed on the corresponding chips. In the inference stage, the input X is segmented by the segmentation algorithm, and the segmented input X is calculated by the segmented operators respectively, and the calculation results are output to the inference result Y through the merging algorithm. Operator a can be a matrix multiplication operator.

[0074] Figure 7 Schematic diagram of an operator fusion process in the model compilation stage provided in the embodiment of the present application. Figure 7 As shown in the figure, each segmented sub-model will perform some optimization operations including operator fusion according to the structure of the model.

[0075] For example, Figure 7 As shown in the figure, in the training phase, the input X needs to pass through the convolution operator 1 and the normalization (BatchNorm, BN) operator to output Y. In the inference phase, the convolution operator 1 and the BN operator can be fused into the convolution operator 2. Such fusion can reduce the frequency of copying data in and out of the cache during the inference phase, thereby improving the inference speed of the entire model.

[0076] f(x)=WX+B (1)

[0077] In formula (1), W represents the weight matrix of convolution operator 1, X represents input matrix data, B represents bias, and f(x) represents output matrix data.

[0078]

[0079] In formula (2), r represents the linear ratio, β represents the linear bias, X represents the input matrix data, μ represents the mean of the input matrix data, and σ 2 represents the variance of the input matrix data, ∈ is a very small positive number to prevent division by zero errors, and f(x) represents the output matrix data. Weights, r, β are used to linearly scale the learning parameters.

[0080]

[0081] For example, convolution operator 1 is shown in formula (1), BN operator is shown in formula (2), and the fused convolution operator 2 is shown in formula (3). That is, before the operator fusion, the weight parameters of convolution operator 1 and BN operator are (W, B), (r, μ, σ, ∈, β) respectively; after the fusion, the weight parameters of convolution operator 2 are From weight parameters (W, B), (r, μ, σ, ∈, β) to weight parameters The mapping relationship is an example of the weight mapping relationship of the records in the embodiment of the present application.

[0082] Figure 8 This is a flow chart of a record mapping relationship in the weight update phase provided in the embodiment of the present application. Figure 8 As shown, when the model trained for the Nth time on the training side (i.e., the training model) outputs updated weight parameters, the inference side performs model segmentation, model compilation and other processing based on the updated weight parameters to obtain an inference model, where N is an integer greater than or equal to 1. In an embodiment of the present application, the processing process from the training model to the inference model is recorded to obtain a mapping relationship between the weights before processing and the weights after processing. When the model trained for the N+Pth time on the training side (i.e., the training model) outputs updated weight parameters, the inference side directly updates the weights of the inference model based on the recorded mapping relationship of the weights, where N is an integer greater than or equal to 1. The embodiment of the present application omits the process of performing model segmentation, model compilation and other processing on the training model to obtain an inference model, thereby improving the efficiency of weight updating and the efficiency of the inference stage.

[0083] Alternatively, in deep learning applications, a common fine-tuning method is to incrementally train the last fully connected layer of the model without changing the model structure, or simply add a part of the training data to incrementally train the model process. Figure 8As shown, the weight mapping relationship of each stage can be recorded in the first model segmentation and model compilation stage (i.e. N is 1). Then, for all weight updates in the second and subsequent stages, only the first mapping formula needs to be applied to update and assign the model weights, without the need to re-segment the model and perform the model fusion process.

[0084] In an embodiment of the present application, for models with extremely large parameters, when updating the parameters, the steps of re-segmenting the model, compiling the model, deploying the model, etc. can be omitted, thereby efficiently updating the weight parameters; for ordinary small models, or models with single-card reasoning, the steps of re-compilation of the model, etc. can be omitted, which can improve the efficiency of weight updating in the reasoning stage.

[0085] In the embodiments of the present application, it can be used in some model fine-tuning scenarios, and can also be applied to weight update scenarios for unilateral fine-tuning.

[0086] like Fig. 9 As shown, the embodiment of the present application provides a weight updating device for an AI model, which mainly includes:

[0087] Acquisition module 901 is used to acquire a target mapping relationship, where the target mapping relationship indicates a mapping relationship between a first weight parameter and a second weight parameter, where the first weight parameter is a weight parameter of a training model, where the training model is an AI model of the target AI model after initial training in the training phase, and the second weight parameter is a weight parameter of an inference model; the inference model is an AI model of the target AI model in the inference phase, and the inference model is obtained based on the training model.

[0088] The weight update module 902 is used to determine the first weight parameter of the target AI model after retraining or fine-tuning during the training phase when the target AI model is retrained or fine-tuned; and to update the second weight parameter of the inference model corresponding to the target AI model based on the updated first weight parameter and the target mapping relationship.

[0089] In one possible implementation, the inference model is deployed on a single card, and the processing of the training model includes model compilation; the acquisition module 901 is specifically used to: record the weight parameters of the training model after the initial training is completed, and obtain the first weight parameters; record the weight parameters of the training model after model compilation processing, and obtain the second weight parameters; based on the first weight parameters and the second weight parameters, determine the target mapping relationship.

[0090] In another possible implementation, the inference model is used to be deployed on multiple cards, and the processing of the training model includes model segmentation and model compilation; the acquisition module 901 is specifically used to: record the weight parameters of the training model after the initial training is completed, and obtain the first weight parameters; record the weight parameters of the training model after the model segmentation processing, and obtain the third weight parameters; determine the first mapping relationship based on the first weight parameters and the third weight parameters; record the weight parameters of the training model after the model compilation processing, and obtain the second weight parameters; determine the second mapping relationship based on the third weight parameters and the third weight parameters; determine the target mapping relationship based on the first mapping relationship and the second mapping relationship.

[0091] In another possible implementation, model segmentation includes operator segmentation; an acquisition module 901 is specifically used to: segment the first operator into at least a second operator and a third operator; the second operator and the third operator are deployed on different cards; the parameters of the first operator include a first weight parameter, and the parameters of the second operator and the third operator include a third weight parameter; based on the mapping relationship between the parameters of the first operator and the parameters of the second operator and the third operator, determine the first mapping relationship.

[0092] In another possible implementation, model compilation includes operator fusion; an acquisition module 901 is specifically used to: fuse at least the fourth operator and the fifth operator to obtain a sixth operator; the parameters of the fourth operator and the fifth operator include a first weight parameter, and the parameters of the sixth operator include a second weight parameter; based on the mapping relationship between the parameters of the fourth operator, the fifth operator and the parameters of the sixth operator, determine the target mapping relationship.

[0093] In another possible implementation, the weight update module 902 is specifically used to: increase the training data to incrementally train the training model process to retrain the target AI model; or to fine-tune the target AI model by incrementally training the network layer of the training model; the network layer includes a fully connected layer.

[0094] Based on the same concept as the aforementioned embodiment, a computing device is also provided in the embodiment of the present application. The computing device includes at least a processor and a memory, and a program is stored in the memory. When the processor executes the program, the algorithm function embodied by the above method or device can be implemented.

[0095] Fig.10 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. Fig.10As shown, the computing device 1000 includes at least one processor 1001, a memory 1002 and a communication interface 1003. The processor 1001, the memory 1002 and the communication interface 1003 are connected in communication, and the communication connection can be realized by wired means (such as a bus) or by wireless means. The communication interface 1003 is used to receive data sent by other devices (such as packet information sent by the cloud management platform); the memory 1002 stores computer instructions, and the processor 1001 executes the computer instructions to execute the method in the aforementioned method embodiment.

[0096] It should be understood that in the embodiment of the present application, the processor 1001 may be a central processing unit CPU, and the processor 1401 may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0097] The memory 1002 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1001. The memory 1002 may also include a nonvolatile random access memory.

[0098] The memory 1002 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct ram bus RAM (DR RAM).

[0099] It should be understood that the computing device 1000 according to the embodiment of the present application can execute the method mentioned in the embodiment of the present application. The detailed description of the implementation of the method is shown above, and for the sake of brevity, it will not be repeated here.

[0100] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer instructions are executed by a processor, the above-mentioned technical solution is implemented.

[0101] An embodiment of the present application provides a chip, which includes at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the technical solution mentioned above.

[0102] An embodiment of the present application provides a computer program or a computer program product, which includes instructions. When the instructions are executed, the computer executes the technical solution mentioned above.

[0103] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented by hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0105] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A weight updating method for an AI model, characterized in that: include: Obtain a target mapping relationship, where the target mapping relationship indicates a mapping relationship between a first weight parameter and a second weight parameter, where the first weight parameter is a weight parameter of a training model, where the training model is an AI model of a target AI model after initial training in a training phase, and the second weight parameter is a weight parameter of an inference model; the inference model is an AI model of the target AI model in an inference phase, and the inference model is obtained based on the training model; When the target AI model is retrained or fine-tuned, determine the first weight parameter of the target AI model after retraining or fine-tuning during the training phase; and based on the updated first weight parameter and the target mapping relationship, update the second weight parameter of the inference model corresponding to the target AI model.

2. The method according to claim 1, characterized in that: The inference model is used to be deployed on a single card, and the processing of the training model includes model compilation; The obtaining of the target mapping relationship includes: Recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; Recording the weight parameters of the training model after the model compilation process to obtain the second weight parameters; The target mapping relationship is determined based on the first weight parameter and the second weight parameter.

3. The method according to claim 1, characterized in that The inference model is used to be deployed on multiple cards, and the processing of the training model includes model segmentation and model compilation; The obtaining of the target mapping relationship includes: Recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; Recording the weight parameters of the training model after the model segmentation process to obtain a third weight parameter; Determining a first mapping relationship based on the first weight parameter and the third weight parameter; Recording the weight parameters of the training model after the model compilation process to obtain the second weight parameters; Determining a second mapping relationship based on the third weight parameter and the third weight parameter; The target mapping relationship is determined based on the first mapping relationship and the second mapping relationship.

4. The method according to claim 3, characterized in that: The model segmentation includes operator segmentation; The determining a first mapping relationship based on the first weight parameter and the third weight parameter includes: The first operator is divided into at least a second operator and a third operator; the second operator and the third operator are deployed on different cards; the parameters of the first operator include the first weight parameter, and the parameters of the second operator and the third operator include the third weight parameter; The first mapping relationship is determined based on a mapping relationship between parameters of the first operator and parameters of the second operator and the third operator.

5. The method according to any one of claims 2 to 4, characterized in that: The model compilation includes operator fusion; The obtaining of the target mapping relationship includes: After at least the fourth operator and the fifth operator are merged, a sixth operator is obtained; the parameters of the fourth operator and the fifth operator include the first weight parameter, and the parameters of the sixth operator include the second weight parameter; The target mapping relationship is determined based on the mapping relationship between the parameters of the fourth operator, the fifth operator and the parameter of the sixth operator.

6. The method according to any one of claims 1 to 5, characterized in that: The target AI model is retrained or fine-tuned, including: Adding training data to incrementally train the training model process to retrain the target AI model; Alternatively, the target AI model is fine-tuned by incrementally training the network layer of the training model; the network layer includes a fully connected layer.

7. A weight updating device for an AI model, characterized in that: include: An acquisition module, used to acquire a target mapping relationship, wherein the target mapping relationship indicates a mapping relationship between a first weight parameter and a second weight parameter, wherein the first weight parameter is a weight parameter of a training model, wherein the training model is an AI model of a target AI model after initial training in a training phase, and the second weight parameter is a weight parameter of an inference model; the inference model is an AI model of the target AI model in an inference phase, and the inference model is obtained based on the training model; A weight update module is used to determine the first weight parameter of the target AI model after retraining or fine-tuning during the training phase when the target AI model is retrained or fine-tuned; and, based on the updated first weight parameter and the target mapping relationship, update the second weight parameter of the inference model corresponding to the target AI model.

8. The device according to claim 7, characterized in that The inference model is used to be deployed on a single card, and the processing of the training model includes model compilation; the acquisition module is specifically used to: Recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; Recording the weight parameters of the training model after the model compilation process to obtain the second weight parameters; The target mapping relationship is determined based on the first weight parameter and the second weight parameter.

9. The device according to claim 7, characterized in that The inference model is used to be deployed on multiple cards, and the processing of the training model includes model segmentation and model compilation; the acquisition module is specifically used to: Recording the weight parameters of the training model after the initial training is completed to obtain the first weight parameters; Recording the weight parameters of the training model after the model segmentation process to obtain a third weight parameter; Determining a first mapping relationship based on the first weight parameter and the third weight parameter; Recording the weight parameters of the training model after the model compilation process to obtain the second weight parameters; Determining a second mapping relationship based on the third weight parameter and the third weight parameter; The target mapping relationship is determined based on the first mapping relationship and the second mapping relationship.

10. The device according to claim 9, characterized in that The model segmentation includes operator segmentation; the acquisition module is specifically used for: The first operator is divided into at least a second operator and a third operator; the second operator and the third operator are deployed on different cards; the parameters of the first operator include the first weight parameter, and the parameters of the second operator and the third operator include the third weight parameter; The first mapping relationship is determined based on a mapping relationship between parameters of the first operator and parameters of the second operator and the third operator.

11. The device according to any one of claims 8 to 10, characterized in that: The model compilation includes operator fusion; the acquisition module is specifically used for: After at least the fourth operator and the fifth operator are merged, a sixth operator is obtained; the parameters of the fourth operator and the fifth operator include the first weight parameter, and the parameters of the sixth operator include the second weight parameter; The target mapping relationship is determined based on the mapping relationship between the parameters of the fourth operator, the fifth operator and the parameter of the sixth operator.

12. The device according to any one of claims 7 to 11, characterized in that: The weight updating module is specifically used for: Adding training data to incrementally train the training model process to retrain the target AI model; Alternatively, the target AI model is fine-tuned by incrementally training the network layer of the training model; the network layer includes a fully connected layer.

13. A cloud management platform, characterized in that: include: At least one computing node, used for the inference phase of the AI ​​model; In the inference stage, based on the method described in any one of claims 1-6, the second weight parameter of the AI ​​model is updated.

14. A computing device comprising a memory and a processor, characterized in that: Instructions are stored in the memory, and when the instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

16. A computer program product, characterized in that The computer program product includes program instructions, and when the program instructions are executed by a computer, the computer executes the method according to any one of claims 1 to 6.

Images