Model training method and related equipment

Abstract

The embodiment of the invention discloses a model training method, and the method comprises the steps: converting a model weight in a preset attention module into an initialized model weight of an MLA module according to the correlation between a model parameter of the preset attention module in a first model and a model parameter in an MLA module in a second model, the method comprises the following steps: initializing an MLA module to transfer knowledge in a preset attention module to the initialized MLA module to provide a better knowledge basis for an initialized second model, and then performing small-scale training such as fine tuning on the MLA module to obtain a trained second model with better performance. And a complete whole-process training process does not need to be executed on the second model from zero, so that the training efficiency of the second model is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a model training method and related equipment. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, attention mechanisms such as multi-head attention (MHA) have been introduced in fields such as natural language processing (NLP) and computer vision (CV) to process sequential data, such as text and image sequences. This effectively utilizes the contextual information in the sequence data and achieves more accurate processing results. While MHA, as a powerful attention mechanism, performs well in processing sequential data and leveraging contextual information, it consumes significant computing and storage resources.

[0003] In order to solve the problem of MHA's high consumption of computing resources and storage resources, MHA variants such as multi-query attention (MQA), grouped-query attention (GQA), and multi-head latent attention (MLA) have been proposed to optimize the attention mechanism, in order to improve the reasoning efficiency of AI models based on attention mechanisms such as transformer models.

[0004] In actual applications, MQA and GQA can be obtained by increasing or decreasing the number of attention heads that provide key-value pairs for MHA. However, since MLA innovates the model structure based on MHA, it is currently only possible to obtain an AI model containing MLA in the reasoning scenario by executing a complete training process. The training process is relatively complex and consumes more resources. Summary of the Invention

[0005] The present invention provides a model training method that can solve the problem that the training process of the current AI model including MLA is relatively complicated. The present invention also provides corresponding devices, equipment, computer-readable storage media, and computer program products.

[0006] The first aspect of the present application provides a model training method, which includes: obtaining a first model and a second model, the first model includes a preset attention module, the preset attention module includes one or more of the following modules: a multi-head attention MHA module, a multi-head query attention MQA module, and a group query attention GQA module, the second model includes a multi-head potential attention MLA module, the first model is a trained AI model, the second model is an AI model to be trained, and the first model and the second model are used to process text and / or images, the model weight of the MLA module in the second model is obtained by transforming the model weight of the preset attention module; using the first model as a teacher model and the second model as a student model, the MLA module is trained to obtain a trained second model.

[0007] In the first aspect, based on the correlation between the model parameters of the preset attention module in the first model and the model parameters in the MLA module in the second model, the model weights in the preset attention module can be transformed into the initialized model weights of the MLA module, so as to migrate the knowledge in the preset attention module to the initialized MLA module, thereby providing a better knowledge basis for the initialized second model. Then, small-scale training such as fine-tuning the MLA module can be performed to obtain a trained second model with better performance, without the need to perform a complete full-process training process for the second model from scratch, thereby greatly improving the training efficiency of the second model.

[0008] In a possible implementation of the first aspect, the model weight of the first module excluding the preset attention module in the first model is the same as the model weight of the second module excluding the MLA module in the second model, the structure of the first module is the same as the structure of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model; the first model is used as a teacher model and the second model is used as a student model, and the MLA module is trained to obtain a trained second model, including: while fixing the model weight of the second module, the first model is used as a teacher model and the second model is used as a student model, and the MLA module is trained to obtain a trained second model.

[0009] In this possible implementation, similar or corresponding structures may exist in the model structures of the trained first model and the second model to be trained, thereby achieving relatively similar functions. For example, the structure of the first module is the same as that of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model. There is a correlation between the structure and function of the preset attention module and the MLA module. Therefore, the model weight of the second module in the second model can be set to be the same as the model weight of the first module in the trained first model, so that the knowledge of the first module in the trained first model can be reused in the second module of the second model, and then the MLA module can be fine-tuned or other small-scale training to obtain a trained second model with better performance, without having to perform a complete full-process training process on the second model from scratch, thereby improving the training efficiency of the second model.

[0010] In a possible implementation manner of the first aspect, the model weight of the second module is obtained by reusing the model weight of the first module to the second model.

[0011] In this possible implementation, the model weights of the second modules in the second model, excluding the MLA module, can be determined based on the model weights of the first modules in the first model, excluding the preset attention module. In this way, the knowledge of the first module in the trained first model can be reused in the second module of the initialized second model through the migration of model weights, thereby reducing the amount of knowledge that the second model needs to learn during the training phase, effectively utilizing the existing knowledge of the first model, and reducing the difficulty of training the second model.

[0012] For example, the preset attention module may include a first model parameter, the MLA module may include a second model parameter, the first model parameter corresponds to the second model parameter, and the dimension of the first model parameter is different from the dimension of the second model parameter. Then, the weight of the first model parameter may be dimensionally transformed to obtain the weight of the second model parameter. The correspondence between the first model parameter and the second model parameter may mean that the function and / or parameter structure of the first model parameter and the second model parameter are the same or similar.

[0013] In a possible implementation of the first aspect, the MLA module is trained based on a loss function, where the loss function includes a first loss term and / or a second loss term, the first loss term is used to evaluate the difference between the output of the first model and the output of the second model, and the second loss term is used to evaluate the difference between the output of the preset attention module and the output of the MLA module.

[0014] In this possible implementation, training with the first loss term can align the output performance of the trained second model with that of the first model; while training with the second loss term allows the MLA module of the second model to learn the feature representations in the preset attention module, thereby achieving alignment with the first model at the feature level. This shows that, based on the knowledge of the first model, training the MLA module in the second model using a loss function that includes the first loss term and / or the second loss term can efficiently achieve accurate alignment of the trained second model with the first model, ensuring the performance of the trained second model.

[0015] In a possible implementation manner of the first aspect, the loss function is obtained by weighted summing the first loss term and the second loss term.

[0016] In a possible implementation of the first aspect, the first model is used as a teacher model and the second model is used as a student model to train the MLA module to obtain a trained second model, including: using the first model as the teacher model and the second model as the student model to train the MLA module; after the MLA module training is completed, while fixing the model weight of the trained MLA module, training the second module and / or a third module in the second model other than the MLA module to obtain a trained second model, where the third module is different from the second module.

[0017] In this possible implementation, multiple stages of fine-tuning and other training can be performed, but the training intensity and time consumption of each stage are generally low, and the performance of the second model can be gradually improved through multiple stages of training. After completing multiple stages of fine-tuning and other training, a trained second model with better performance can be obtained.

[0018] A second aspect of the present application provides a model training device that has the function of implementing the method of the first aspect or any possible implementation of the first aspect. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions, such as an interface module and a training module.

[0019] A third aspect of the present application provides a computing device cluster, which includes at least one computing device, and the at least one computing device includes a processor and a memory. The memory of at least one computing device stores computer-executable instructions that can be run on the processor. When the computer-executable instructions are executed by the processor, the processor executes the method as described in the first aspect or any possible implementation of the first aspect.

[0020] The fourth aspect of the present application provides a computer-readable storage medium storing one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method as described in the first aspect or any possible implementation of the first aspect.

[0021] The fifth aspect of the present application provides a computer program product that stores one or more computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method as described in the first aspect or any possible implementation of the first aspect.

[0022] A sixth aspect of the present application provides a chip system, which includes a processor for supporting the processor in implementing the functions involved in the first aspect or any possible implementation of the first aspect. In one possible design, the chip system may also include a memory for storing necessary program instructions and data. The chip system may be composed of a chip or may include a chip and other discrete devices.

[0023] Among them, the technical effects brought about by the second to sixth aspects or any possible implementation methods thereof can refer to the technical effects brought about by the first aspect or the relevant possible implementation methods of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is an exemplary schematic diagram of a data center provided in an embodiment of the present application; Figure 2 This is an exemplary schematic diagram of the system framework provided by the embodiment of the present application; Figure 3 This is an exemplary schematic diagram of the model training method provided in the embodiment of the present application; Figure 4 is an exemplary schematic diagram of training the second model based on the loss function provided in an embodiment of the present application; Figure 5 This is an exemplary diagram of the training process provided in the embodiment of the present application; Figure 6 Schematic diagram of an embodiment of a model training device provided in an embodiment of the present application; Figure 7 is a structural diagram of a computing device provided in an embodiment of the present application; Figure 8 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application; Figure 9 This is a structural diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0026] Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0027] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate. This is merely a way of distinguishing objects with the same properties when describing them in the embodiments of this application. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent to such process, method, product, or apparatus.

[0028] Below, the technical terms involved in the embodiments of the present application are first explained.

[0029] 1. Model fine-tuning Model fine-tuning refers to the use of weights obtained by training an AI model on other datasets, and training and adjusting it on a new dataset with a small learning rate to obtain an AI model suitable for the new dataset.

[0030] 2. Model Distillation Model distillation is a model compression technique that transfers knowledge from a large, trained model (the teacher model) to a smaller model (the student model), improving the performance and generalization ability of the student model. Model distillation is particularly suitable for deploying models on resource-constrained devices, such as end devices or edge computing devices.

[0031] 3. Attention Conceptually, the attention mechanism can be understood as selectively filtering out a small amount of important information from a large amount of information and focusing on this important information, while ignoring the majority of unimportant information. This focusing process is reflected in the calculation of weight coefficients. The larger the weight, the more attention is paid to its corresponding value. In other words, the weight represents the importance of the information, while the value represents the corresponding information.

[0032] The attention mechanism provides an effective modeling approach for capturing global contextual information through the use of queries (Q), keys (K), and values ​​(V). Assuming the input is a query tensor (Q), which stores context as key-value pairs (key, value), the attention mechanism is essentially a mapping function from the query to a series of key-value pairs (key, value). This allows the global spatial information of the feature map to be captured through simple query and assignment.

[0033] The calculation of attention is mainly divided into three steps. The first step is to calculate the similarity between the query and each key to obtain the weight. Common similarity functions include dot product, concatenation, perceptron, etc. Then the second step is generally to use the softmax function (on the one hand, it can be normalized to obtain a probability distribution where the sum of all weight coefficients is 1. On the other hand, the characteristics of the softmax function can be used to highlight the weights of important elements) to normalize these weights. Finally, the weight is weighted and summed with the corresponding value to obtain the final eigenvalue y. The specific calculation formula can be as follows:

[0034] in, Can be a dimension for query and a key.

[0035] In the embodiment of the present application, the neural network used to implement the attention mechanism is called an attention network.

[0036] 4. Multi-head attention (MHA) mechanism Multi-head attention is an extended self-attention mechanism that uses multiple attention heads to capture different feature representations from the input in parallel, thereby capturing richer features and contextual information, and capturing dependencies in the input sequence. Multiple attention heads can each calculate attention, and then their results are concatenated together, allowing the AI ​​model to focus on different parts of the input sequence and learn a variety of different contextual information. Each attention head uses an independent weight matrix to perform linear transformations on the query, key, and value during calculation, thereby obtaining a different attention distribution. The outputs of all heads can then be merged, allowing the AI ​​model to focus on different parts of the input sequence and learn a variety of different contextual information. The output of the multi-head attention is then obtained through linear transformations and layer normalization.

[0037] In scenarios with attention mechanisms such as multi-head attention mechanisms, key-value caches are often involved.

[0038] 5. Key-value cache (KVCache) KVCache is a common optimization technique used during inference in large language models (LLMs). Its primary purpose is to reduce repeated computation in inference tasks such as autoregressive generation, thereby improving AI model inference efficiency. Specifically, the self-attention mechanism in transformer models is computationally intensive. For each input token, the transformer model must calculate its attention score relative to all other tokens in the sequence. In autoregressive generation tasks, each time a new token is generated, it must be added to the input sequence and the attention score for the entire sequence recalculated. This approach is computationally inefficient due to the significant amount of repeated computation. KVCache, on the other hand, caches the key and value tensors (key-value pairs) calculated for each token as it passes through the self-attention layer. When the next token is generated, only the query tensor for the new token needs to be calculated and compared with the cached key-value pairs, without having to recalculate the key-value pairs for the entire sequence. This reduces repeated computation and significantly improves inference speed.

[0039] 6. Multi-query attention (MQA) mechanism MQA is an attempt to reduce KVCache usage. Its core idea is to have all attention heads share the same key and value tensors, reducing the KVCache size to 1 / h (where h is the number of attention heads). MQA can significantly reduce memory consumption, but may result in reduced performance on some tasks.

[0040] 7. Grouped-query attention (GQA) mechanism GQA divides all attention heads into g groups (g must be divisible by h, where h is the number of attention heads). Each group shares the same key-value pair (key and value). This reduces the size of the KV Cache while preserving a certain degree of diversity. GQA provides a natural transition from MHA to MQA. When g = h, it is MHA; when g = 1, it is MQA; and when 1 < g < h, it is GQA. GQA strikes a balance between KV Cache compression rate and performance.

[0041] 8. Multi-head latent attention (MLA) mechanism MLA uses low-rank projection to replace the splitting and repetition operations used in GQA. MLA enhances model capabilities through different projection matrices and reduces KVCache usage through identity transformation techniques. The core concept of MLA is to enhance model capabilities through different projection matrices and reduce KVCache storage and computational overhead through identity transformation techniques during the inference phase. This reduces memory consumption while maintaining high feature expression capabilities, ensuring good inference performance.

[0042] With the rapid development of AI technology, attention mechanisms such as the MHA mechanism have been introduced in fields such as natural language processing and computer vision to process sequence data such as text and image sequences, so as to effectively utilize the contextual information of sequence data and obtain more accurate processing results.

[0043] As a powerful attention mechanism, MHA performs well in processing sequence data and utilizing contextual information, but it also has some shortcomings: 1) High computational cost: MHA increases the number of parameters and computational complexity of the AI ​​model. Especially when the number of attention heads is large, it places greater demands on computing resources, which may become a burden in resource-constrained environments.

[0044] 2) High memory bandwidth requirements: In model scenarios such as autoregressive models, each decoding step involved in MHA requires loading the decoder weights and the keys and values ​​of all attention heads. This process is not only computationally intensive but also places high demands on memory bandwidth. As the model scale increases, this transmission overhead also increases, making it increasingly difficult to scale the model.

[0045] 3) Large memory overhead: In model scenarios such as autoregressive models, MHA requires loading a large number of keys and values ​​at each decoding step, resulting in significant memory overhead, especially in the inference phase of the decoder.

[0046] In order to solve the problem of MHA's high consumption of computing resources, storage resources and other resources, MHA variants such as MQA, GQA and MLA have been proposed to optimize the attention mechanism to improve the reasoning efficiency of AI models based on attention mechanisms such as transformer models.

[0047] In other words, MQA, GQA, and MLA are all optimization solutions designed to address the efficiency issues of AI models like transformer models when processing long sequences. MQA reduces computational complexity by sharing key-value pairs, GQA provides greater flexibility based on MQA, and MLA optimizes the KV Cache during inference through low-rank projection and identity transformation, achieving faster inference speed and lower cost.

[0048] In actual applications, MQA and GQA can be obtained by increasing or decreasing the number of attention heads that provide key-value pairs for MHA. However, since MLA innovates the model structure based on MHA, the AI ​​model containing MLA in the reasoning scenario can only be obtained by executing a complete training process.

[0049] The complete training process for an AI model that includes MLA can include the following steps: 1) Data preparation: This includes steps such as data cleaning, feature engineering, and data partitioning. Data cleaning is used to remove outliers, missing values, and duplicate values ​​from the dataset; feature engineering is used to convert training data into features with better representation capabilities as input; and data partitioning is to divide the dataset into training, validation, and test sets.

[0050] 2) AI model selection and construction: Specifically, an AI model including MLA can be constructed, and a training environment can be built using a training framework such as TensorFlow, PyTorch, etc. The specific functions and types of the AI ​​model are not limited here.

[0051] 3) Model training: Determine the hyperparameters of the AI ​​model, such as the learning rate and number of iterations, and use methods such as gradient descent to minimize the loss function. Complete the training after meeting the specified convergence conditions and / or reaching the specified number of iterations, and obtain the trained AI model.

[0052] 4) Evaluation and Optimization: After the AI ​​model training is completed, the performance of the trained AI model is evaluated using evaluation indicators such as accuracy, precision, recall rate, and F1 score. The trained AI model is then optimized based on the evaluation results, such as adjusting hyperparameters, changing the model structure, and fine-tuning the model.

[0053] After optimization, the model can be deployed to obtain an AI model that can be applied to reasoning scenarios.

[0054] It can be seen that executing a complete training process for an AI model using MLA takes a long time, the training process is complex, and requires the collection of a large amount of data, the use of a large amount of processing resources, and multiple iterations to achieve. Moreover, if an AI model containing MHA already exists in the same current application scenario, if the AI ​​model containing MLA is retrained, the experience of the existing AI model containing MHA cannot be reused. In addition, the AI ​​model containing MLA trained through the new complete training process is difficult to align with the existing AI model containing MHA in terms of accuracy, and often cannot achieve the same performance as the existing AI model containing MHA. Therefore, when migrating from the existing AI model containing MHA to the AI ​​model containing MLA in business, it will lead to differences in the service level of the business, making it difficult to achieve smooth migration in business.

[0055] Based on this, how to quickly and cheaply obtain the weights of AI models containing MLA has become an urgent problem that needs to be solved.

[0056] In an embodiment of the present application, the weight of the AI ​​model including MLA can be efficiently obtained based on an existing AI model including modules such as MLA, so as to obtain an AI model including MLA that can perform reasoning tasks, without having to completely train the AI ​​model including MLA from scratch.

[0057] The method of the embodiment of the present application can be applied to a computing device cluster, which may include one or more computing devices.

[0058] The type of any computing device is not limited herein. For example, any computing device may be a terminal device, or may be a server, container, or virtual machine. Different computing devices may be of the same type or may differ in type.

[0059] In one example, the computing device cluster can be used to implement a cloud management platform. In other words, the embodiments of the present application can be applied to a cloud management platform.

[0060] The cloud management platform is used to manage the infrastructure that provides cloud services. It can provide computing, network, and storage capabilities based on hardware and software resources. For example, the cloud management platform and infrastructure can be located in one or more data centers to provide cloud resources through one or more data centers.

[0061] The following combination Figure 1 An architectural diagram shown is used to provide an exemplary introduction to a data center.

[0062] Figure 1 In the cloud management platform, the cloud management platform connects to one or more servers (such as Figure 1 The server interacts with servers 1 and 2 in the example. The server consists of a hardware layer and a software layer. The hardware layer includes the server's hardware configuration, including PCI devices such as network cards, graphics processing units (GPUs), and offload cards that can be plugged into the server's peripheral component interconnect (PCI) or peripheral component interconnect express (PCIe) slots. The software layer includes the operating system installed and running on the server (referred to as the host operating system, as opposed to the virtual machine's operating system). The host operating system includes a virtual machine manager (also known as a hypervisor). The hypervisor implements computing, network, and storage virtualization for the virtual machines and manages them. A virtual machine (VM) is a complete computer system emulated by software, with complete hardware system functionality, running in a completely isolated environment. Figure 1 In the system architecture shown, the infrastructure consists of multiple servers, each of which can run virtual machines (VMs). These VMs can have the same or different specifications. VMs are also referred to as elastic compute services (ECS) or elastic instances, and different cloud service providers may use different names for them.

[0063] In one example embodiment of the present application, the cloud management platform may be a public cloud platform. In this case, cloud service providers such as individuals or software developers with cloud resource development capabilities can provide cloud services to users. Users access cloud services through the Internet but do not own cloud computing resources. In other embodiments of the present application, the cloud management platform may be a private cloud platform or a hybrid cloud platform, which is not a limitation of the present application.

[0064] Specifically, in Figure 1 In the example shown, the cloud management platform can provide an access interface (such as an interface or application programming interface (API)). Users of the cloud management platform and cloud service providers can operate the client remote access access interface to register a cloud account and password on the cloud management platform, and after the cloud management platform successfully authenticates the cloud account and password, log in to the cloud management platform to create, manage, log in to and operate virtual machines in the cloud data center.

[0065] For example, reference Figure 2 In the example shown, the cloud management platform can provide users with a training framework that can be used to train AI models containing MLA.

[0066] Before training, users can obtain a trained AI model including a preset attention module as the first model, and obtain an uninitialized AI model including an MLA module as the initial second model. Then, the model weights of the second model can be initialized based on the model weights of the first model. Through this training framework, the initialized second model can be trained (specifically, the MLA module in the second model can be trained) to obtain the trained second model.

[0067] It should be noted that in other embodiments of the present application, the cloud management platform may be a private cloud platform or a hybrid cloud platform. Figure 2 The services mentioned are only examples of services and processing methods provided by the cloud management platform, and are not limiting.

[0068] For example, in other examples, the second model can be initialized through an initialization service, and then the initialized second model can be input into the training framework for training. Alternatively, the initialization service can be used as a plug-in or tool in the training framework. As can be seen, the specific form and deployment method of the service used to implement the model training method in the embodiments of the present application can be various, and this application does not limit this.

[0069] Based on the above computing device cluster, refer to Figure 1 as well as Figure 2 The system architecture shown in Figure 3As shown, the model training method of the embodiment of the present application may include steps 301-302.

[0070] Step 301: Obtain a first model and a second model.

[0071] Among them, the first model includes a preset attention module, which includes one or more of the following modules: MHA module, MQA module, GQA module, the second model includes an MLA module, the first model is a trained AI model, the second model is an AI model to be trained, and the first model and the second model are used to process text and / or images.

[0072] In an embodiment of the present application, the first model may be a trained AI model, which may include but is not limited to a preset attention module. For example, it may also include a fully connected layer, a feedforward neural network (FNN), a normalization and residual connection layer, etc.

[0073] Among them, the preset attention module may include but is not limited to one or more of the MHA module, the MQA module, and the GQA module. In some examples, the preset attention module may also include one or more layers such as layer normalization and residual connection, which are not limited in the embodiments of the present application. Among them, the specific introduction of MHA, MQA and GQA can refer to the relevant content of the explanation part of technical terms, which will not be repeated here. In addition, in the embodiments of the present application, the specific number and structure of the MHA module, MQA module or GQA module in the preset attention module may have multiple situations. For example, the number of attention heads contained in the MHA module, MQA module or GQA module in the preset attention module is not limited here. Moreover, the MHA module, MQA module or GQA module in the preset attention module may be multi-layer stacked. For example, the preset attention module may include L layers of MHA modules, L is a positive integer, and when L is greater than 1, the preset attention module may include multi-layer stacked MHA modules.

[0074] In some examples, considering that the performance achieved by the current MHA module is often better than that of the MQA module and the GQA module, the preset attention module may include an MHA module to provide better existing knowledge for the subsequent training of the second model and provide a better knowledge basis for the training of the second model.

[0075] The first model can be a computing device cluster input by a user to execute the embodiment of the present application, or it can be pre-stored in the computing device cluster. The specific method of obtaining the first model is not limited here.

[0076] The second model is the model to be trained in the embodiment of the present application. The second model may include but is not limited to the MLA module. For example, it may also include a fully connected layer, a feedforward neural network, a normalization and residual connection layer, etc.

[0077] The MLA module is used to implement MLA calculations. For a detailed introduction to MLA, please refer to the relevant content in the explanation of technical terms section.

[0078] In the embodiments of the present application, the second model can be considered as an initialized model for subsequent training. In addition, in some embodiments, the second model can be initialized based on the model weights of the first model. Among them, the model weights can also be called parameter weights, which are the values ​​learned by the AI ​​model during the training process. The model weights determine how the AI ​​model maps input data to output results.

[0079] In order to enable efficient training of the second model based on the model weights of the first model, similar or corresponding structures may exist in the model structures of the first model and the second model to achieve relatively similar functions. For example, the first model may include a word embedding layer, a preset attention module, and an output layer after the preset attention module, wherein the output layer may also be called a prediction head, a language model output layer, or an LM head, and may include one or more of a multilayer perceptron (MLP), a classifier (e.g., a fully connected layer), etc.; and similarly, the second model may include a word embedding layer, an MLA module, and an output layer after the MLA module. It can be seen that there are corresponding structures between the first model and the second model, so that the second model (for ease of description, in subsequent embodiments, the second model to be initialized will be referred to as the initial second model) that has not undergone model weight initialization based on the model weights of the first model to obtain the second model. The method of obtaining the initial second model can be the same as that of the first model, or it can be different from that of the first model. For example, in some examples, the first model may be a trained model pre-stored in the computing device cluster, and the initial second model may be transmitted by the user from the client to the computing device cluster, or may be constructed by the user in the computing device cluster.

[0080] When initializing the initial second model, different initialization strategies may be used for the MLA module and the parts other than the MLA module in the initial second model. For ease of description, the second model may be considered as a model that has been parameter-initialized before training.

[0081] Below, specific methods of initializing the initial second model based on the model weights of the first model are exemplarily introduced based on the MLA module in the initial second model and the parts other than the MLA module.

[0082] 1. Initialize the MLA module in the initial second model.

[0083] In some embodiments, the model weight of the MLA module in the second model is obtained according to the model weight of the preset attention module.

[0084] According to the correlation between the model weights of the preset attention module and the model weights in the MLA module, one or more model weights in the preset attention module can be transformed into one or more initialized model weights of the MLA module, so as to migrate the knowledge in the preset attention module to the initialized MLA module, provide a better knowledge basis for the initialized second model, and facilitate subsequent efficient model fine-tuning and other model training of the initialized second model.

[0085] Specifically, in some embodiments, the model weights of the MLA module in the second model are obtained by transforming the model weights of the preset attention module.

[0086] In an embodiment of the present application, the model weight of the MLA module in the second model can be obtained by transforming the model weight of the preset attention module based on the dimension of the model weight of the preset attention module and the dimension of the model weight of the MLA module.

[0087] In an embodiment of the present application, the model parameters of the MLA module in the second model can be initialized by performing a dimensional transformation on the model weights of the preset attention module in the first model.

[0088] For example, the preset attention module may include a first model parameter, the MLA module may include a second model parameter, the first model parameter corresponds to the second model parameter, and the dimension of the first model parameter is different from the dimension of the second model parameter. Then, the weight of the first model parameter can be dimensionalized (for example, tensor dimension increase or dimension reduction) to obtain the weight of the second model parameter. The correspondence between the first model parameter and the second model parameter may mean that the function and / or parameter structure of the first model parameter and the second model parameter are the same or similar.

[0089] For example, in the MHA module in the preset attention module, each attention head includes a weight matrix corresponding to the query tensor, key tensor, and value tensor when generating the query tensor, key tensor, and value tensor. Specifically, a certain attention head in the MHA module Output It can be described specifically by the following formula:

[0090]

[0091]

[0092] in, The attention head in the MHA module is The query tensor, key tensor, and value tensor of The attention head The weight matrices corresponding to the query tensor, key tensor, and value tensor are the model weights contained in the preset attention module.

[0093] And a certain attention head in the MLA module Output It can be described specifically by the following formula:

[0094] in, They are the attention heads in the MLA module The query tensor, key tensor, and value tensor of The attention head The weight matrices corresponding to the query tensor, key tensor, and value tensor are the model weights contained in the MLA module.

[0095] In this example, the attention head Position and attention head in the second model The position in the first model corresponds to, for example, the attention head The number of layers and attention heads in the MLA module The number of layers in the preset attention module is the same, and the attention head Position and attention head among multiple heads in the layer The positions of the multiple heads in the same layer are the same. Then, we can use the attention heads in the preset attention module The model weights of the attention head are obtained The model weights in are used as the model weights after model initialization.

[0096] For example, according to and / or get ,according to get ,according to get , and according to get Among them, due to and Respectively The dimensions of are different. Therefore, in order to obtain the initialized model weights in the MLA module, the model weights in the preset attention module need to be transformed. Specifically, the tensor dimension can be increased and / or decreased according to the specific conditions of different dimensions. For example, and / or Perform dimensionality reduction to obtain ,right Perform dimensionality reduction to obtain ,right Perform dimensionality reduction to obtain . And for ,like and The dimensions are the same, you can Directly ,like The dimension is greater than The dimension can be Perform dimensionality reduction to obtain ,like The dimension is less than The dimension can be Upgrade to obtain .

[0097] When performing transformations such as dimensionality increase or decrease, there may be multiple specific methods used in this example, and the transformation methods used may also be different for different model weights, which is not limited in the embodiments of the present application.

[0098] It is understood that the preset attention module model weights, the MLA module model weights, and the initialization parameters for the MLA module model weights obtained by transforming the preset attention module model weights in the above examples are merely examples and are not limiting. In actual application scenarios, the preset attention module model weights and the MLA module model weights may also include other model weights.

[0099] For example, the model weight of the preset attention module may include a rotational position encoding (RoPE), and the MLA module also includes a rotational position encoding. If the dimension of the weight matrix of the rotational position encoding in a certain attention head in the MLA module is different from the weight matrix of the corresponding rotational position encoding in the preset attention module, the initialization parameters of the weight matrix of the rotational position encoding in a certain attention head in the MLA module can be obtained by transforming the weight matrix of the corresponding rotational position encoding in the preset attention module; if the dimension of the weight matrix of the rotational position encoding in a certain attention head in the MLA module is different from the weight matrix of the corresponding rotational position encoding in the preset attention module, the weight matrix of the rotational position encoding in a certain attention head in the MLA module can be obtained by transforming the weight matrix of the corresponding rotational position encoding in the preset attention module.

[0100] There are various specific methods for this dimensional transformation. For example, based on the difference between the dimensions of the model weights of the preset attention module and the dimensions of the model weights of the MLA module, the elements in the model weights of the preset attention module can be merged and expanded to obtain the elements in the model weights of the MLA module. Alternatively, existing or subsequently developed tensor dimensionality reduction and / or tensor dimensionality increase methods can be used to transform the model weights of the preset attention module to obtain the model weights of the MLA module in the second model.

[0101] For example, tensor dimensionality reduction can be performed based on a linear dimensionality reduction method, a nonlinear dimensionality reduction method, a tensor decomposition method, or a neural network method. Exemplarily, the linear dimensionality reduction method may include one or more of principal component analysis (PCA), linear discriminant analysis (LDA), or a random projection method using a random matrix, while the nonlinear dimensionality reduction method may include one or more of t-distributed stochastic neighbor embedding (t-SNE) and uniform manifold approximation and projection (UMAP), the tensor decomposition method may include one or more of canonical polyadic decomposition (CPD), Tucker decomposition, or tensor singular value decomposition (SVD), and the neural network method may include one or more of autoencoders, pooling layers, and the like.

[0102] The tensor dimensionality increase method may include but is not limited to one or more linear dimensionality increase methods such as the inverse transformation of PCA, pseudo-inverse matrix recovery, decoders, tensor reconstruction methods, and interpolation methods such as nearest neighbor interpolation, bilinear interpolation, and trilinear interpolation.

[0103] Alternatively, in some other embodiments, the ML module in the initial second model may be randomly initialized to obtain the model weight of the MLA module in the second model.

[0104] 2. Initialize the MLA module and parts other than the MLA module in the initial second model.

[0105] Specifically, in some embodiments, the model weights of the second modules excluding the MLA module in the second model are obtained by reusing the model weights of the first modules excluding the preset attention module in the first model to the second model. The structure of the first module is the same as that of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model.

[0106] In this way, the model weight of the first module excluding the preset attention module in the first model is the same as the model weight of the second module excluding the MLA module in the second model.

[0107] In other words, the model weights of the first module excluding the preset attention module in the first model can be applied to the initial second model as the model weights of the second module excluding the MLA module.

[0108] If there is a structural part in the first model other than the preset attention module that is identical to the structural part in the second model other than the MLA module, the identical structural part in the first model can be used as the first module, and the identical structural part in the second model can be used as the second module, and the model weight of the first module can be used as the model weight of the second module to reuse the knowledge learned in the first module in the second module through the migration of the model weight. The correspondence between the position of the first module in the first model and the position of the second module in the second model can indicate that the position of the first module in the first model is the same as or similar to the position of the second module in the second model.

[0109] For example, in the first model, after the preset attention module, a first FNN, a first normalization layer, and a first output layer may be included. In the second model, after the MLA module, a second FNN, a residual connection layer, a second normalization layer, and a second output layer may be included. The structure of the first FNN is the same as that of the second FNN and their positions in the AI ​​model are close, and the functions they implement are the same. Furthermore, the structure of the first normalization layer is the same as that of the second normalization layer and their positions in the AI ​​model are close, and the functions they implement are the same, but the structure of the first output layer is different from that of the second output layer. In this case, the first module may include the first FNN and the first normalization layer, and the second module may include the second FNN and the second normalization layer. Furthermore, the model weights of the first FNN and the first normalization layer in the first model may be used as the model weights of the second FNN and the second normalization layer in the second model, respectively.

[0110] In the second model, the model weights of modules other than the MLA module and the second module can be determined in a variety of ways. In other words, the model weights of modules other than the MLA module and the second module in the initial second model can be initialized in a variety of ways. For example, the model weights of the modules other than the MLA module and the second module can be randomly initialized, or the model weights initialized based on other experience can be used as the model weights of the modules in the second model.

[0111] It can be seen that in the embodiment of the present application, the model weights of the initial second model can be initialized according to the model weights in the first model to obtain the second model, so that in the initialization stage, the knowledge in the trained first model can be reused in the second model obtained after initialization through the migration of model weights, without the need to perform a complete full-process training process for the second model from scratch, thereby greatly improving the subsequent training efficiency of the second model.

[0112] In step 302 , the first model is used as a teacher model and the second model is used as a student model to train the MLA module to obtain a trained second model.

[0113] In an embodiment of the present application, the second model obtained after initialization can be trained. In some examples, specifically, the MLA module can be trained according to the loss function and the first model while fixing the model weights of the second module. In some examples, after the model weights of the MLA module are initialized according to the model weights of the preset attention module and the initialized MLA module is obtained, the initialized MLA module can be trained according to the loss function and the first model while fixing the model weights of the second module to obtain the trained second model.

[0114] In the process of training the MLA module of the second model, the multiple training data used can be in various situations, which are not limited here. For example, in step 302, the multiple training data used to train the MLA module can be the same as the historical training data used when training the first model, or can include only part of the historical training data, or can also include other data different from the historical training data. In some scenarios, considering that it is not necessary to train all model weights in the second model, but only part of the model weights in the second model, it can be considered that the training process of the second model in step 302 is a fine-tuning of the second model, and the amount of training data required will also be less than the historical training data required for training the first model. And because the knowledge and experience of the existing first model are reused, it is possible to achieve efficient learning of the weights of the MLA module in the second model with less training data, and the training efficiency is higher.

[0115] Furthermore, any of the multiple training data may be labeled or unlabeled unsupervised training data. Any of the training data may include text and / or images. Furthermore, in scenarios such as multimodal processing, the multiple training data may also include other types of data besides text and images.

[0116] In an embodiment of the present application, the MLA module may be trained using a training method such as model distillation.

[0117] Specifically, in the embodiment of the present application, the existing first model can be used as the teacher model, and the second model can be used as the student model for multiple iterative training. In each iterative training, the training data of this iterative training can be input into the teacher model and the student model respectively, and forward reasoning can be performed on the teacher model and the student model to obtain the output data of each of the teacher model and the student model in this iterative training.

[0118] Then, the loss value of the student model in this iterative training can be calculated based on the loss function and the output data of the teacher model and the student model in this iterative training, so as to backpropagate the student model according to the loss value to update the weight of the MLA module in the student model.

[0119] In some embodiments, the loss function may include a first loss term and / or a second loss term, wherein the first loss term is used to evaluate the difference between the output of the first model and the output of the second model, and the second loss term is used to evaluate the difference between the output of the preset attention module and the output of the MLA module.

[0120] In this way, training with the first loss term can align the output performance of the trained second model with that of the first model; while training with the second loss term allows the MLA module of the second model to learn the feature representations in the preset attention module, thereby achieving alignment with the first model at the feature level. This shows that, based on the knowledge of the first model, training the MLA module in the second model using a loss function that includes the first loss term and / or the second loss term can efficiently achieve accurate alignment of the trained second model with the first model, ensuring the performance of the trained second model.

[0121] The following is a specific example to illustrate a specific implementation of the embodiment of the present application.

[0122] In this example, Figure 4 Taking the structure shown in FIG2 as an example, the second model is introduced as a student model, and the knowledge in the first model as the teacher model is learned through knowledge distillation to fine-tune the second model.

[0123] exist Figure 4 In the example, the first model includes a word embedding layer, a preset attention module, and an output layer. In this example, the preset attention module can be specifically an MHA module. The MHA module can include a stacked structure of L layers, and each layer includes W attention heads. Since the MHA module is located in the middle layer of the first model, it can also be considered as a hidden layer of the first model. The output layer can also be called a prediction head, a language model output layer, or an LM head, etc., and is used to convert the hidden state output by the hidden layer such as the MHA module into the output data of the first model.

[0124] The structural framework of the second model is similar to that of the first model. Figure 4 In

[15] , the second model may include a word embedding layer, an MLA module, and an output layer. The word embedding layer and output layer in the first model may be consistent with or different from those in the second model. The MLA module in the second model also includes a stacked structure of L layers, and each layer includes W attention heads. Furthermore, the MLA module can also be considered as a hidden layer in the second model.

[0125] for Figure 4 The model structure shown in the figure shows the loss function used when training the MLA module in the second model. The specific form is as follows:

[0126] in, is the first loss term, is the loss weight corresponding to the first loss item; is the second loss term, is the loss weight corresponding to the second loss term. as well as It can be set by the user or developer based on experience to describe the importance of the first loss term and the second loss term respectively. It can be seen that by adjusting the loss weight corresponding to the first loss term and the loss weight corresponding to the second loss term, the influence of the first loss term and the second loss term on the model weight of the MLA module during training can be adjusted.

[0127] Specifically, in the first loss item, is the output data of the second model, is the output data of the first model. Through the first loss term, the second model can reuse the existing knowledge and experience of the first model to train the second model efficiently.

[0128] In the second loss item, including output data of the MLA module in the second model (for example, when the MLA module includes an L-layer stacked structure, it may be the output of the last layer in the L-layer stacked structure), This includes the output data of the MHA module in the first model (for example, when the MHA module includes an L-layer stack, this may be the output of the last layer in the L-layer stack). This second loss term allows the MLA module to be trained to efficiently learn the knowledge of the trained attention module from the existing MHA module, thereby improving the performance of the second model.

[0129] After training the MLA module, a trained second model can be obtained. Alternatively, further fine-tuning can be performed on the second model to further improve performance, and then the trained second model can be obtained and deployed.

[0130] In some embodiments, step 302 includes: The MLA module is trained by using the first model as the teacher model and the second model as the student model; After the MLA module training is completed, while fixing the model weight of the trained MLA module, the second module and / or the third module in the second model other than the MLA module is trained to obtain a trained second model, where the third module is different from the second module.

[0131] In embodiments of the present application, after the MLA module is trained, while the model weights of the trained MLA module are fixed, fine-tuning can be performed on some or all of the structure in the second module, and / or training can be performed on some or all of the structure in a third module in the second model other than the MLA module. The third module may be a module with no corresponding identical structure in the first model (e.g., a module with a similar but not identical structure in the first model, or a module completely different from the first model). Therefore, it is difficult to accurately determine the model weights of the third module during the model initialization phase based on the model weights in the first model, requiring further fine-tuning. For example, the second module may include the output layer of the second model and have the same structure as the output layer of the first model. For example, the second module may include a fully connected layer or an MLP. In this case, while the model weights of the trained MLA module are fixed, fine-tuning can be performed on the second module to further improve accuracy. Alternatively, the third module may include the output layer of the second model but have a different structure from the output layer of the first model. In this case, fine-tuning can also be performed on the third module while the model weights of the trained MLA module are fixed. As can be seen, in the embodiments of the present application, multiple stages of fine-tuning and other training can be performed, but the training intensity and time consumption of each stage are generally small, and the performance of the second model can be gradually improved through multiple stages of training. After completing multiple stages of fine-tuning and other training, a trained second model with better performance can be obtained.

[0132] In this way, through training, the trained second model can be aligned with the accuracy of the first model. When migrating from the existing first model containing MHA to the second model containing MLA, a relatively uniform service level of the business can be guaranteed, thereby achieving smooth business migration.

[0133] It can be seen that in some examples, reference Figure 5 In the example shown, the training process of the second model in the embodiment of the present application may include the following steps: Design a second model (for example, design the structure of the second model), build a training environment, initialize the model weights of the second model (specifically, initializing the model weights of the MLA module according to the model weights of the preset attention module, and determining and fixing the model weights of the parts of the second model other than the MLA module according to the model weights of the parts of the first model other than the preset attention module), and train the MLA module using the first model as a teacher model and the second model as a student model to obtain the trained second model, and the trained second model can be deployed.

[0134] Since this solution fully utilizes the existing knowledge in the first model during the initialization stage of the second model and the training process of the second model, compared with traditional solutions, it greatly reduces the difficulty of model training, reduces the consumption of processing resources such as computing resources and storage resources, and has higher training efficiency.

[0135] The above describes the model training method provided in the embodiment of the present application from multiple aspects. The following describes the model training device provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0136] like Figure 6 As shown, an embodiment of the present application provides a model training device 60, which includes: Interface module 601 is used to obtain a first model and a second model, where the first model includes a preset attention module, which includes one or more of the following modules: a multi-head attention (MHA) module, a multi-head query attention (MQA) module, and a grouped query attention (GQA) module; the second model includes a multi-head latent attention (MLA) module; the first model is a trained AI model, and the second model is an AI model to be trained, and the first model and the second model are used to process text and / or images, and the model weight of the MLA module in the second model is obtained by transforming the model weight of the preset attention module; The training module 602 is used to: The MLA module is trained with the first model as a teacher model and the second model as a student model to obtain a trained second model.

[0137] Optionally, the model weight of the first module excluding the preset attention module in the first model is the same as the model weight of the second module excluding the MLA module in the second model, the structure of the first module is the same as the structure of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model; The training module is used to: when the model weight of the second module is fixed, use the first model as the teacher model and the second model as the student model to train the MLA module to obtain a trained second model.

[0138] Optionally, the model weight of the second module is obtained by reusing the model weight of the first module to the second model.

[0139] Optionally, the MLA module is trained based on a loss function, which includes a first loss term and / or a second loss term, the first loss term is used to evaluate the difference between the output of the first model and the output of the second model, and the second loss term is used to evaluate the difference between the output of the preset attention module and the output of the MLA module.

[0140] Optionally, the loss function is obtained by weighted summation of the first loss term and the second loss term.

[0141] Optionally, the training module 602 is used to: The MLA module is trained by using the first model as the teacher model and the second model as the student model; After the MLA module training is completed, while fixing the model weight of the trained MLA module, the second module and / or the third module in the second model other than the MLA module is trained to obtain a trained second model, where the third module is different from the second module.

[0142] The training module and the interface module can be implemented in software or hardware. For example, the implementation of the training module will be described below using the training module as an example. Similarly, the implementation of modules B and C can refer to the implementation of the training module.

[0143] As an example of a software functional unit, a training module may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, a training module may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0144] Similarly, the multiple hosts / virtual machines / containers used to run the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0145] As an example of a hardware functional unit, a training module may include at least one computing device, such as a server. Alternatively, the training module may be implemented using a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU), a neural network processing unit (NPU), a system on chip (SoC), an offload card, an accelerator card, or any combination thereof.

[0146] The multiple computing devices included in the training module can be distributed in the same region or in different regions. The multiple computing devices included in the training module can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the training module can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, DPUs, NPUs, SoCs, offload cards, accelerator cards, and other computing devices.

[0147] It should be noted that, in other embodiments, the training module can be used to execute any step in the model training method, and the interface module can be used to execute any step in the model training method. The steps that the training module and the interface module are responsible for implementing can be specified as needed. The full functions of the model training device can be realized by respectively implementing different steps in the model training method through the training module and the interface module.

[0148] The present application embodiment also provides a computing device 70. Figure 7 As shown, computing device 70 includes a bus 72, a processor 74, a memory 76, and a communication interface 78. Processor 74, memory 76, and communication interface 78 communicate with each other via bus 72. Computing device 70 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 70.

[0149] Bus 72 can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a unified bus (UBus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. Among them, the unified bus is also called the Lingqu bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus 72 may include a path for transmitting information between various components of the computing device 70 (eg, memory 76, processor 74, communication interface 78).

[0150] The processor 74 may include any one or more computing devices such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP), an ASIC, an FPGA, a CPLD, an NPU, a SoC, an offload card, an accelerator card, etc.

[0151] Memory 76 may include volatile memory, such as random access memory (RAM). Memory 76 may also include non-volatile memory, such as one or more of read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD). Furthermore, memory 76 may be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0152] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 76, or two or more types of storage media can be configured to realize the function of memory 76. This application does not limit this.

[0153] The memory 76 stores executable program code, and the processor 74 executes the executable program code to respectively implement the functions of the aforementioned interface module and training module, thereby realizing the model training method applied to the computing device cluster in the above-mentioned embodiment, that is, the memory 76 stores instructions for executing the model training method applied to the computing device cluster in the above-mentioned embodiment.

[0154] The communication interface 78 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 70 and other devices or a communication network.

[0155] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0156] like Figure 8 As shown, the computing device cluster includes at least one computing device 70. The memory 76 in one or more computing devices 70 in the computing device cluster may store the same instructions for executing the model training method.

[0157] In some possible implementations, the memory 76 of one or more computing devices 70 in the computing device cluster may also store some instructions for executing the model training method. In other words, the combination of one or more computing devices 70 can jointly execute the instructions for executing the model training method.

[0158] It should be noted that the memory 76 in different computing devices 70 in the computing device cluster can store different instructions, each for executing a portion of the functions of the model training method. In other words, the instructions stored in the memory 76 in different computing devices 70 can implement the functions of one or more modules in the interface module and the training module.

[0159] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 9 A possible implementation is shown. Figure 9As shown, computing device 70A and computing device 70B are connected via a network. Specifically, they are connected to the network via the communication interfaces in each computing device. In this type of possible implementation, the memory 76 in computing device 70A may store instructions for executing the functions of the interface module. Simultaneously, the memory 76 in computing device 70B may store instructions for executing the functions of the training module. Alternatively, the memory 76 in computing device 70A may store instructions for executing part of the functions of the training module. Simultaneously, the memory 76 in computing device 70B may store instructions for executing another part of the functions of the training module.

[0160] It should be understood that Figure 9 The functionality of the computing device 70A shown in FIG. 7 may also be implemented by multiple computing devices 70. Similarly, the functionality of the computing device 70B may also be implemented by multiple computing devices 70.

[0161] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similarly referred to as Figure 8 and Figure 9 The connection mode of the computing device cluster is different in that the memory 76 of one or more computing devices 70 in the computing device cluster may store the same instructions for executing the model training method.

[0162] In some possible implementations, the memory 76 of one or more computing devices 70 in the computing device cluster may also store some instructions for executing the model training method. In other words, the combination of one or more computing devices 70 can jointly execute the instructions for executing the model training method.

[0163] It should be noted that the memory 76 in different computing devices 70 in the computing device cluster can store different instructions for executing part of the functions of the model training method. In other words, the instructions stored in the memory 76 in different computing devices 70 can implement the functions of one or more modules in the interface module and the training module.

[0164] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the model training method.

[0165] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the model training method.

[0166] The present application also provides a chip system, which includes a processor configured to implement the steps performed by the computing device cluster. In one possible design, the chip system may also include a memory configured to store necessary program instructions and data. The chip system may be composed solely of a chip or may include a chip and other discrete components.

[0167] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0169] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A model training method, characterized in that: The method comprises: Obtain a first model and a second model, wherein the first model includes a preset attention module, and the preset attention module includes one or more of the following modules: a multi-head attention MHA module, a multi-head query attention MQA module, and a group query attention GQA module, and the second model includes a multi-head potential attention MLA module, the first model is a trained AI model, the second model is an AI model to be trained, and the first model and the second model are used to process text and / or images, and the model weight of the MLA module in the second model is obtained by transforming the model weight of the preset attention module; The first model is used as a teacher model and the second model is used as a student model, and the MLA module is trained to obtain the trained second model.

2. The method according to claim 1, characterized in that The model weight of the first module excluding the preset attention module in the first model is the same as the model weight of the second module excluding the MLA module in the second model, the structure of the first module is the same as the structure of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model; The step of using the first model as a teacher model and the second model as a student model to train the MLA module to obtain the trained second model includes: With the model weight of the second module fixed, the first model is used as a teacher model and the second model is used as a student model, and the MLA module is trained to obtain the trained second model.

3. The method according to claim 2, characterized in that The model weight of the second module is obtained by applying the model weight of the first module to the second model.

4. The method according to any one of claims 1 to 3, characterized in that: The MLA module is trained based on a loss function, which includes a first loss term and / or a second loss term, wherein the first loss term is used to evaluate the difference between the output of the first model and the output of the second model, and the second loss term is used to evaluate the difference between the output of the preset attention module and the output of the MLA module.

5. The method according to claim 4, characterized in that The loss function is obtained by weighted summing the first loss term and the second loss term.

6. The method according to claim 1, characterized in that The step of using the first model as a teacher model and the second model as a student model to train the MLA module to obtain the trained second model includes: Using the first model as a teacher model and the second model as a student model, training the MLA module; After the training of the MLA module is completed, while fixing the model weight of the trained MLA module, the second module and / or the third module in the second model other than the MLA module is trained to obtain the trained second model, where the third module is different from the second module.

7. A model training device, characterized in that: include: An interface module, used to obtain a first model and a second model, wherein the first model includes a preset attention module, and the preset attention module includes one or more of the following modules: a multi-head attention MHA module, a multi-head query attention MQA module, and a group query attention GQA module, and the second model includes a multi-head potential attention MLA module, the first model is a trained AI model, the second model is an AI model to be trained, and the first model and the second model are used to process text and / or images, and the model weight of the MLA module in the second model is obtained by transforming the model weight of the preset attention module; Training modules for: The first model is used as a teacher model and the second model is used as a student model, and the MLA module is trained to obtain the trained second model.

8. The device according to claim 7, characterized in that The model weight of the first module excluding the preset attention module in the first model is the same as the model weight of the second module excluding the MLA module in the second model, the structure of the first module is the same as the structure of the second module, and the position of the first module in the first model corresponds to the position of the second module in the second model; The training module is used to: when the model weight of the second module is fixed, use the first model as a teacher model and the second model as a student model to train the MLA module to obtain the trained second model.

9. The device according to claim 8, characterized in that The model weight of the second module is obtained by applying the model weight of the first module to the second model.

10. The device according to any one of claims 7 to 9, characterized in that: The MLA module is trained based on a loss function, which includes a first loss term and / or a second loss term, wherein the first loss term is used to evaluate the difference between the output of the first model and the output of the second model, and the second loss term is used to evaluate the difference between the output of the preset attention module and the output of the MLA module.

11. The device according to claim 10, characterized in that The loss function is obtained by weighted summing the first loss term and the second loss term.

12. The device according to claim 7, characterized in that The training module is used to: Using the first model as a teacher model and the second model as a student model, training the MLA module; After the training of the MLA module is completed, while fixing the model weight of the trained MLA module, the second module and / or the third module in the second model other than the MLA module is trained to obtain the trained second model, where the third module is different from the second module.

13. A computing device cluster, characterized in that: comprising at least one computing device, the at least one computing device comprising a processor and a memory; The processor is used to execute instructions stored in the memory so that the computing device cluster executes the method according to any one of claims 1-6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the processor is enabled to execute the method according to any one of claims 1 to 6.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images