Model processing method and device

Abstract

One or more embodiments of the invention disclose a model processing method and device, and the method comprises the steps: firstly determining a pre-training language model, setting a low-rank adaptation matrix on an initial weight matrix bypass of the pre-training language model, and obtaining a first pre-training language model; then obtaining a plurality of pieces of historical task information, historical request data corresponding to each piece of historical task information and used for representing a user request, and a historical response label corresponding to each piece of historical request data in a current service scene, and finally taking the historical task information and the historical request data corresponding to each piece of historical task information as input data, and based on an output result of a first pre-training language model corresponding to the input data and a historical response label corresponding to each piece of historical request data, keeping a pre-training weight of the first pre-training language model unchanged, and training an increment weight corresponding to the low-rank adaptation matrix through a preset loss function, and obtaining the trained target large language model.

Description

Technical Field

[0001] This document relates to the technical field of fine-tuning large language models, and more particularly to a method and device for processing a model. Background Art

[0002] In the field of deep learning, there are two common ways to train large models: model fine-tuning and retraining. Taking large language models as an example, due to the rapid development of natural language processing technology, the model training method of fine-tuning based on pre-trained language models has been increasingly widely used.

[0003] In the related art, when the model needs to accurately understand instructions and follow complex instructions to complete tasks, full fine-tuning based on a pre-trained language model is a common model training method. For example, as people pay more and more attention to their privacy data, in risk prevention and control business, respond to risk assessment instructions input by users, or in query business, respond to user query instructions for querying historical transaction records of specified accounts. However, in the case of limited training resources (hardware training resources for model training, such as video memory, memory, processor, etc.), full fine-tuning training based on a pre-trained language model is not easy to implement, and in the case of limited training resources, the large language model obtained by full fine-tuning training based on a pre-trained language model cannot meet user needs. Therefore, it is necessary to provide a model training method for fine-tuning a pre-trained language model that is suitable for limited training resources, complex instructions, and multi-task situations. Summary of the invention

[0004] On the one hand, one or more embodiments of the present specification provide a model processing method, including: determining a pre-trained language model, and setting a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity; obtaining multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, wherein the historical response labels are sample labels determined based on the user's expected output information under the historical task information; taking the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeping the pre-trained weights of the first pre-trained language model unchanged, and training the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function to obtain a trained target large language model.

[0005] On the other hand, one or more embodiments of the present specification provide a model processing device, including: a first pre-trained language model determination module, which determines a pre-trained language model, and sets a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity; an information acquisition module, which obtains multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, wherein the historical response labels are sample labels determined based on the user's expected output information under the historical task information; a training module, which uses the historical task information and the historical request data corresponding to each historical task information as input data, and based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeps the pre-trained weights of the first pre-trained language model unchanged, and trains the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function to obtain a trained target large language model.

[0006] On the other hand, one or more embodiments of the present specification provide an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, which, when the executable instructions are executed, enable the processor to: determine a pre-trained language model, and set a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables the learning capacity of the first pre-trained language model to be greater than the preset learning capacity; obtain multiple historical task information in the current business scenario, each historical task information The historical task information corresponds to historical request data for representing user requests, and the historical response label corresponding to each historical request data, wherein the historical response label is a sample label determined based on the expected output information of the user under the historical task information; the historical task information and the historical request data corresponding to each historical task information are used as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

[0007] On the other hand, one or more embodiments of the present specification provide a storage medium for storing a computer program, which can be executed by a processor to implement the following process: determining a pre-trained language model, and setting a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity; obtaining multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, wherein the historical response labels are sample labels determined based on the expected output information of the user under the historical task information; taking the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeping the pre-trained weights of the first pre-trained language model unchanged, and training the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function to obtain a trained target large language model.

[0008] On the other hand, one or more embodiments of the present specification also provide a computer program product, including a computer program, which implements the following process when executed by a processor: determining a pre-trained language model, and setting a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity; obtaining multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, wherein the historical response labels are sample labels determined based on the expected output information of the user under the historical task information; taking the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeping the pre-trained weights of the first pre-trained language model unchanged, and training the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function to obtain a trained target large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 is a schematic flow chart of a model processing method according to an embodiment of this specification; Figure 2 is a schematic diagram of the principle of the low-rank adaptation technology according to an embodiment of this specification; Figure 3 is a schematic flow chart of another model processing method according to an embodiment of this specification; Figure 4 is a schematic diagram of training samples and test samples for model training according to an embodiment of this specification; Figure 5 is a schematic block diagram of a processing device of a model according to an embodiment of the present specification; Figure 6 It is a schematic block diagram of an electronic device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0011] One or more embodiments of the present specification provide a method and device for processing a model.

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0013] like Figure 1 As shown, the embodiment of this specification provides a method for processing a model, and the execution subject of the method can be a server. The server can be an independent server, or a server cluster composed of multiple servers, etc. The server can be a background server in the financial field or online shopping field, or a background server of an application, etc. In this embodiment, the server is used as an example for detailed description. For the execution process of the terminal device, please refer to the following related content, which will not be repeated here. The method can specifically include the following steps: In step S102, a pre-trained language model is determined, and a low-rank adaptation matrix for describing the incremental weights generated during the model fine-tuning process is set on the bypass of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model. The rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity.

[0014] Model fine-tuning (i.e. instruction fine-tuning) is to fine-tune the pre-trained language model on the tasks described by a series of natural language instructions, thereby stimulating the language understanding ability of the pre-trained language model so that it can accurately respond to the natural language instructions used by humans in daily life. Through model fine-tuning, the gap between the training goal of the pre-trained language model and the goal that users expect the large language model to follow natural language instructions (i.e. the expected output information of following the instructions) can be effectively bridged. The instruction data set in the fine-tuning process can be defined as: ,in, Representative task set The feature tasks in Representation Task The corresponding natural language instructions, Indicates input information. Indicates that in the task Download and input information The purpose of model fine-tuning is to enable the large language model to Generate matching response information. For example, "Please answer the following question: Which city is the capital of China?", answer: "The capital of China is Beijing". Among them, "Please answer the following question:" is the task information (Task description information), "Which city is the capital of China?" is the input information , "The capital of China is Beijing" is the output result .

[0015] Low Rank Adaptation (LoRA) assumes that the updated value of the model weight has a lower "intrinsic rank", hereinafter referred to as rank. The pre-trained language model is built based on the neural network. The low-rank adaptation technology can be used on each weight matrix in the neural network. Taking the weight matrix of the C network layer in the neural network as an example, the principle of the low-rank adaptation technology is to add a side branch network to the C network layer of the pre-trained language model, such as Figure 2As shown in the figure, the side network uses the product of two low-rank adaptation matrices (i.e., rank decomposition matrices) to approximate the update value of the C network layer weight during parameter fine-tuning, where Pretrained Weights is the pre-trained weight of the pre-trained language model, that is, the initial weight of the pre-trained language model.

[0016] Continuing with the C network layer weight matrix as an example, assume that the initial weight matrix of a certain layer C network layer of the pre-trained language model is , and when the model is updated, the updated value of the C network layer weight can be processed by low-rank decomposition, that is, ,in , During the model training process, the weight of the first pre-trained language model consists of two parts: the initial weight of the pre-trained language model and the incremental weight. Based on this assumption, the parameter update of the C network layer becomes the product of two low-rank adaptation matrices A and B: ,in, is the input information of the model, is the output of the model. When the model is trained on downstream tasks, the initial weight matrix of the C network layer weights It is a fixed value and there is no gradient update. Only the parameters of the low-rank adaptation matrices A and B are updated. The elements in matrix A are initialized using a random Gaussian initialization strategy, while the elements in matrix B are all initialized to 0. Therefore, the initial value of ∆W is 0 during model training.

[0017] Since the number of trainable parameters is reduced when the pre-trained language model is fine-tuned using low-rank adaptation technology, it is generally believed that the limited learning ability in the fine-tuning method based on low-rank adaptation technology will lead to a decline in model performance compared to the full fine-tuning method, and based on this, it is concluded that the fine-tuning method under low-rank adaptation technology is more effective than the full fine-tuning method only when data resources are limited and the fine-tuning task is the priority. The embodiments of this specification adjust the intrinsic rank of the low-rank adaptation matrix so that the low-rank adaptation technology can also be used in application scenarios with diversified fine-tuning tasks, sufficient learning capacity, and limited model training resources such as video memory, memory, and processors, thereby providing new ideas for constructing a widely applicable instruction fine-tuning method. In implementation, the pre-trained language model may be a BERT (Bidirectional Encoder Representations from Transformers) model, a GPT (Generative Pre-Trained) model, etc. By setting a low-rank adaptation matrix on a bypass path of an initial weight matrix of the pre-trained language model, a first pre-trained language model for model fine-tuning is obtained.

[0018] The rank of the low-rank adaptation matrix in the first pre-trained language model is greater than the preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity. Among them, the preset rank can be determined according to the specific business scenario, that is, adaptive adjustment is performed according to the specific business (task adaptive processing), for example: the preset rank of the risk prevention and control business and the query business can be set to different values ​​with small differences, or can be set to the same value. The rank of the low-rank adaptation matrix in the first pre-trained language model is set to be greater than the preset rank, that is, the learning capacity of the first pre-trained language model is expanded by increasing the intrinsic rank of the first pre-trained language model. The larger the learning capacity of the first pre-trained language model, the more parameters that can be trained in the first pre-trained language model, and the more model parameters that can be changed in the first pre-trained language model, thereby effectively improving the learning ability of the first pre-trained language model.

[0019] In implementation, the learning capacity of the first pre-trained language model includes the proportion of trainable model parameters of the first pre-trained language model, and the learning capacity of the first pre-trained language model is determined according to the current business scenario. The rank of the learning capacity of the first pre-trained language model being greater than the preset learning capacity includes: the proportion of trainable model parameters of the first pre-trained language model is greater than the preset trainable model parameter proportion, wherein the proportion of trainable model parameters of the first pre-trained language model is proportional to the rank of the low-rank adaptation matrix in the first pre-trained language model, and the larger the rank of the low-rank adaptation matrix, the larger the proportion of trainable model parameters of the first pre-trained language model.

[0020] For common risk prevention and control services, user information query services, customer service services, etc., the relationship between the rank of the low-rank adaptation matrix and the ratio of trainable model parameters in the first pre-trained language model in the embodiment of this specification can be shown in Table 1 below. In implementation, the rank of the low-rank adaptation matrix can be determined according to the current business scenario, or the relationship between the rank of the low-rank adaptation matrix and the ratio of trainable model parameters can be appropriately adjusted according to different business scenarios.

[0021] Table 1

[0022] In Table 1, r is the rank of the low-rank adaptation matrix in the first pre-trained language model, α is a hyperparameter, and α is a multiple of r. Taking the rank of the low-rank adaptation matrix as 64 as an example, when the rank of the low-rank adaptation matrix is ​​64, the proportion of trainable model parameters is 0.9861%, when the rank of the low-rank adaptation matrix is ​​256, the proportion of trainable model parameters is 3.831%, and when the rank of the low-rank adaptation matrix is ​​512, the proportion of trainable model parameters is 7.3794%. In conventional low-rank adaptation technology, the rank of the low-rank adaptation matrix is ​​usually much smaller than the intrinsic rank of the original matrix dimension (that is, the rank of the low-rank adaptation matrix is ​​much smaller than the rank of the initial weight matrix). For example, for the LLaMA-7B model with a hidden dimension of 4096, the rank of the low-rank adaptation matrix is ​​usually less than or equal to 16, such as 4, 8 or 16. Taking the parameters in the first row of Table 1 as an example, the rank of the low-rank adaptation matrix is ​​16, and the corresponding trainable model parameter ratio is 0.2484%, and only 0.2484% of the parameters are trained each time the model is trained. In the embodiment of this specification, by making the rank of the low-rank adaptation matrix greater than the preset rank to increase the rank of the first pre-trained language model, the proportion of trainable model parameters can be effectively increased (up to a ratio of 1%-10%), thereby expanding the learning capacity of the first pre-trained language model, and then improving the performance of the trained model. Among them, the preset rank can be adaptively adjusted according to the specific business, and the rank of the first pre-trained language model can also be determined according to the preset trainable model parameter ratio. The embodiment of this specification does not limit this.

[0023] Taking the LLaMA-7B model as the basic model (i.e., the pre-trained language model) as an example, the four weight matrices in the self-attention module of the first pre-trained language model based on the LLaMA-7B model ( , , and ), where the intrinsic rank of the low-rank adaptation matrix can be updated from 16 to 4096, thereby expanding the learning capacity of the first pre-trained language model.

[0024] In implementation, a low-rank adaptation matrix for describing the incremental weights generated during model fine-tuning is set on a bypass path of the initial weight matrix of the pre-trained language model to obtain a first pre-trained language model. The low-rank adaptation matrix describing the incremental weights can be set in any neural network layer of the pre-trained language model, or some neural network layers (such as fully connected layers) can be selected to set the low-rank adaptation matrix. This specification does not limit this embodiment.

[0025] In step S104, multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data are obtained. The historical response label is a sample label determined based on the user's expected output information under the historical task information.

[0026] Historical task information is the historical task description information (natural language instructions corresponding to historical tasks), each historical task information corresponds to historical request data used to represent user requests, that is, the input information under the historical task, and each historical request data corresponds to the historical response label, that is, the target answer corresponding to the input information under the historical task.

[0027] Taking the risk prevention and control business as an example, according to step S104, multiple historical transaction task information, historical transaction request data corresponding to each historical transaction task information for representing the user's request, and historical transaction response tags corresponding to each historical transaction request data are obtained under the current risk prevention and control business scenario. The historical transaction response tags are sample tags determined based on the user's expected output information under the historical transaction task information. Among them, each historical task information and the historical request data corresponding to each historical task information for representing the user's request constitute an instruction sample, that is, the input data of the model.

[0028] In step S106, historical task information and historical request data corresponding to each historical task information are used as input data. Based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

[0029] Since the weight of the first pre-trained language model includes two parts: the initial weight of the pre-trained language model and the incremental weight, during the model training process of the first pre-trained language model, the pre-trained weight of the first pre-trained language model is kept unchanged, and the incremental weight is trained until the target large language model converges or the preset number of model training iterations is reached. During each model iteration training process, the loss information in the model training is determined based on the output result corresponding to each historical request data and the historical response label corresponding to each historical request data, and the update value of the weight matrix in the model training process is determined based on the loss information using the low-rank adaptation matrix.

[0030] Among them, the preset loss function can adopt the cross entropy loss function.

[0031] The embodiment of the present specification provides a method for processing a model, by determining a pre-trained language model, and setting a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model, to obtain a first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity, and then obtain multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, the historical response labels are sample labels determined based on the expected output information of the user under the historical task information, and finally, the historical task information and the historical request data corresponding to each historical task information are used as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained by a preset loss function to obtain a trained target large language model. In this way, by setting a low-rank adaptation matrix on the bypass of the initial weight matrix of the pre-trained language model, and using the low-rank adaptation matrix to implement model fine-tuning training, compared with the model training method of full fine-tuning, the number of trainable model parameters of the pre-trained language model in the fine-tuning process can be effectively reduced, parameter optimization can be achieved, and computing resources and model training time can be significantly reduced under the premise of ensuring the model training effect, thereby effectively improving the practicality and scalability of the model in a resource-limited environment. Moreover, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank, the rank of the first pre-trained language model is increased, which can effectively increase the proportion of trainable model parameters, thereby expanding the learning capacity of the first pre-trained language model, and then improving the performance of the target large language model obtained by training. Especially in the case of limited model training resources such as video memory, memory, and processor, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank to expand the rank of the low-rank adaptation matrix, the learning capacity of the model during the model training process can be expanded, which can not only improve the model's ability to solve known tasks, but also enhance the model's generalization ability on complex tasks and new tasks.

[0032] In the embodiment of the present specification, in the above step S102, a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process is set on the bypass of the initial weight matrix of the pre-trained language model, and the processing method for obtaining the first pre-trained language model can be various. An optional processing method is provided below, and the specific processing can refer to the following steps S1022-S1026.

[0033] In step S1022, a low-rank adaptation matrix is ​​set in the fully connected layer weight matrix of the pre-trained language model in the form of a side branch network.

[0034] By setting a low-rank adaptation matrix used to describe the increment of the fully connected layer weight matrix in the form of a side branch network in the fully connected layer weight matrix, the optimization degree of the model parameters can be further improved, which is beneficial to reducing computing resources and improving resource utilization while ensuring the model training effect.

[0035] In step S1024, the weight of the pre-trained language model is determined based on the pre-trained weight of the pre-trained language model and the incremental weight corresponding to the low-rank adaptation matrix.

[0036] The weight of the pre-trained language model is obtained by summing the pre-trained weights of the pre-trained language model and the incremental weights corresponding to the low-rank adaptation matrix.

[0037] In step S1026, a first pre-trained language model is determined based on the initial weight matrix and the low-rank adaptation matrix of the pre-trained language model.

[0038] like Figure 3 As shown, after step S106, steps S108 and S110 are also included.

[0039] In step S108, based on the target task information related to the preset service, request data for representing the user request in the preset service is obtained.

[0040] In implementation, the preset business may be a risk prevention and control business, an inquiry business, a recommendation business or a customer service business. If it is a risk prevention and control business, the target task information may be a risk assessment instruction, such as: "Please conduct a risk assessment on the following users", and the corresponding request data may be the transaction data of the user who is applying for a loan. If it is an inquiry business, the target task information may be a customer information inquiry instruction, such as "Please query the transaction record of customer A", and the corresponding request data may be customer information or historical transaction record information of customer A. If it is a recommendation business, the target task information may be a recommendation instruction, such as: "Please recommend items based on user preferences", and the corresponding request data may be user information, item information and user behavior information. For customer service business, the target task information may be customer service task description information, such as "Please answer the following questions", and the corresponding request data may be the user's customer service question information or service request information.

[0041] In step S110, the target task information and the request data are input into the trained target large language model to obtain response data matching the request data based on the target task information.

[0042] During implementation, if it is a risk prevention and control business, the response data may be the risk assessment result; if it is an inquiry business, the response data may be the customer information inquiry result; if it is a recommendation business, the response data may be the preset recommendation result; if it is a customer service business, the response data may be the response result corresponding to the customer service question or the response result corresponding to the service request information.

[0043] In the embodiments of this specification, after the target large language model is obtained in the above step S106, there are various processing methods for model optimization of the target large language model. An optional processing method is provided below. For details, please refer to the processing of the following steps S112-S116.

[0044] In step S112, an instruction fine-tuning test data set is constructed based on the data corresponding to the trained tasks and the data corresponding to the unseen training tasks in the model training process.

[0045] like Figure 4 As shown, let the fine-tuning test dataset be sampled from known training tasks and unseen tasks respectively. The known training tasks include the tasks used to train the model in the instruction fine-tuning phase, and the unseen tasks refer to new tasks that the model has not seen during the training process.

[0046] In step S114, based on the constructed instruction fine-tuning test dataset, the in-task generalization performance and cross-task generalization performance of the target large language model are verified, the data corresponding to the trained tasks are used to verify the in-task generalization performance of the target large language model, and the data corresponding to the untrained tasks are used to verify the cross-task generalization performance of the target large language model.

[0047] The data corresponding to the trained tasks are used to verify the in-task generalization performance of the target large language model, thereby evaluating the model's memory performance and measuring the effectiveness of the model on the training tasks. The data corresponding to the unseen training tasks are used to verify the cross-task generalization performance of the target large language model and measure the generalization performance of the model on different tasks.

[0048] In step S116, parameters of the target large language model are adjusted based on the verification result to obtain a target large language model after parameter adjustment.

[0049] By constructing an instruction fine-tuning test data set and obtaining the corresponding verification results, and finally further adjusting the parameters of the target large language model according to the verification results, the accuracy of the parameters of the obtained target large language model can be improved, thereby improving the performance of the target large language model.

[0050] In the embodiment of this specification, the pre-trained language model in the above step S102 is a model built based on a Transformer neural network.

[0051] In the embodiment of the present specification, the rank value of the matrix preset in the above step S102 is 64, and the rank value of the low-rank adaptation matrix is ​​512.

[0052] In implementation, when the rank of the preset matrix is ​​64, the rank of the low-rank adaptation matrix of the first pre-trained language model is greater than 64, then the proportion of trainable model parameters of the first pre-trained language model basically reaches more than 1%, which can greatly improve the proportion of trainable model parameters of the model compared to the conventional intrinsic rank, and can effectively expand the learning capacity of the model. The larger the rank of the low-rank adaptation matrix, the larger the proportion of trainable model parameters of the first pre-trained language model. Based on the preset business, when the rank of the low-rank adaptation matrix is ​​512, the effect of fine-tuning the first pre-trained language model based on the low-rank adaptation matrix is ​​basically the same as the effect of full fine-tuning, and the proportion of trainable model parameters is about 7%, which can not only ensure the effect of model training but also effectively reduce the computing resources of model training.

[0053] In order to compare the performance of the target large language model obtained by model fine-tuning based on the low-rank adaptation technology in the embodiments of this specification with the large language model obtained by full fine-tuning, the corresponding experimental process and experimental data are provided below.

[0054] Step 1: Build a benchmark dataset based on the English data in the SUPER-NATURALINSTRUCTION (NIv2) dataset. The benchmark dataset contains 757 training tasks and 119 unseen test tasks, covering a variety of tasks.

[0055] Step 2: Build an instruction fine-tuning training dataset.

[0056] In order to evaluate the performance of low-rank adaptation matrices with different learning capacities in diverse training tasks, a training set is constructed. , the training set includes four independent training subsets, named , , as well as Specifically, by strategically sampling from the official NIv2 training tasks, we obtained task sets of 100, 250, 500, and all available tasks. Then, we selected 100 samples from each selected task to construct the training subset. , , as well as It should be noted that in the process of selecting tasks, sampling can be performed according to the distribution ratio of task categories in the NIv2 official training set to ensure that each constructed training subset is consistent with the NIv2 official training set in terms of task diversity distribution.

[0057] The above four training subsets all contain all the training data of their previous datasets. For example, its content fully covers To improve the robustness and credibility of the verification results, three rounds of sampling can be performed to construct three independent training data sets. .

[0058] Step three: construct the first instruction fine-tuning test data set.

[0059] The first instruction fine-tuning test dataset is further divided into two different test subsets: and . Includes the tasks used to train the model during the instruction fine-tuning phase to evaluate the in-task generalization performance of the instruction fine-tuning model. It can be expressed as: The generalization test set within this task can evaluate the model's memory ability and is used to measure the effectiveness of the model on the training task. Includes tasks that the model has not seen during training, which is used to measure the cross-task generalization performance of the model. It can be expressed as: , which is used to measure the model's ability to generalize across tasks that have not been seen before.

[0060] Step 4: Determine task instructions and experimental indicators.

[0061] Continuing to take the NIv2 dataset as an example, the task instructions used in the NIv2 dataset include: task definition, positive examples and negative examples. In the embodiments of this specification, only the task definition is used as the task instruction, and the instruction does not contain any positive examples or negative examples.

[0062] The exact match accuracy and Rouge-L score are used to evaluate the model performance and the model performance on three training datasets is reported. The average score on the test set is used to improve the robustness and reliability of the experimental results.

[0063] Step 5: Conduct experiments based on the above experimental setting data and obtain experimental data.

[0064] In order to determine the relationship between the learning capacity of the pre-trained language model and the complexity of the task, the embodiments of this specification adjust the trainable model parameters and the number of learning tasks of the low-rank adaptation technology, and test the in-task generalization ability and cross-task generalization ability of the instruction fine-tuning model under different conditions. The in-task generalization experimental results and the cross-task generalization experimental results are shown in Tables 2 and 3 below, respectively.

[0065] Table 2

[0066] Table 3

[0067] From the experimental results in Table 2 and Table 3 above, we can see that: (1) In high-resource, multi-task application scenarios, the target large language model based on a low-rank adaptation matrix with large learning capacity exhibits intra-task generalization capabilities and cross-task generalization capabilities, which in some cases can reach or even exceed the level of a large language model using the full fine-tuning method (full fine-tuning model).

[0068] (2) Low-rank adaptation technology with large learning capacity can enhance the model's performance in cross-task generalization. The large language model based on a low-rank adaptation matrix with small learning capacity is inferior to the full fine-tuning model in terms of memory ability, and its performance on the in-task generalization test set is slightly insufficient. However, as the number of learnable parameters increases, the cross-task generalization ability of the target large language model based on a low-rank adaptation matrix with large learning capacity gradually improves, showing better cross-task generalization performance than the full fine-tuning model.

[0069] (3) The target large language model based on the low-rank adaptation matrix with large learning capacity shows better robustness in generalization ability. As the model learning capacity increases, the in-task generalization ability and cross-task generalization ability of the target large language model based on the low-rank adaptation matrix with large learning capacity continue to improve. Combined with Table 1, it can be seen that by adopting the model training method in the embodiment of this specification, only 7% of the parameters in the training model are needed to achieve an effect similar to 100% of the parameters of the training model (full fine-tuning processing method), that is: increasing the rank of the low-rank adaptation matrix to 512 can achieve the model training effect of the full fine-tuning processing method, therefore, it can effectively save computing resources.

[0070] The embodiment of the present specification provides a method for processing a model, by determining a pre-trained language model, and setting a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model, to obtain a first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity, and then obtain multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, the historical response labels are sample labels determined based on the expected output information of the user under the historical task information, and finally, the historical task information and the historical request data corresponding to each historical task information are used as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained by a preset loss function to obtain a trained target large language model. In this way, by setting a low-rank adaptation matrix on the bypass of the initial weight matrix of the pre-trained language model, and using the low-rank adaptation matrix to implement model fine-tuning training, compared with the model training method of full fine-tuning, the number of trainable model parameters of the pre-trained language model in the fine-tuning process can be effectively reduced, parameter optimization can be achieved, and computing resources and model training time can be significantly reduced under the premise of ensuring the model training effect, thereby effectively improving the practicality and scalability of the model in a resource-limited environment. Moreover, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank, the rank of the first pre-trained language model is increased, which can effectively increase the proportion of trainable model parameters, thereby expanding the learning capacity of the first pre-trained language model, and then improving the performance of the target large language model obtained by training. Especially in the case of limited model training resources such as video memory, memory, and processor, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank to expand the rank of the low-rank adaptation matrix, the learning capacity of the model during the model training process can be expanded, which can not only improve the model's ability to solve known tasks, but also enhance the model's generalization ability on complex tasks and new tasks.

[0071] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0072] The above is a model processing method provided in one or more embodiments of this specification. Based on the same idea, one or more embodiments of this specification also provide a model processing device, such as Figure 5 shown.

[0073] The model processing device includes: a first pre-trained language model determination module 210, an information acquisition module 220 and a training module 230, wherein: A first pre-trained language model determination module 210 determines a pre-trained language model, and sets a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on an initial weight matrix bypass of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; The information acquisition module 220 acquires a plurality of historical task information in the current business scenario, historical request data corresponding to each historical task information and used to represent a user request, and a historical response label corresponding to each historical request data, wherein the historical response label is a sample label determined based on the user's expected output information under the historical task information; The training module 230 takes the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeps the pre-trained weights of the first pre-trained language model unchanged, and trains the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function to obtain the trained target large language model.

[0074] In the embodiment of this specification, the first pre-trained language model determination module 210 includes: A low-rank adaptation matrix setting unit, which sets a low-rank adaptation matrix in a side branch network manner in a fully connected layer weight matrix of a pre-trained language model; A pre-trained language model weight determination unit, which determines the weight of the pre-trained language model based on the pre-trained weight of the pre-trained language model and the incremental weight corresponding to the low-rank adaptation matrix; The first pre-trained language model determination unit determines the first pre-trained language model based on the initial weight matrix and the low-rank adaptation matrix of the pre-trained language model.

[0075] In the embodiment of this specification, the processing device of the model also includes: A request data acquisition module, based on target task information related to the preset business, acquires request data used to represent user requests in the preset business; The response data determination module inputs the target task information and the request data into the trained target large language model to obtain the response data matching the request data based on the target task information.

[0076] In the embodiment of this specification, the processing device of the model also includes: The test data set construction module constructs the instruction fine-tuning test data set based on the data corresponding to the trained tasks and the data corresponding to the unseen training tasks in the model training process; The performance verification module verifies the in-task generalization performance and cross-task generalization performance of the target large language model based on the constructed instruction fine-tuning test dataset. The data corresponding to the trained tasks are used to verify the in-task generalization performance of the target large language model, and the data corresponding to the untrained tasks are used to verify the cross-task generalization performance of the target large language model. The parameter adjustment module adjusts the parameters of the target large language model based on the verification result to obtain the target large language model after parameter adjustment.

[0077] In the embodiment of the present specification, the pre-trained language model determined in the first pre-trained language model determination module 210 is constructed based on a Transformer neural network, and the loss function includes a cross-entropy loss function.

[0078] In the embodiment of the present specification, the learning capacity of the first pre-trained language model in the first pre-trained language model determination module 210 includes the ratio of trainable model parameters of the first pre-trained language model, the learning capacity of the first pre-trained language model is determined according to the current business scenario, and the rank of the low-rank adaptation matrix is ​​determined according to the current business scenario.

[0079] In the embodiment of this specification, the rank value of the matrix preset in the first pre-trained language model determination module 210 is 64, and the rank value of the low-rank adaptation matrix is ​​512.

[0080] In the embodiment of this specification, the preset services corresponding to the current business scenario in the information acquisition module 220 include: risk prevention and control services, query services, recommendation services or customer service services.

[0081] The embodiment of the present specification provides a model processing device, which determines a pre-trained language model through a first pre-trained language model determination module, and sets a low-rank adaptation matrix for describing the incremental weights generated in the model fine-tuning process on the bypass of the initial weight matrix of the pre-trained language model to obtain the first pre-trained language model, the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that can make the learning capacity of the first pre-trained language model greater than the preset learning capacity. Then, based on the information acquisition module, multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data are obtained, and the historical response labels are sample labels determined based on the expected output information of the user under the historical task information. Finally, through the training module, the historical task information and the historical request data corresponding to each historical task information are used as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain a trained target large language model. In this way, by setting a low-rank adaptation matrix on the bypass of the initial weight matrix of the pre-trained language model, and using the low-rank adaptation matrix to implement model fine-tuning training, compared with the model training method of full fine-tuning, the number of trainable model parameters of the pre-trained language model in the fine-tuning process can be effectively reduced, parameter optimization can be achieved, and computing resources and model training time can be significantly reduced under the premise of ensuring the model training effect, thereby effectively improving the practicality and scalability of the model in a resource-limited environment. Moreover, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank, the rank of the first pre-trained language model is increased, which can effectively increase the proportion of trainable model parameters, thereby expanding the learning capacity of the first pre-trained language model, and then improving the performance of the target large language model obtained by training. Especially in the case of limited model training resources such as video memory, memory, and processor, by setting the rank of the low-rank adaptation matrix to be greater than the preset rank to expand the rank of the low-rank adaptation matrix, the learning capacity of the model during the model training process can be expanded, which can not only improve the model's ability to solve known tasks, but also enhance the model's generalization ability on complex tasks and new tasks.

[0082] Those skilled in the art should understand that the processing device of the above-mentioned model can be used to implement the processing method of the model described above, and the detailed description thereof should be similar to the description of the method part above, and will not be further described here to avoid tediousness.

[0083] Based on the same idea, one or more embodiments of this specification also provide an electronic device, such as Figure 6As shown. The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 302 may be a temporary storage or a permanent storage. The application stored in the memory 302 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input and output interfaces 305, and one or more keyboards 306.

[0084] Specifically in this embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including the following computer executable instructions: Determine a pre-trained language model, and set a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on a bypass of an initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; Obtain multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, where the historical response labels are sample labels determined based on the user's expected output information under the historical task information; Taking historical task information and historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

[0085] One or more embodiments of this specification provide a storage medium, the storage medium is used to store computer executable instructions, and the executable instructions implement the following process when executed by a processor: Determine a pre-trained language model, and set a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on a bypass of an initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; Obtain multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, where the historical response labels are sample labels determined based on the user's expected output information under the historical task information; Taking historical task information and historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

[0086] One or more embodiments of the present specification further provide a computer program product, including a computer program, which implements the following process when executed by a processor: Determine a pre-trained language model, and set a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on a bypass of an initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; Obtain multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing user requests, and historical response labels corresponding to each historical request data, where the historical response labels are sample labels determined based on the user's expected output information under the historical task information; Taking historical task information and historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

[0087] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0089] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0090] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0091] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0092] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system, or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] One or more embodiments of this specification are described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0097] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0098] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0099] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0100] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0101] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0102] The above description is only one or more embodiments of this specification and is not intended to limit this application. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification should be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A model processing method, comprising: Determine a pre-trained language model, and set a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on a bypass of an initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; Acquire multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing a user request, and a historical response label corresponding to each historical request data, wherein the historical response label is a sample label determined based on the user's expected output information under the historical task information; Taking the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

2. The method according to claim 1, wherein the step of setting a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on an initial weight matrix bypass of the pre-trained language model to obtain a first pre-trained language model comprises: Setting a low-rank adaptation matrix in a side branch network manner in a fully connected layer weight matrix of the pre-trained language model; Determining a weight of the pre-trained language model based on the pre-trained weight of the pre-trained language model and the incremental weight corresponding to the low-rank adaptation matrix; A first pre-trained language model is determined based on the initial weight matrix of the pre-trained language model and the low-rank adaptation matrix.

3. The method according to claim 1, further comprising: Based on the target task information related to the preset service, obtaining request data for representing the user request in the preset service; The target task information and the request data are input into the trained target large language model to obtain response data matching the request data based on the target task information.

4. According to the method of claim 1, after obtaining the trained target large language model, the method further comprises: Based on the data corresponding to the trained tasks and the data corresponding to the unseen training tasks during the model training process, a test dataset for instruction fine-tuning is constructed; Based on the constructed instruction fine-tuning test dataset, the in-task generalization performance and cross-task generalization performance of the target large language model are verified, the data corresponding to the trained tasks are used to verify the in-task generalization performance of the target large language model, and the data corresponding to the unseen training tasks are used to verify the cross-task generalization performance of the target large language model; Parameters of the target large language model are adjusted based on the verification result to obtain a target large language model after parameter adjustment.

5. According to the method of claim 1, the pre-trained language model is constructed based on a Transformer neural network, and the loss function includes a cross entropy loss function.

6. According to the method according to claim 1, the learning capacity of the first pre-trained language model includes the proportion of trainable model parameters of the first pre-trained language model, the learning capacity of the first pre-trained language model is determined according to the current business scenario, and the rank of the low-rank adaptation matrix is ​​determined according to the current business scenario.

7. According to the method of claim 1, the rank value of the preset matrix is ​​64, and the rank value of the low-rank adaptation matrix is ​​512.

8. According to the method of claim 1, the preset service corresponding to the current service scenario includes: Risk prevention and control business, inquiry business, recommendation business or customer service business.

9. A model processing device, comprising: A first pre-trained language model determination module determines a pre-trained language model, and sets a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on an initial weight matrix bypass of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than a preset learning capacity; An information acquisition module, which acquires multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing a user request, and a historical response label corresponding to each historical request data, wherein the historical response label is a sample label determined based on the user's expected output information under the historical task information; A training module takes the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, keeps the pre-trained weights of the first pre-trained language model unchanged, trains the incremental weights corresponding to the low-rank adaptation matrix through a preset loss function, and obtains a trained target large language model.

10. An electronic device comprising: processor; as well as a memory arranged to store computer executable instructions which, when executed, cause the processor to: Determine a pre-trained language model, and set a low-rank adaptation matrix for describing incremental weights generated in a model fine-tuning process on a bypass of an initial weight matrix of the pre-trained language model to obtain a first pre-trained language model, wherein the rank of the low-rank adaptation matrix is ​​greater than a preset rank, and the preset rank is a rank that enables a learning capacity of the first pre-trained language model to be greater than the preset learning capacity; Acquire multiple historical task information in the current business scenario, historical request data corresponding to each historical task information for representing a user request, and a historical response label corresponding to each historical request data, wherein the historical response label is a sample label determined based on the user's expected output information under the historical task information; Taking the historical task information and the historical request data corresponding to each historical task information as input data, based on the output result of the first pre-trained language model corresponding to the input data and the historical response label corresponding to each historical request data, the pre-trained weights of the first pre-trained language model are kept unchanged, and the incremental weights corresponding to the low-rank adaptation matrix are trained through a preset loss function to obtain the trained target large language model.

Images