Model fine tuning method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a model fine tuning method and device, equipment and a storage medium, the method is applied to a server, and the method comprises the following steps: obtaining an input instruction of a user; determining a hidden state vector according to the input instruction; performing dimension reduction on the hidden state vector to obtain a dimension reduction result; performing self-attention pooling on a dimension reduction result to obtain a prompt vector; aggregating the prompt vectors through a preset aggregation function to obtain an aggregated vector; performing dimension raising on the aggregation vector to obtain soft prompt information; splicing the soft prompt information to the next layer of the exchanger of the model, and multiplexing the soft prompt information through a key value storage structure; according to the method, the soft prompt information can be spliced to the next layer of the exchanger, targeted guide information can be directly injected into the key structure of the model, the calculation process of the model is guided to be carried out in the direction more conforming to the expectation, and the output performance of the model is optimized.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a model fine-tuning method, device, equipment and storage medium. Background Art

[0002] With the development of science and technology, large language models continue to emerge and have achieved good results in various natural language tasks. Many methods of efficient parameter fine-tuning have been effective in many models and task verifications, and can usually achieve similar results as full parameter fine-tuning. Among these parameter efficient fine-tuning schemes, LoRa is regarded as the most effective and efficient scheme. Although LoRa is effective in the original parameter setting and brings stable results, it must add LoRa modules to multiple weights of the transformer layer in a multi-tenant setting, which significantly increases the additional computing overhead. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a model fine-tuning method, apparatus, device and storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, an embodiment of the present invention discloses a model fine-tuning method, which is applied to a server. The method includes:

[0005] Get the user's input command;

[0006] Determine a hidden state vector according to the input instruction;

[0007] Performing dimensionality reduction on the hidden state vector to obtain a dimensionality reduction result;

[0008] Performing self-attention pooling on the dimension reduction result to obtain a prompt vector;

[0009] Aggregating the prompt vectors by using a preset aggregation function to obtain an aggregated vector;

[0010] Performing dimension upgrading on the aggregated vector to obtain soft prompt information;

[0011] The soft prompt information is spliced ​​into the next layer of the switch of the model, and the soft prompt information is reused through a key-value storage structure.

[0012] Optionally, performing self-attention pooling on the dimension reduction result to obtain a prompt vector includes:

[0013] Generate a trainable matrix of the dimension reduction result;

[0014] Determining a self-attention value according to the trainable matrix and the hidden state vector;

[0015] Normalizing the self-attention values ​​to obtain a self-attention weight matrix;

[0016] The prompt vector is determined based on the self-attention matrix and the hidden state vector.

[0017] Optionally, the preset aggregation function includes a learnable activation function, and aggregating the prompt vectors by using the preset aggregation function to obtain an aggregated vector includes:

[0018]

[0019] Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b i refers to the second learned parameter and x refers to the prompt vector.

[0020] Optionally, the model includes a multi-layer neural network and further includes:

[0021] Determine the score value of the classifier of each layer of the multi-layer neural network;

[0022] Determine the cross-layer average score according to the score values ​​of the classifiers of each layer of the neural network;

[0023] The parameters of the model are adjusted according to the cross-layer average score.

[0024] Optionally, adjusting the model according to the cross-layer average score includes:

[0025] If the cross-layer average score is less than the preset average score, the number of layers of the neural network of the model is increased.

[0026] Optionally, it also includes:

[0027] determining the number of layers of the neural network in the model;

[0028] Determining the cross-layer standard deviation of each layer of the neural network according to the number of layers of the neural network in the model and the cross-layer average score;

[0029] The parameters of the model are adjusted according to the cross-layer standard deviation.

[0030] Optionally, it also includes:

[0031] During iterative training of the model, detecting performance indicators of the model on a training set, a validation set, and a test set, wherein the performance indicators include at least one of accuracy and loss value;

[0032] If the accuracy is lower than a first preset threshold, or the loss value does not reach a second preset threshold within a preset number of iterations, it is determined that the performance of the model does not meet the standard, and a reminder message is output.

[0033] The present invention also discloses a model fine-tuning device, which is applied to a server, and the device comprises:

[0034] An acquisition module is used to obtain input instructions from users;

[0035] A determination module, used to determine a hidden state vector according to the input instruction;

[0036] A dimension reduction module, used for reducing the dimension of the hidden state vector to obtain a dimension reduction result;

[0037] A pooling module, used for performing self-attention pooling on the dimension reduction result to obtain a prompt vector;

[0038] An aggregation module, used for aggregating the prompt vectors by using a preset aggregation function to obtain an aggregation vector;

[0039] A dimension-increasing module, used for performing dimension-increasing on the aggregated vector to obtain soft prompt information;

[0040] The splicing module is used to splice the soft prompt information into the next layer of the switch of the model, and reuse the soft prompt information through a key-value storage structure.

[0041] Optionally, the dimension reduction module includes:

[0042] A generation submodule, used to generate a trainable matrix of the dimensionality reduction result;

[0043] A first determination submodule, configured to determine a self-attention value according to the trainable matrix and the hidden state vector;

[0044] A normalization submodule, used to normalize the self-attention value to obtain a self-attention weight matrix;

[0045] The second determination submodule is used to determine the prompt vector according to the self-attention matrix and the hidden state vector.

[0046] Optionally, the preset aggregation function includes a learnable activation function, and aggregating the prompt vectors by using the preset aggregation function to obtain an aggregated vector includes:

[0047]

[0048] Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b irefers to the second learned parameter and x refers to the prompt vector.

[0049] Optionally, the model includes a multi-layer neural network and further includes:

[0050] A score determination module, used to determine the score value of the classifier of each layer of the neural network in the multi-layer neural network;

[0051] An average score determination module, used to determine the cross-layer average score according to the score values ​​of the classifiers of each layer of the neural network;

[0052] The first adjustment module is used to adjust the parameters of the model according to the cross-layer average score.

[0053] Optionally, the first adjustment module includes:

[0054] An additional submodule is used to increase the number of layers of the neural network of the model if the cross-layer average score is less than a preset average score.

[0055] Optionally, it also includes:

[0056] A layer number determination module, used to determine the number of layers of the neural network in the model;

[0057] A standard deviation determination module, used to determine the cross-layer standard deviation of each layer of the neural network according to the number of layers of the neural network in the model and the cross-layer average score;

[0058] The second adjustment module is used to adjust the parameters of the model according to the cross-layer standard deviation. Optionally, it also includes:

[0059] A detection module, used to detect the performance indicators of the model on the training set, the validation set and the test set during the iterative training of the model, wherein the performance indicators include at least one of accuracy and loss value;

[0060] The output module is used to determine that the performance of the model does not meet the standard and output a reminder message if the accuracy is lower than a first preset threshold, or the loss value does not reach a second preset threshold within a preset number of iterations.

[0061] The present invention also discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the above-mentioned model fine-tuning method when executed by the processor.

[0062] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned model fine-tuning method are implemented.

[0063] The embodiments of the present invention include the following advantages:

[0064] The present invention discloses a model fine-tuning method, which can splice soft prompt information to the next layer of the switch, and can directly inject targeted guidance information at the key structure of the model, guide the calculation process of the model to proceed in a more expected direction, and optimize the output performance of the model. At the same time, by using a key-value storage structure to reuse soft prompt information, it is possible to avoid repeated calculation and processing of the same prompt content, further save computing resources, and ensure that in the fine-tuning process corresponding to different stages and different inputs, these processed key prompt information can be stably and efficiently used, enhance the consistency and reliability of fine-tuning, and ultimately improve the fine-tuning effect and performance of the overall model. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flowchart of the steps of a model fine-tuning method provided by an embodiment of the present invention;

[0066] Figure 2 is a structural block diagram of a neural network model provided by an embodiment of the present invention;

[0067] Figure 3 It is a structural block diagram of a model fine-tuning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] One of the core concepts of the embodiments of the present invention is that the soft prompt information can be spliced ​​into the next layer of the switch, and targeted guidance information can be directly injected into the key structure of the model to guide the model's calculation process in a more desired direction and optimize the model's output performance. At the same time, the use of a key-value storage structure to reuse soft prompt information can avoid repeated calculations and processing of the same prompt content, further saving computing resources, and ensuring that these processed key prompt information can be used stably and efficiently during the fine-tuning process corresponding to different stages and different inputs, thereby enhancing the consistency and reliability of fine-tuning and ultimately improving the overall fine-tuning effect and performance of the model.

[0070] Reference Figure 1 , shows a flow chart of the steps of a model fine-tuning method provided by an embodiment of the present invention, which is applied to a server, and the method includes:

[0071] Step 101: Obtain the user's input instruction.

[0072] In the embodiment of the present invention, the user's input instruction can be obtained. The input instruction refers to the specific content or requirements provided by the user to the server applying the model fine-tuning method. It is the source information for starting the entire fine-tuning process. It has various forms of expression, such as the common ones:

[0073] In terms of text classification tasks: it can be an article, a short descriptive sentence, etc., such as "An Internet company has launched a new office software with powerful functions and easy operation." The purpose is to allow the model to accurately determine the category of this text after fine-tuning, such as the introduction of technology products.

[0074] In terms of sentiment analysis tasks: it can be a one-sentence evaluation, such as "The food in this restaurant tastes great, the service is very thoughtful, and I will come again next time." The model then uses the fine-tuning process to analyze the positive, negative, or neutral sentiment tendencies contained in such instructions.

[0075] In terms of machine translation tasks: the input command is the source language text to be translated, such as "I like to take a walk in the park on weekends". The server-side model is fine-tuned based on this to translate it into the target language more accurately, such as translating it into English "I like to take a walk in the park on weekends".

[0076] Assume that the server is building a news classification model, and the user input command is a news article, such as "Technology giant releases new smartphone with super camera function and innovative design". This step is to obtain the complete news text as the raw material for subsequent processing.

[0077] Step 102, determining a hidden state vector according to an input instruction.

[0078] In an embodiment of the present invention, the model converts each word in the news article into a word vector (for example, through a pre-trained word embedding model), and then performs a series of calculations and transformations on these word vectors based on a specific neural network structure (such as a recurrent neural network or the encoder part in the Transformer architecture) to obtain a hidden state vector. For example, for the sentence "Technology giant releases a new smartphone with super camera function and innovative design", after processing, the hidden state vector will encode information such as the semantic relationship and grammatical structure between the words in the sentence, such as the subject-predicate relationship between "technology giant" and "release", and the belonging relationship between "smartphone" and "camera function" and "innovative design".

[0079] Step 103, reducing the dimension of the hidden state vector to obtain a dimension reduction result.

[0080] In an embodiment of the present invention, the original hidden state vector may have a high dimension, for example, 1024 dimensions. The dimensionality reduction operation can reduce it to 256 dimensions using algorithms such as principal component analysis (PCA). In the example of news classification, this step can remove some information that may not be critical to classification, such as some subtle semantic changes or complex information representations brought about by low-frequency words, making the data more compact and focusing on the main classification features, reducing the complexity and resource consumption of subsequent calculations.

[0081] Step 104, perform self-attention pooling on the dimension reduction result to obtain a prompt vector;

[0082] In an embodiment of the present invention, during the self-attention pooling process, the model assigns attention weights according to the correlation between the vector elements after dimensionality reduction. For example, in a news classification task, for "Tech giant releases new smartphone with super camera function and innovative design", "smartphone", "camera function" and "innovative design" which are related to the core characteristics of the product may be assigned higher attention weights because they are more important for judging that the news belongs to the category of technology products. The prompt vector obtained in this way highlights the information that has an important indicative role in classification.

[0083] Step 105, aggregating the prompt vectors by using a preset aggregation function to obtain an aggregated vector;

[0084] In the embodiment of the present invention, it is assumed that a sum aggregation function is used to aggregate the prompt vectors obtained by self-attention pooling. In the news classification example, this step will integrate the prompt vectors related to each key feature to form a comprehensive vector representation. For example, the prompt vectors about product characteristics, publishing entities, etc. are merged, so that the model can grasp the key classification basis of the news as a whole, avoiding the problem of information fragmentation caused by processing each prompt vector separately.

[0085] Step 106: Upgrading the aggregated vector to obtain soft prompt information.

[0086] In an embodiment of the present invention, if the aggregation vector is reduced to 64 dimensions, the dimension increase operation can restore it to 128 dimensions. In the news classification task, the dimension increase can supplement some semantic information that may be lost in the dimension reduction and aggregation process, making the soft prompt information richer and more complete, so as to better interact with other structures of the model and transmit key classification information.

[0087] Step 107, splicing the soft prompt information into the next layer of the switch of the model, and reusing the soft prompt information through the key-value storage structure.

[0088] In an embodiment of the present invention, in the architecture of a news classification model, a switch can be used as a key module for feature exchange and integration, and the soft prompt information is spliced ​​into the next layer, just like providing additional guidance information to the key decision-making layer of the model. For example, when judging the news category, this soft prompt information can assist the switch to better weigh the importance of different features, and reuse the soft prompt information through a key-value storage structure. When a new news article is input, there is no need to recalculate these key prompt information, but directly obtain and use them from the storage structure, thereby improving processing efficiency and ensuring the consistency and stability of the use of key information in different news classification processes, thereby improving the fine-tuning effect and performance of the entire model for news classification tasks.

[0089] The present invention discloses a model fine-tuning method, which can splice soft prompt information to the next layer of the switch, and can directly inject targeted guidance information at the key structure of the model, guide the calculation process of the model to proceed in a more expected direction, and optimize the output performance of the model. At the same time, by using a key-value storage structure to reuse soft prompt information, it is possible to avoid repeated calculation and processing of the same prompt content, further save computing resources, and ensure that in the fine-tuning process corresponding to different stages and different inputs, these processed key prompt information can be stably and efficiently used, enhance the consistency and reliability of fine-tuning, and ultimately improve the fine-tuning effect and performance of the overall model.

[0090] In one embodiment of the present invention, self-attention pooling is performed on the dimensionality reduction result to obtain a prompt vector, including: generating a trainable matrix of the dimensionality reduction result; determining a self-attention value based on the trainable matrix and the hidden state vector; normalizing the self-attention value to obtain a self-attention weight matrix; and determining a prompt vector based on the self-attention matrix and the hidden state vector.

[0091] In an embodiment of the present invention, when self-attention pooling is performed on the dimensionality reduction result, a trainable matrix can be generated first. This trainable matrix is ​​usually created by random initialization or the like, and its parameters will be continuously adjusted in the subsequent model training process. For example, assuming that the dimensionality reduction result is a matrix of dimension , then the generated trainable matrix can be two matrices of dimensions and , which will play a key role in subsequent calculations. By learning appropriate parameters to adapt the characteristics of the data, the correlation between the elements in the dimensionality reduction result can be better captured.

[0092] The self-attention value is calculated using the previously generated trainable matrix and the original hidden state vector. The specific calculation method often follows the calculation formula of the self-attention mechanism, which is roughly through a series of linear transformation operations such as multiplying the hidden state vector by the trainable matrix to obtain the query, key, and value vectors, and then the self-attention value is determined by operations such as calculating the dot product of the query vector and the key vector. For example, for the element at each position, such calculation can measure the degree of correlation between it and other elements, and the numerical value reflects the intensity of mutual attention between different elements at the semantic level.

[0093] After obtaining the self-attention value, it needs to be normalized. Generally, the softmax function is used to implement this normalization operation. The softmax function converts these self-attention values ​​into a probability distribution form, so that the results corresponding to all self-attention values ​​are between 0 and 1, and the sum is 1. The self-attention weight matrix is ​​obtained. The significance of this weight matrix is ​​that it assigns corresponding weights to each element in the dimensionality reduction result, highlighting which elements are more important in the current semantic or task context, that is, elements with high weights have a greater impact on the whole and are more worthy of the model's attention.

[0094] Finally, the prompt vector is determined based on the obtained self-attention weight matrix and the hidden state vector. Usually, the self-attention weight matrix is ​​multiplied by the hidden state vector, so that the elements in the hidden state vector are recombined and weighted summed according to their respective weights to generate a prompt vector. Such a prompt vector integrates the key information filtered out by the self-attention mechanism, and can better reflect the content that is of great value to subsequent tasks such as model fine-tuning, guiding the model to focus on these key information for corresponding adjustments and learning.

[0095] The present invention can utilize self-attention pooling to mine key prompt vectors from the dimensionality reduction results, providing more targeted and important guiding information for subsequent model fine-tuning work.

[0096] In an embodiment of the present invention, the preset aggregation function includes a learnable activation function, and the prompt vector is aggregated by the preset aggregation function to obtain an aggregate vector, including:

[0097]

[0098] Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b i refers to the second learned parameter and x refers to the prompt vector.

[0099] In the embodiment of the present invention, the preset aggregation function may include a learnable activation function. Taking the learnable activation function as a Padé activation function as an example, the aggregation vector may be determined by formula (1):

[0100]

[0101] Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b i refers to the second learned parameter and x refers to the prompt vector.

[0102] In one embodiment of the present invention, the model includes a multi-layer neural network, and also includes: determining the score value of the classifier of each layer of the neural network in the multi-layer neural network; determining the cross-layer average score based on the score value of the classifier of each layer of the neural network; and adjusting the parameters of the model based on the cross-layer average score.

[0103] In an embodiment of the present invention, in a model including a multi-layer neural network, each layer of the neural network is equipped with a corresponding classifier. Determining the score values ​​of the classifiers of each layer of the neural network is to obtain the quantitative results of the classification judgment made by these classifiers on the input data. For example, in a multi-layer neural network model for recognizing handwritten numbers, it is assumed that there are 5 layers of neural networks, and the classifiers of each layer output a probability distribution of the 10 numbers 0-9 that the current input image may be. For example, the probability of the first layer classifier outputting the number "3" is 0.1, the probability of the number "5" is 0.2, and so on. These probability values ​​are the score values ​​of the classifiers of this layer. Classifiers of different layers perform different degrees of feature extraction and classification judgment on the input data based on the network level they are in, thereby obtaining their corresponding score values.

[0104] After obtaining the scores of each layer of neural network classifiers, the cross-layer average score can be calculated. This cross-layer average score aims to comprehensively consider the results of the output of each layer of classifiers, and use a unified value to reflect the average performance of each layer on the classification task. The calculation method may be to simply add the scores of each layer of classifiers for the same category, and then divide it by the number of layers to get the average score. For example, for the above-mentioned model for recognizing handwritten digits, for the category of the number "5", the scores of the 5-layer neural network classifiers are 0.2, 0.3, 0.25, 0.35, and 0.4 respectively. Add these values ​​(0.2+0.3+0.25+0.35+0.4=1.5), and then divide it by the number of layers 5 to get the cross-layer average score of 0.3. This value roughly reflects the average ability level of each layer of the entire model for the classification of the number "5".

[0105] Finally, the model parameters are adjusted based on the calculated cross-layer average score. If the cross-layer average score shows that the classification performance of a certain category is poor and the average score is low, then the model can adjust the weights, biases and other parameters in each layer of the neural network based on this feedback information through algorithms such as back propagation. For example, the weights corresponding to the features that have a positive effect on the correct classification of the number "5" may be increased, and the weights of related connections that cause confusion or misclassification may be reduced, so that the model can more accurately identify the category in subsequent operations, thereby improving the overall classification performance and optimizing the performance of the model in actual tasks. By continuously adjusting parameters based on comprehensive evaluation indicators such as the cross-layer average score, the model can continue to develop and improve in a more accurate and efficient direction.

[0106] The present invention can comprehensively evaluate the overall situation of the model based on the performance of each layer of classifiers, and optimize and adjust the model parameters in a targeted manner, which helps to improve the accuracy and reliability of the model in corresponding classification tasks.

[0107] In one embodiment of the present invention, the model is adjusted according to the cross-layer average score, including: if the cross-layer average score is less than the preset average score, the number of layers of the neural network of the model is increased.

[0108] In an embodiment of the present invention, the cross-layer average score is a quantitative indicator obtained by comprehensively considering the average performance of the neural network classifiers in each layer of the model when performing tasks. It reflects the average level of task processing of the entire model under the synergy of each layer, and the preset average score is a benchmark value set based on past experience, expected model performance or in a specific application scenario, which is used to measure whether the actual performance of the current model has reached the expected qualification standard.

[0109] When the cross-layer average score is less than the preset average score, it means that the model's current overall classification and other task execution capabilities have not met expectations, and the effect achieved by the collaborative work of each layer is not ideal. There is room for improvement. At this time, corresponding adjustment measures need to be taken to improve model performance, and increasing the number of layers of the model neural network is one possible strategy.

[0110] As the number of neural network layers increases, the model can mine deeper and more complex features from the input data. For example, in image recognition tasks, shallow neural networks may only be able to extract basic features such as image edges and simple textures. After adding more layers, subsequent layers can further extract more abstract and semantically informative high-level features based on the features extracted by the previous layers, such as the shape of objects and partial structural features. This helps the model to more accurately distinguish different image categories, thereby potentially improving the accuracy of classification, and then improving the cross-layer average score, so that it moves towards reaching or exceeding the preset average score.

[0111] In one embodiment of the present invention, it also includes: determining the number of layers of the neural network in the model; determining the cross-layer standard deviation of each layer of the neural network based on the number of layers of the neural network in the model and the cross-layer average score; and adjusting the parameters of the model based on the cross-layer standard deviation.

[0112] In an embodiment of the present invention, in a complex model architecture, a neural network is often composed of multiple stacked layers. First, the specific number of layers needs to be clarified, which can be obtained by directly viewing the definition structure of the model or the corresponding configuration file. For example, when a deep convolutional neural network is used for an image recognition task, starting from the input layer, through multiple convolutional layers, pooling layers, and fully connected layers, etc., the total number of layers contained in the neural network in the model can be determined by counting the total number of these different types of layers. Assuming that the number of layers is 8 after statistics, this information provides a basic structural reference for subsequent calculations and adjustments.

[0113] After determining the number of neural network layers and the cross-layer average score, we need to further calculate the cross-layer standard deviation of each layer of the neural network. The cross-layer standard deviation can be used to measure the degree of dispersion of the classifier scores of each layer relative to the cross-layer average score, reflecting the stability and differences in the classification performance of each layer. When calculating specifically, first obtain the score of each layer classifier for each category, and then for each category, calculate the sum of the squares of the difference between the score of each layer and the cross-layer average score of the category, divide it by the number of layers, and finally take the square root to get the cross-layer standard deviation.

[0114] In one example, for a 3-classification task (category A, B, C), the model has 5 layers of neural networks. Taking category A as an example, the scores of the 5 layers of classifiers are 0.3, 0.4, 0.2, 0.35, and 0.45 respectively. The cross-layer average score is assumed to be 0.34. First, the square of the difference between the score of each layer and the average score is calculated, which is (0.3-0.34) 2 、(0.4-0.34) 2 、(0.2-0.34) 2 、(0.35-0.34) 2 、(0.45-0.34) 2 , add these square values ​​and divide them by the number of layers 5, and then take the square root of the result to get the cross-layer standard deviation corresponding to category A. The present invention can calculate the corresponding cross-layer standard deviation for each category, which can intuitively present the fluctuation of each layer in different category classifications.

[0115] Finally, the model parameters are adjusted based on the calculated cross-layer standard deviation. A larger cross-layer standard deviation means that the classification performance of each layer is more different and the stability is poor, which means that the contribution of some layers to the classification of specific categories may not be reasonable or there may be problems such as overfitting or underfitting.

[0116] If the cross-layer standard deviation of a layer for a certain category is large, for example, in image recognition, for the identification of the "cat" category, the cross-layer standard deviation of a layer is significantly higher than that of other layers, then it may be necessary to focus on adjusting the parameters of this layer. The back-propagation algorithm can be used to appropriately reduce the connection weights that cause large fluctuations, or adjust parameters such as the bias term, so that the layer can more stably and reasonably participate in the "cat" category and the overall classification task in subsequent operations, making the classification performance of each layer more balanced and stable, thereby improving the classification accuracy and robustness of the entire model, and optimizing the overall performance of the model in actual tasks.

[0117] The present invention can calculate the cross-layer standard deviation and adjust the model parameters accordingly, so as to optimize the model more carefully from the perspective of the differences between each layer and ensure the efficiency and reliability of the model when performing tasks.

[0118] In one embodiment of the present invention, it also includes: during the iterative training of the model, detecting the performance indicators of the model on the training set, the validation set and the test set, the performance indicators including at least one of the accuracy rate and the loss value; if the accuracy rate is lower than the first preset threshold value, or the loss value does not reach the second preset threshold value within the preset number of iterations, it is determined that the performance of the model is not up to standard, and a reminder message is output.

[0119] In an embodiment of the present invention, when the model is iteratively trained, its performance indicators on the training set, validation set and test set can be detected. The training set is used to allow the model to learn patterns and regularities in the data, and the validation set is used to adjust the model's hyperparameters and evaluate the model's generalization ability during the training process to avoid overfitting. The test set is to simulate the final performance of the model in the actual application scenario.

[0120] The performance indicator of accuracy intuitively reflects the proportion of correct predictions made by the model. For classification tasks, it clearly shows the accuracy of the model's judgment. For example, in image classification tasks, accuracy is the proportion of images that the model correctly identifies as belonging to a certain category, while the loss value measures the degree of difference between the model's prediction results and the true labels. It is the goal of optimization during model training. The smaller the loss value, the closer the model's prediction is to the actual situation. It reflects the fitting effect of the model during the learning process. By paying attention to at least two performance indicators at the same time, the performance of the model at different stages can be comprehensively evaluated from different angles.

[0121] The first preset threshold is set based on the consideration of the minimum accuracy level that the model is expected to achieve in the corresponding task scenario. If, during the iterative training process, it is found that the accuracy of the model on the training set, validation set or test set is lower than this threshold, it means that the current prediction accuracy of the model has not met the expected standard.

[0122] For example, in a text sentiment analysis task, the first preset threshold is set to 80%. If the model has an accuracy of only 70% on the test set after multiple iterations of training, this indicates that the model may not have learned the emotional features contained in the text very well, or there are serious problems such as misjudgment of some situations, and it cannot accurately distinguish whether the text is positive, negative or neutral, thus affecting its reliability and effectiveness in practical applications. At this time, it can be determined that the performance of the model does not meet the standards.

[0123] The second preset threshold is a target value set for the loss value, which represents the ideal loss level that the model should achieve when fitting the data. It also limits the preset number of iterations. This is because model training cannot proceed indefinitely and needs to achieve a certain fitting effect within a reasonable amount of time and resource consumption.

[0124] If within the specified range of iterations, the loss value of the model cannot fall below the second preset threshold, for example, in a regression task, the loss value corresponding to the second preset threshold is set to 0.1, and the preset number of iterations is 100. However, after 100 iterations, the loss value of the model on the validation set is still 0.2. This means that although the model has undergone multiple rounds of learning, it still fails to approximate the real data relationship well, and the fitting effect on the input data is not good. There is room for improvement, and it can also be judged that the performance of the model does not meet the standard.

[0125] When it is determined that the performance of the model does not meet the requirements, it is necessary to output a reminder message. This reminder message can inform relevant researchers, developers, or operation and maintenance personnel, so that they can be aware of the current performance deficiencies of the model in a timely manner, so that further measures can be taken, such as adjusting the model structure, modifying training parameters, increasing the amount of training data, or replacing more appropriate optimization algorithms, etc., thereby prompting the model to continue to improve and optimize in the direction of meeting performance requirements, ensuring that the model can ultimately meet the needs of actual applications and play its due role in the corresponding task scenarios.

[0126] The present invention can effectively monitor the training status of the model, discover problems in time and provide a basis for subsequent improvements, thereby improving the overall quality and practicality of the model through a mechanism that detects performance indicators during model iterative training and determines whether they meet the standards based on set thresholds and then outputs reminder information.

[0127] like Figure 2 , showing a structural block diagram of a neural network model provided by an embodiment of the present invention, the model may include: adjustable parameters (Tunable): refers to parameters that can be adjusted during the training process. By adjusting these parameters, the performance of the model can be optimized.

[0128] Fixed part (Frozen): This part contains pre-trained weights or specific layers, which are fixed during the training process and do not change. Fixed weights or layers can speed up the training process and retain useful information in the pre-trained model.

[0129] Language model head ((C / M)LM head): It is the language model part of the model, which is used to generate text or perform other natural language processing tasks. "(C / M)" means that the language model can process both contextual information and single sentence information.

[0130] Layer L: This is a hidden layer in the neural network, which is used to extract features of the input data. In this model, there can be multiple such hidden layers.

[0131] Adjustable parameters may include: Up proj: related to upsampling, used to increase the resolution of feature maps. Upsampling is a common step in computer vision and image processing, and can be used for tasks such as image super-resolution and image enlargement; learnable activation functions: they can be optimized by the back-propagation algorithm. Learnable activation functions can make the model more flexible and adaptable to different tasks and data; adaptive pooler (SApooler): can extract features from the input data set. The adaptive pooler can automatically adjust the size and step size of the pooling window according to the size and shape of the input data; down proj: related to downsampling, used to reduce the resolution of the feature map. Downsampling can reduce the amount of computation and extract higher-level features.

[0132] The model can also include soft prompts: Soft prompts are used to guide the behavior of the model, such as generating text. They can provide additional information or constraints to help the model generate outputs that are more in line with expectations.

[0133] The fixed part may include: Embedding layer: The embedding layer converts words into vector representations for subsequent processing. In natural language processing tasks, the embedding layer is often used to convert text data into a numerical form that the model can understand.

[0134] The entire model consists of multiple such layers, each of which is connected to the next layer by arrows, indicating the direction of information flow. This structure enables the model to gradually extract the features of the input data and perform nonlinear transformations through learnable parameters and activation functions to ultimately generate the desired output.

[0135] The model is a complex deep learning model, which includes multiple adjustable parameters, fixed parts, language model heads, hidden layers, upsampling and downsampling projections, learnable activation functions, adaptive poolers, soft prompts and embedding layers. These parts work together to enable the model to handle complex natural language processing tasks or other types of tasks. The present invention can be combined with an instruction-aware mechanism to enable the model to dynamically generate optimal prompts according to task instructions without manually designing complex prompt words. The present invention uses efficient parameter fine-tuning technology and only adjusts a small number of embedding parameters, thereby reducing the complexity and resource consumption of fine-tuning. At the same time, through enhanced instruction adaptability, the model can maintain good performance on unseen tasks.

[0136] The present invention discloses a model fine-tuning method, which can splice soft prompt information to the next layer of the switch, and can directly inject targeted guidance information at the key structure of the model, guide the calculation process of the model to proceed in a more expected direction, and optimize the output performance of the model. At the same time, by using a key-value storage structure to reuse soft prompt information, it is possible to avoid repeated calculation and processing of the same prompt content, further save computing resources, and ensure that in the fine-tuning process corresponding to different stages and different inputs, these processed key prompt information can be stably and efficiently used, enhance the consistency and reliability of fine-tuning, and ultimately improve the fine-tuning effect and performance of the overall model.

[0137] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0138] Reference Figure 3 , shows a structural block diagram of a model fine-tuning device provided by an embodiment of the present invention, which is applied to a server, and the device includes:

[0139] The acquisition module 201 is used to acquire the user's input instruction;

[0140] A determination module 202, configured to determine a hidden state vector according to an input instruction;

[0141] A dimension reduction module 203 is used to reduce the dimension of the hidden state vector to obtain a dimension reduction result;

[0142] A pooling module 204 is used to perform self-attention pooling on the dimension reduction result to obtain a prompt vector;

[0143] Aggregation module 205, used to aggregate the prompt vectors by using a preset aggregation function to obtain an aggregated vector;

[0144] A dimension-increasing module 206 is used to perform dimension-increasing on the aggregated vector to obtain soft prompt information;

[0145] The splicing module 207 is used to splice the soft prompt information into the next layer of the switch of the model, and reuse the soft prompt information through the key-value storage structure.

[0146] The present invention discloses a model fine-tuning device, which can splice soft prompt information to the next layer of the switch, and can directly inject targeted guidance information at the key structure of the model, guide the calculation process of the model to proceed in a more expected direction, and optimize the output performance of the model. At the same time, by using a key-value storage structure to reuse soft prompt information, it is possible to avoid repeated calculation and processing of the same prompt content, further save computing resources, and ensure that in the fine-tuning process corresponding to different stages and different inputs, these processed key prompt information can be stably and efficiently used, enhance the consistency and reliability of fine-tuning, and ultimately improve the overall fine-tuning effect and performance of the model.

[0147] In one embodiment of the present invention, the dimension reduction module includes:

[0148] The generation submodule is used to generate a trainable matrix of the dimension reduction result;

[0149] A first determination submodule is used to determine a self-attention value according to a trainable matrix and a hidden state vector;

[0150] The normalization submodule is used to normalize the self-attention value to obtain the self-attention weight matrix;

[0151] The second determination submodule is used to determine the prompt vector according to the self-attention matrix and the hidden state vector.

[0152] In an embodiment of the present invention, the preset aggregation function includes a learnable activation function, and the prompt vector is aggregated by the preset aggregation function to obtain an aggregate vector, including:

[0153]

[0154] Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b i refers to the second learned parameter and x refers to the prompt vector.

[0155] In one embodiment of the present invention, the model includes a multi-layer neural network and further includes:

[0156] A score determination module, used to determine the score value of the classifier of each layer of the multi-layer neural network;

[0157] An average score determination module, used to determine the cross-layer average score according to the score values ​​of the classifiers of each layer of the neural network;

[0158] The first adjustment module is used to adjust the parameters of the model according to the cross-layer average score.

[0159] In one embodiment of the present invention, the first adjustment module includes:

[0160] Add a submodule to increase the number of neural network layers of the model if the cross-layer average score is less than the preset average score.

[0161] In one embodiment of the present invention, it also includes:

[0162] A layer number determination module is used to determine the number of layers of the neural network in the model;

[0163] A standard deviation determination module is used to determine the cross-layer standard deviation of each layer of the neural network according to the number of layers of the neural network in the model and the cross-layer average score;

[0164] The second adjustment module is used to adjust the parameters of the model according to the cross-layer standard deviation

[0165] In one embodiment of the present invention, it also includes:

[0166] A detection module is used to detect the performance indicators of the model on the training set, validation set and test set during the iterative training of the model, and the performance indicators include at least one of accuracy and loss value;

[0167] The output module is used to determine that the performance of the model does not meet the standard and output a reminder message if the accuracy is lower than a first preset threshold, or the loss value does not reach a second preset threshold within a preset number of iterations.

[0168] The present invention discloses a model fine-tuning device, which can splice soft prompt information to the next layer of the switch, and can directly inject targeted guidance information at the key structure of the model, guide the calculation process of the model to proceed in a more expected direction, and optimize the output performance of the model. At the same time, by using a key-value storage structure to reuse soft prompt information, it is possible to avoid repeated calculation and processing of the same prompt content, further save computing resources, and ensure that in the fine-tuning process corresponding to different stages and different inputs, these processed key prompt information can be stably and efficiently used, enhance the consistency and reliability of fine-tuning, and ultimately improve the overall fine-tuning effect and performance of the model.

[0169] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] An embodiment of the present invention further provides an electronic device, including:

[0171] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned model fine-tuning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0172] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned model fine-tuning method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0174] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0175] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0179] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0180] The above is a detailed introduction to a model fine-tuning method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A model fine-tuning method, characterized in that: Applied to a server, the method comprises: Get the user's input command; Determine a hidden state vector according to the input instruction; Performing dimensionality reduction on the hidden state vector to obtain a dimensionality reduction result; Performing self-attention pooling on the dimension reduction result to obtain a prompt vector; Aggregating the prompt vectors by using a preset aggregation function to obtain an aggregated vector; Performing dimension upgrading on the aggregated vector to obtain soft prompt information; The soft prompt information is spliced ​​into the next layer of the switch of the model, and the soft prompt information is reused through a key-value storage structure.

2. The method according to claim 1, characterized in that: The self-attention pooling is performed on the dimension reduction result to obtain a prompt vector, including: Generate a trainable matrix of the dimension reduction result; Determining a self-attention value according to the trainable matrix and the hidden state vector; Normalizing the self-attention values ​​to obtain a self-attention weight matrix; The prompt vector is determined based on the self-attention matrix and the hidden state vector.

3. The method according to claim 1, characterized in that: The preset aggregation function includes a learnable activation function, and the prompt vector is aggregated by the preset aggregation function to obtain an aggregate vector, including: Among them, R(x) refers to the aggregation vector, a j refers to the first learning parameter, b i refers to the second learned parameter and x refers to the prompt vector.

4. The method according to claim 1, characterized in that: The model includes a multi-layer neural network and also includes: Determine the score value of the classifier of each layer of the multi-layer neural network; Determine the cross-layer average score according to the score values ​​of the classifiers of each layer of the neural network; The parameters of the model are adjusted according to the cross-layer average score.

5. The method according to claim 4, characterized in that The adjusting the model according to the cross-layer average score includes: If the cross-layer average score is less than the preset average score, the number of layers of the neural network of the model is increased.

6. The method according to claim 4, characterized in that Also includes: determining the number of layers of the neural network in the model; Determining the cross-layer standard deviation of each layer of the neural network according to the number of layers of the neural network in the model and the cross-layer average score; The parameters of the model are adjusted according to the cross-layer standard deviation.

7. The method according to claim 1, characterized in that Also includes: During iterative training of the model, detecting performance indicators of the model on a training set, a validation set, and a test set, wherein the performance indicators include at least one of accuracy and loss value; If the accuracy is lower than a first preset threshold, or the loss value does not reach a second preset threshold within a preset number of iterations, it is determined that the performance of the model does not meet the standard, and a reminder message is output.

8. A model fine-tuning device, characterized in that: Applied to a server, the device comprises: An acquisition module is used to obtain input instructions from users; A determination module, used to determine a hidden state vector according to the input instruction; A dimension reduction module, used for reducing the dimension of the hidden state vector to obtain a dimension reduction result; A pooling module, used for performing self-attention pooling on the dimension reduction result to obtain a prompt vector; An aggregation module, used for aggregating the prompt vectors by using a preset aggregation function to obtain an aggregation vector; A dimension-increasing module, used for performing dimension-increasing on the aggregated vector to obtain soft prompt information; The splicing module is used to splice the soft prompt information into the next layer of the switch of the model, and reuse the soft prompt information through a key-value storage structure.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the steps of the model fine-tuning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the model fine-tuning method according to any one of claims 1 to 7 are implemented.

Images