Fine adjustment method and device of large language model, storage medium and computer equipment

Abstract

The invention discloses a fine tuning method and device for a large language model, a storage medium and computer equipment. The fine tuning method comprises the following steps: acquiring a base large model, and acquiring a sample vertical type business data set in a target business scene and a sample general business data set in multiple business scenes; classifying data in the sample general service data set to obtain sample general service data of different classification categories, and clustering the sample general service data of each classification category to obtain sample general service data under a plurality of clustering categories; respectively determining the data volume of the sample general service data under each clustering category, and respectively determining the sampling quantity of the sample general service data in each clustering category based on the data volume; and based on the sampling number, performing data sampling in the corresponding clustering category, expanding the sample sag type service data set based on the sampled sample general service data, and performing fine tuning on the base large model by using the expanded sample sag type service data set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model fine-tuning, and in particular to a large language model fine-tuning method and device, a storage medium and a computer device. BACKGROUND

[0002] In recent years, large model technology has developed rapidly and is widely used in many scenarios. During the pre-training of a base large model, the data that can be used is mostly general data, which is used to build basic world knowledge and basic common sense. However, in specific businesses, specific business data is often used to fine-tune the base large model. Fine-tuning of the base model can damage the original capabilities of the model, that is, catastrophic forgetting. If a certain amount of pre-training data is added during fine-tuning, it can improve the catastrophic forgetting to some extent.

[0003] Currently, some data is randomly sampled from pre-training data and added to specific business data to fine-tune the model. However, this sampling method leads to the problem that data with high consistency is over-sampled and data with low consistency cannot be sampled, which is not conducive to the diversity of data, thereby causing the model to overfit data with high consistency during fine-tuning, and the feature representation learned by the model has limitations, thereby reducing the generalization ability of the model. SUMMARY

[0004] The present application provides a large language model fine-tuning method and device, a storage medium and a computer device, which can increase the diversity of sample data required for fine-tuning of a large model, thereby improving the generalization ability of the fine-tuned large model, that is, improving the fine-tuning effect of the large model.

[0005] According to a first aspect of the present application, a large language model fine-tuning method is provided, comprising:

[0006] obtaining a base large model, and obtaining a sample vertical business data set in a target business scenario and a plurality of sample general business data sets in different business scenarios;

[0007] classifying the data in the sample general business data sets to obtain sample general business data in different classification categories, and clustering the sample general business data in each classification category to obtain sample general business data in a plurality of clustering categories corresponding to each classification category;

[0008] determining the data quantity of the sample general business data in each clustering category, and based on the data quantity, determining the sampling quantity of sample general business data in each clustering category corresponding to each classification category;

[0009] Based on the sampling quantity, sample general business data sampling is performed in the sample general business data corresponding to the clustering category, the sample general business data obtained through sampling is used to expand the sample vertical business data set, the base large model is fine-tuned by using the expanded sample vertical business data set, and the fine-tuned base large model is used as a large language model suitable for the target business scenario.

[0010] Optionally, based on the data quantity, the sampling quantity of sample general business data is determined in each clustering category corresponding to each classification category, comprising:

[0011] A preset sampling quantity prediction function is constructed.

[0012] The data quantity of sample general business data in each clustering category is substituted into the preset sampling quantity prediction function for sampling quantity prediction, and the sampling quantity of sample general business data in each clustering category corresponding to each classification category is obtained.

[0013] Optionally, the preset sampling quantity prediction function is constructed, comprising:

[0014] A preset initial sampling quantity prediction function is constructed, wherein the preset initial sampling quantity prediction function contains a to-be-solved sampling growth coefficient and a to-be-solved sampling offset coefficient.

[0015] An initial sample data set is obtained, wherein the initial sample data set contains sample data under different clustering categories with annotation information, and the annotation information is the sampling quantity of sample data corresponding to different clustering categories.

[0016] In the initial sample data set, a first clustering category with the largest data quantity of sample data and a second clustering category with the smallest data quantity of sample data corresponding to different clustering categories are determined, and a sample data set is formed by sample data under the first clustering category and sample data under the second clustering category with annotation information.

[0017] The to-be-solved sampling growth coefficient and the to-be-solved sampling offset coefficient in the preset initial sampling quantity prediction function are solved by using the sample data set, and the sampling growth coefficient and the sampling offset coefficient are obtained, and the preset initial sampling quantity prediction function with the sampling growth coefficient and the sampling offset coefficient is used as the preset sampling quantity prediction function.

[0018] Optionally, the data in the sample general business data set is classified to obtain sample general business data under different classification categories, comprising:

[0019] A plurality of candidate categories are determined, and candidate category keywords in each candidate category are determined.

[0020] perform word segmentation on each data in the sample general business data set respectively to obtain respective segmented words corresponding to each data respectively, and determine classification keywords affecting classification in the respective segmented words;

[0021] perform similarity matching between the classification keywords in each data and the candidate classification keywords in each candidate classification respectively, and divide each data into each candidate classification based on the similarity matching result to obtain sample general business data under different classification categories.

[0022] Optionally, the sample general business data under each classification category is clustered respectively to obtain sample general business data under a plurality of clustering categories corresponding to each classification category, including:

[0023] determine a data feature vector of the sample general business data;

[0024] initialize a centroid vector corresponding to each cluster, calculate the distance between the data feature vector and the centroid vector corresponding to each cluster respectively, and divide the sample general business data into each cluster based on the distance;

[0025] determine an updated centroid vector corresponding to each cluster based on the data feature vector corresponding to the sample general business data in each cluster, re-divide the sample general business data into each cluster based on the updated centroid vector, until the updated centroid vector does not change, and determine the sample general business data finally divided into each cluster as sample general business data under different clustering categories.

[0026] Optionally, based on the sampling quantity, sample general business data is sampled from the sample general business data under the corresponding clustering category, including:

[0027] any clustering category in any classification category in each classification category is taken as a target clustering category respectively, and a sampling processing node is allocated to each target clustering category respectively;

[0028] based on the sampling quantity, a random index is generated for each target clustering category, and based on the random index, parallel sampling of sample general business data is performed on the sample general business data corresponding to the target clustering category under the corresponding sampling processing node to obtain sample general business data sampled in the target clustering category.

[0029] Optionally, the expanded sample general business data set is used with annotation information of different business scenarios;

[0030] Fine-tune the base large model, and take the fine-tuned base large model as a large language model applicable to the target business scenario.

[0031] The expanded sample vertical business data set is divided into a training set and a test set.

[0032] The base large model is trained using the training set, and the trained base large model is tested using the test set, and the trained base large model that meets the test condition is taken as the large language model.

[0033] According to a second aspect of the present application, a fine-tuning device for a large language model is provided, comprising:

[0034] The acquisition unit is configured to acquire a base large model, and acquire a sample vertical business data set under a target business scenario and a sample general business data set under multiple business scenarios;

[0035] The clustering unit is configured to classify the data in the sample general business data set to obtain sample general business data under different classification categories, and cluster the sample general business data under each classification category to obtain sample general business data under multiple clustering categories corresponding to each classification category;

[0036] The determination unit is configured to determine the data amount of the sample general business data under each clustering category, and determine the sampling number of sample general business data in each clustering category corresponding to each classification category based on the data amount;

[0037] The fine-tuning unit is configured to sample the sample general business data in the corresponding clustering category based on the sampling number, expand the sample vertical business data set based on the sampled sample general business data, fine-tune the base large model based on the expanded sample vertical business data set, and take the fine-tuned base large model as a large language model applicable to the target business scenario.

[0038] According to a third aspect of the present application, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the above fine-tuning method of the large language model.

[0039] According to a fourth aspect of the present application, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the above fine-tuning method of the large language model when executing the program.

[0040] According to the fine-tuning method, device, storage medium and computer equipment of the large language model provided by the application, compared with the method of randomly sampling some data from the pre-training data and adding the data to the specific business data to fine-tune the model, the application classifies the data in the sample general business data set first, and then clusters the data in different classification categories, and then determines the data sampling quantity corresponding to different clustering categories, samples the sample general business data in different clustering categories based on the data sampling quantity, adds the sampled sample general business data to the sample vertical business data set, obtains the expanded sample vertical business data set, and finally fine-tunes the base large model by using the expanded sample vertical business data set. According to the application, the classification and clustering of the sample general business data set can ensure that the data in each category can be sampled, and the data sampling quantity of each clustering category can be determined by the data quantity of the sample general business data in each clustering category, so that each clustering category can be reasonably sampled, the problems of over-sampling or neglecting some categories are avoided, so that the base large model can contact more diversified data during fine-tuning, which helps the model to learn more stable feature representation, improves the generalization ability and robustness of the model, and further improves the fine-tuning effect of the model. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0042] Figure 1 A fine-tuning method flowchart of a large language model provided by an embodiment of the application is shown;

[0043] Figure 2 A fine-tuning method flowchart of another large language model provided by an embodiment of the application is shown;

[0044] Figure 3 A structural schematic diagram of a fine-tuning device of a large language model provided by an embodiment of the application is shown;

[0045] Figure 4 A structural schematic diagram of another fine-tuning device of a large language model provided by an embodiment of the application is shown;

[0046] Figure 5 An entity structure schematic diagram of a computer equipment provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0047] The application will be described in detail below with reference to the drawings and in combination with the embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0048] Currently, the way of adding some data randomly sampled from pre-training data to specific business data to fine-tune the model is not conducive to the diversity of data, thereby causing the model to overfit the data with high consistency during fine-tuning, and the feature representation learned by the model has limitations, thereby reducing the generalization ability of the model.

[0049] To solve the above problems, the embodiment of the application provides a fine-tuning method of a large language model, as shown in the formula: Figure 1 The method comprises the following steps:

[0050] 101. Obtain a base large model, and obtain a sample vertical business data set in a target business scenario and a sample general business data set in multiple business scenarios.

[0051] The target business scenario can be any of a drug regimen prediction scenario, an insurance category prediction scenario, an intelligent question and answer scenario, etc. The multiple business scenarios can include a drug regimen prediction scenario, an insurance category prediction scenario, an intelligent question and answer scenario, a customer service matching scenario, etc. The sample vertical business data set in the drug regimen prediction scenario includes sample disease information with a drug regimen label. The sample vertical business data set in the insurance category prediction scenario includes user feature information with an insurance category label. The sample vertical business data set in the intelligent question and answer scenario includes sample question description information with an answer label. The sample general business data set contains sample data in multiple scenarios, such as sample disease information with a drug regimen label in the drug regimen prediction scenario, user feature information with an insurance category label in the insurance category prediction scenario, sample question description information with an answer label in the intelligent question and answer scenario, sample access user feature information with a customer service label in the customer service matching scenario, and sample customer service feature information.

[0052] For the embodiment of the application, according to the prediction task requirements, a suitable base large model is selected, and a sample vertical business data set in a target business scenario corresponding to the prediction task requirements and a sample general business data set in multiple business scenarios are obtained. Then, data is sampled from the sample general business data set and added to the sample vertical business data set for training of the base large model. Since the vertical data is small in scale and narrow in field, the model is prone to overfit the vertical task during fine-tuning, resulting in a decline in the general ability of the model, i.e., a situation of catastrophic forgetting of the model. Based on this, the application can concentrate on training of the base large model by adding general business data to the sample vertical business data, can improve the multi-aspect ability of the model, and avoids the problem of catastrophic forgetting of the model.

[0053] 102. Classify the data in the sample general business data set to obtain sample general business data in different classification categories, and cluster the sample general business data in each classification category to obtain sample general business data in multiple cluster categories corresponding to each classification category.

[0054] For the embodiments of the present application, in order to ensure the diversity of the sampling data, it is necessary to first classify the data in the sample general business data set. Based on this, step 102 specifically includes: determining a plurality of candidate categories, and determining candidate category keywords in each candidate category; performing word segmentation processing on each data in the sample general business data set to obtain respective segmented words corresponding to each data, and determining classification keywords that affect classification in each segmented word; performing similarity matching between the classification keywords in each data and the candidate category keywords in each candidate category, and dividing each data into each candidate category based on the similarity matching result to obtain sample general business data in different classification categories.

[0055] The plurality of candidate categories include: finance, sports, culture, etc.; the candidate category keywords are core words representing the corresponding candidate categories, for example, the candidate category keywords in the finance category are, for example, sci-tech board, northbound funds, quantitative easing, etc.; the candidate category keywords in the sports category are, for example, e-sports team, sports lottery, sports equipment brand, etc.; the candidate category keywords in the culture category are, for example, network literature, short video platform, national music, etc.

[0056] Specifically, taking a target data in the sample general business data set as an example, the classification keywords that affect classification are selected from the segmented words corresponding to the target data, wherein the specific determination method of the classification keywords includes: determining the occurrence frequency p of each segmented word in the target data; determining the first data amount N of all data in the sample general business data set, and determining the second data amount D(w) of each segmented word in the sample general business data set; based on the first data amount N and the second data amount D(w), determining the data frequency F(w) of each segmented word: wherein, The data frequency F(w) and the occurrence frequency p are multiplied to obtain the weight of each segmented word in the target data; the segmented words with a weight greater than a preset weight threshold value are taken as classification keywords, wherein the preset weight threshold value is a numerical value set according to actual needs.

[0057] Further, if the classification keyword contains "sports lottery", it is determined to match the candidate category keyword under the sports category, and finally the data is divided into the sports category. If the classification keyword is "network literature" or "literature", etc., it is determined to match the candidate category keyword network literature under the culture category, and finally the data is divided into the culture category. The embodiment of the application classifies the sample general business data set, and independently samples the data under each category, avoids the model from being biased to the majority class due to data skew, improves the recognition ability of the minority class, and thus improves the generalization ability of the model.

[0058] Further, after obtaining the sample general business data under different classification categories, the sample general business data under each classification category also needs to be clustered. For example, the data under the sports category can be clustered into event reporting category, athlete dynamic category, sports project category, sports health category, etc.; the data under the finance category can be clustered into macro economy, financial market, investment strategy, etc.; the data under the culture category can be clustered into artistic creation, cultural heritage, film and television entertainment, etc. The embodiment of the application classifies and clusters the sample general business data set, which can ensure that the data under each category can be sampled, avoid the problem of over-sampling or ignoring some categories, so as to ensure that the base large model is exposed to more diversified data during fine-tuning, which helps the model to learn more stable feature representation, improves the generalization ability and robustness of the model, and thus improves the fine-tuning effect of the model.

[0059] 103, respectively determine the data amount of the sample general business data under each clustering category, and based on the data amount, respectively determine the sampling number of the sample general business data in each clustering category corresponding to each classification category.

[0060] For the embodiment of the application, the preset sampling number prediction function can be used to predict the sampling number based on the data amount of the sample general business data under each clustering category, that is, the sampling number is predicted for each clustering category, and then data sampling is performed in the corresponding clustering category according to the sampling number, which can ensure that the data under each category can be sampled, avoid the problem of over-sampling or ignoring some categories, so as to ensure that the base large model is exposed to more diversified data during fine-tuning, which helps the model to learn more stable feature representation, improves the generalization ability and robustness of the model.

[0061] 104, based on the sampling number, sample the sample general business data in the corresponding clustering category, and based on the sampled sample general business data, expand the sample vertical business data set, and use the expanded sample vertical business data set to fine-tune the base large model. The base large model after fine-tuning is used as a large language model suitable for the target business scenario.

[0062] For the embodiment of the present application, after determining the sampling quantity, sampling of the sample general service data needs to be performed in the corresponding cluster category based on the sampling quantity. Based on this, step 104 comprises: taking any cluster category in any classification category as a target cluster category, and respectively assigning a sampling processing node to each target cluster category; generating a random index for each target cluster category based on the sampling quantity, and performing parallel sampling of the sample general service data in the target cluster category corresponding to the sample general service data under the corresponding sampling processing node based on the random index, to obtain the sample general service data sampled in the target cluster category.

[0063] Specifically, the embodiment of the present application pre-sets a plurality of processing nodes in the model fine-tuning system, and respectively stores each cluster category on different processing nodes. The plurality of processing nodes perform parallel independent sampling of the data under the corresponding cluster category, that is, the sampling task is decomposed into a plurality of sub-tasks for parallel execution, which can improve the sampling efficiency. When some nodes fail, other nodes can continue to work, avoiding the problem of task interruption caused by single-point failure. In the sampling process, for each cluster category, a random index is first generated. The numbers in the random index are not repeated. If the data quantity in a target cluster category is 1000 and the sampling quantity is 100, the random index is 100 random integers that are not repeated. The interval of the random integers is 0-999. If the sample general service data in the target cluster category is [d0, d1,..., d999], the random index is [12, 45, 78,..., 999], and the corresponding sample general service data is extracted from the target cluster category according to the generated random index, which is [d12, d45, d78,..., d999]. The embodiment of the present application performs data sampling by means of random index, without complex algorithms or preprocessing steps, thereby improving the sampling efficiency. At the same time, the probability of selection of each data is the same, ensuring unbiased sampling and avoiding result distortion caused by sampling bias, thereby improving the fine-tuning effect of the model.

[0064] Further, the sampled sample general business data in each cluster category under each classification category is added to the sample vertical business data set, and finally the base large language model is fine-tuned by using the sample vertical business data set after adding the data. First, the embodiment of the present application can avoid the problem of catastrophic forgetting during model fine-tuning by adding general business data to vertical business data, thereby improving the generality of the model. Secondly, the embodiment of the present application can ensure that the data under each category can be sampled by classifying and clustering the sample general business data set, and the data sampling quantity of each cluster category can be determined by the data quantity of the sample general business data in each cluster category, which can ensure that each cluster category can be reasonably sampled, avoiding the problem of over-sampling or ignoring some categories, thereby ensuring that the base large language model is exposed to more diversified data during fine-tuning, which helps the model to learn more stable feature representation, improves the generalization ability and robustness of the model, and further improves the fine-tuning effect of the model.

[0065] According to the fine-tuning method of the large language model provided by the present application, compared with the method of randomly sampling some data from the pre-training data and adding them to the specific business data to fine-tune the model, the present application first classifies the data in the sample general business data set, and then clusters the data in different classification categories. Then determine the data sampling quantity of different cluster categories, sample the sample general business data in different cluster categories based on the data sampling quantity, add the sampled sample general business data to the sample vertical business data set, obtain the expanded sample vertical business data set, and finally fine-tune the base large language model by using the expanded sample vertical business data set. The present application can ensure that the data under each category can be sampled by classifying and clustering the sample general business data set, and the data sampling quantity of each cluster category can be determined by the data quantity of the sample general business data in each cluster category, which can ensure that each cluster category can be reasonably sampled, avoiding the problem of over-sampling or ignoring some categories, thereby ensuring that the base large language model is exposed to more diversified data during fine-tuning, which helps the model to learn more stable feature representation, improves the generalization ability and robustness of the model, and further improves the fine-tuning effect of the model.

[0066] Further, in order to better illustrate the above process of fine-tuning the large language model, as a refinement and expansion of the above embodiment, the embodiment of the present application provides another fine-tuning method of a large language model, as shown in Figure 2 The method comprises:

[0067] 201, obtaining a base large language model, and obtaining a sample vertical business data set under a target business scenario and a sample general business data set under multiple business scenarios.

[0068] 202. Classify the data in the sample general business data set to obtain sample general business data under different classification categories, and cluster the sample general business data under each classification category to obtain sample general business data under multiple clustering categories corresponding to each classification category.

[0069] For the embodiment of the present invention, in order to increase the diversity of data used in the Weitao model, it is first necessary to classify the sample general business data. Based on this, step 202 specifically includes: taking any sample general business data in the sample general business data set as a target sample general business data, and determining multiple candidate categories for the target sample general business data; counting multiple keywords associated with the target sample general business data, and predicting the category probability of the multiple keywords being divided into the multiple candidate categories; accumulating the multiple category probabilities of the multiple keywords corresponding to each candidate category in the multiple candidate categories to obtain the accumulated division probability of each candidate category; determining a target candidate category in each candidate category whose accumulated division probability is greater than a preset probability threshold, and dividing the target sample general business data into the target candidate category.

[0070] Among them, the multiple candidate categories can be financial categories, sports categories, cultural categories, catering categories, etc.; the preset probability threshold is a value set according to actual needs.

[0071] Specifically, first, the target sample general business data is segmented to obtain each segmentation, and the weight of each segmentation in the target sample general business data is determined. A plurality of segmentations whose weights are greater than a preset weight threshold (the preset weight threshold is set according to actual needs) are selected as a plurality of keywords, and then the similarity between each keyword and each candidate category is calculated. The category probability of the plurality of keywords being divided into a plurality of candidate categories is determined based on the similarity. For example, the higher the similarity, the greater the category probability. If the plurality of keywords include keyword 1, keyword 2, and keyword 3, and the plurality of candidate categories include category A, category B, and category C, the probability of keyword 1 being divided into category A is 0.3, the probability of keyword 2 being divided into category A is 0.1, and the probability of keyword 3 being divided into category A is 0.4, then the cumulative division probability of category A is 0.8. Thus, the cumulative division probability of each candidate category can be determined according to the above method, and then the target sample general business data is classified into the candidate category with the maximum cumulative division probability, or the target sample general business data is classified into the candidate category with a cumulative division probability greater than the preset probability threshold. The embodiment of the present invention performs classification by keywords, which directly reflect the text theme and reduce noise interference. At the same time, the association strength between the text and the category is quantified by probability accumulation, avoiding hard matching, thereby improving classification accuracy.

[0072] Furthermore, after classifying the data, it is also necessary to cluster the data under each classification category. Based on this, step 202 specifically includes: determining the data feature vector of the sample common business data; initializing the centroid vectors corresponding to different clusters, respectively calculating the distance between the data feature vector and the centroid vector corresponding to each cluster, and based on the distance, dividing the sample common business data into each cluster; based on the data feature vector corresponding to the sample common business data in each cluster, determining the updated centroid vector corresponding to each cluster, and based on the updated centroid vector, re-dividing the sample common business data into each cluster until the updated centroid vector does not change, and the sample common business data finally divided into each cluster is determined as sample common business data under different clustering categories.

[0073] Specifically, first, the data feature vector of each sample general business data under each cluster category is determined by using word embedding or other methods, and then the centroid vector corresponding to the initial centroid corresponding to each of the K clusters is selected. The distance between each data feature vector and the K centroid vectors is calculated for the data feature vectors corresponding to the multiple sample general business data, and each data feature vector is assigned to the cluster corresponding to the centroid vector closest to it. Then, for each cluster, the centroid of each cluster and its corresponding centroid vector are recalculated, and the multiple sample general business data are re-divided into different clusters. In this way, the sample general business data is continuously divided until the position of the centroid does not change, that is, the centroid vector does not change, and finally the sample general business data divided into different clusters are determined as sample general business data under different cluster categories. The data under each classification category is clustered in the above manner.

[0074] 203. Determine the data volume of the sample general business data under each cluster category respectively, and construct a preset sampling quantity prediction function.

[0075] For the embodiment of the present application, in order to determine the sampling quantity, firstly, a preset initial sampling quantity prediction function needs to be constructed, based on which, step 203 specifically comprises: constructing a preset initial sampling quantity prediction function, wherein the preset initial sampling quantity prediction function contains a to-be-solved sampling growth coefficient and a to-be-solved sampling offset coefficient; obtaining an initial sample data set, wherein the initial sample data set contains sample data under different clustering categories with labeled information, and the labeled information is the sample data sampling quantity corresponding to different clustering categories; determining a first clustering category corresponding to the sample data with the largest data quantity and a second clustering category corresponding to the sample data with the smallest data quantity in the initial sample data set, and constructing a sample data set from the sample data under the first clustering category with labeled information and the sample data under the second clustering category; solving the to-be-solved sampling growth coefficient and the to-be-solved sampling offset coefficient in the preset initial sampling quantity prediction function by using the sample data set, obtaining the sampling growth coefficient and the sampling offset coefficient, and taking the preset initial sampling quantity prediction function with the sampling growth coefficient and the sampling offset coefficient as the preset sampling quantity prediction function.

[0076] Among them, the sample data sampling quantity corresponding to different clustering categories is obtained according to the test verification and meets the demand of the model fine-tuning effect.

[0077] Specifically, the preset initial sampling quantity prediction function can be in the form of y=m·x+b, wherein m is the to-be-solved sampling growth coefficient, b is the to-be-solved sampling offset coefficient, y is the sampling quantity, and x is the data quantity of the sample general business data in each clustering category. For example, if the sample data under the first clustering category with the largest data quantity and the sample data under the second clustering category with the smallest data quantity in different clustering categories are used to solve the to-be-solved sampling growth coefficient and the to-be-solved sampling offset coefficient, the data quantity of the sample data under the first clustering category and the sample data sampling quantity, and the data quantity of the sample data under the second clustering category and the sample data sampling quantity are taken as known data, the above known data is brought into y=m·x+b to form an equation group, and the equation group is solved, the sampling growth coefficient and the sampling offset coefficient can be obtained, for example, the sampling growth coefficient is 0.3 and the sampling offset coefficient is 5, then the preset sampling quantity prediction function is y=0.3·x+5.

[0078] 204, respectively, the data quantity of the sample general business data in each clustering category is substituted into the preset sampling quantity prediction function to predict the sampling quantity, and the sampling quantity of the sample general business data in each clustering category corresponding to each classification category is obtained.

[0079] Specifically, if the preset sampling quantity prediction function is y=0.3·x+5, the data quantity of the sample general business data in a certain clustering category is 100, and the corresponding sampling quantity is 35.

[0080] 205. sampling the sample general service data based on the sampling number in the sample general service data corresponding to the clustering category.

[0081] For the embodiment of the present application, the sample general service data needs to be sampled according to the sampling data, and based on this, the step 205 specifically comprises: taking any clustering category in any classification category as a target clustering category, respectively, calculating the local density value of each sample general service data in the target clustering category; determining the sampling probability of each sample general service data in the target clustering category based on the local density value and the data amount of the sample general service data in the target clustering category, and sorting each sample general service data, determining the cumulative sampling probability corresponding to each sample general service data after sorting in the target clustering category based on the sampling probability; generating a random number as a sampling reference value in a preset interval, and sequentially selecting a preset number of sample general service data with cumulative sampling probability greater than or equal to the sampling reference value in each sample general service data after sorting as sample general service data obtained by sampling, wherein the preset number is the same as the sampling number.

[0082] The method for calculating the local density value of each sample general service data in the target clustering category comprises: taking any sample general service data in each sample general service data in the target clustering category as a target sample general service data, determining a plurality of neighborhood sample general service data of the target sample general service data in each sample general service data, and determining the data distance d(x i , x j ) between the target sample general service data x i and each neighborhood sample general service data x j , respectively; determining the local density value p i of the target sample general service data x j based on the data amount k of the neighborhood sample general service data and the data distance d(x i , x i ), wherein, NN k (x i ) is the neighborhood data set composed of each neighborhood sample general service data x i corresponding to the target sample general service data x j , and x j is the jth neighborhood sample general service data in the neighborhood data set.

[0083] The method for determining the sampling probability of each sample common service data in the target cluster category includes: based on the local density value ρ of each sample common service data i in the target cluster category i , the amount of sample common service data N in the target cluster category, the preset adjustment parameter α, and the sampling probability p of each sample common service data i in the target cluster category are determined i ,in, j is the identifier of a sample of general business data.

[0084] Among them, taking the target sample general business data in each sample general business data under a certain cluster category under a certain classification category as an example, the distance between the target sample general business data and the remaining sample general business data under a certain cluster category is calculated respectively (the remaining sample general business data refers to the sample general business data under a certain cluster category after removing the target sample general business data), and the sample general business data with a distance greater than a preset distance threshold (the preset distance threshold is set according to actual needs) is regarded as the neighborhood sample general business data. The preset adjustment parameter is set according to actual needs.

[0085] The preset interval is set according to actual needs, such as the interval of 0-1. Specifically, each sample general business data in the target cluster category can be sorted according to the timestamp of the sample general business data, and the data set composed of each sample general business data after sorting is D={x1,x2,...x N}, each sample general business data x i The corresponding sampling probability is p i , and determine the general business data x for each sample i The corresponding cumulative sampling probability c i , the set composed of each cumulative sampling probability is C={c1,c2,...c N},in, Among them, j is the identifier of the sample general business data, i is the i-th sample general business data, p j is the sampling probability of the jth sample general service data. Further, a random number is generated within the preset interval as the sampling reference value r, and the first one that satisfies c is found in the cumulative sampling probability. i >r sample general business data, and then continue to look for i >r sample general business data until a preset number of sample general business data is found, at which point c iThe preset number of sample general service data of r is sample general service data obtained by sampling. In this way, the data in each cluster category under each classification category can be sampled in the above manner. The embodiments of the present application determine the sampling probability of the data by the local density of the data, and sample the data according to the sampling probability. Since the sampling probability is dynamically calculated by the local density and the data quantity, a fixed proportion is not required, and the characteristics of different clusters can be adaptively adjusted. The clusters with uneven density or large size difference can still be reasonably sampled, over-sampling or under-sampling is avoided, the diversity of the data is ensured, and the fine-tuning effect of the model is improved.

[0086] 206, the sample vertical service data set is expanded based on the sample general service data obtained by sampling, and the base large model is fine-tuned using the expanded sample vertical service data set, and the fine-tuned base large model is used as a large language model suitable for the target business scenario.

[0087] For the embodiments of the present application, the sample vertical service data set is expanded using the sample general service data obtained by sampling, and then the base large model needs to be fine-tuned using the expanded sample vertical service data set. Based on this, step 206 specifically includes: dividing the expanded sample vertical service data set into a training set and a test set; training the base large model using the training set, and testing the trained base large model using the test set, and the trained base large model that meets the test conditions is used as the large language model.

[0088] Specifically, the data set can be divided first: the expanded sample vertical service data set is divided into a training set and a test set using random or specific strategies (such as stratified sampling). Then, the base large model is trained using the training set, and the loss value, mAP and other indicators in the training process are monitored to evaluate the performance of the large model. The training parameters such as learning rate, optimizer, regularization, etc. are adjusted as needed to optimize the training effect. Finally, the base large model is tested: the trained base large model is tested using the test set to evaluate its performance on unseen data. The mAP, precision, recall and other indicators on the test data are calculated and recorded. If the model performance does not meet the requirements, more iterations or adjustments can be returned to the training stage until the base large model that meets the precision requirements or recall requirements is obtained as the large language model. The present application can improve the generalization ability of the model to multiple field service data by adding some general service data in the vertical service data, and the performance of the model on the vertical task is more stable by introducing diverse data, avoiding performance fluctuations caused by data distribution bias.

[0089] According to another fine-tuning method for a large language model provided by the present invention, compared with the current method of fine-tuning the model by randomly sampling some data from pre-training data and adding it to specific business data, the present invention first classifies the data in the sample general business data set, and then clusters the data in different classification categories, and then determines the number of data sampling corresponding to different clustering categories, samples the sample general business data in different clustering categories based on the data sampling number, and adds the sampled sample general business data to the sample vertical business data set to obtain an expanded sample vertical business data set, and finally uses the expanded sample vertical business data set to fine-tune the base large model. This application classifies and clusters the sample general business data set to ensure that data under each category can be sampled. At the same time, the data sampling quantity of each cluster category is determined by the data volume of the sample general business data under each cluster category, which can ensure that each cluster category can be reasonably sampled to avoid the problem of certain categories being over-sampled or ignored, thereby ensuring that the base large model is exposed to more diverse data during fine-tuning, which helps the model learn more stable feature representations, improve the generalization ability and robustness of the model, and thus improve the fine-tuning effect of the model.

[0090] Further, as Figure 1 The specific implementation of the present invention provides a fine-tuning device for a large language model, such as Figure 3 As shown, the device includes: an acquisition unit 31, a clustering unit 32, a determination unit 33, and a fine-tuning unit 34.

[0091] The acquisition unit 31 can be used to acquire a large base model, and acquire a sample vertical business data set under a target business scenario and a sample general business data set under multiple business scenarios.

[0092] The clustering unit 32 can be used to classify the data in the sample general business data set to obtain sample general business data under different classification categories, and cluster the sample general business data under each classification category respectively to obtain sample general business data under multiple clustering categories corresponding to each classification category.

[0093] The determining unit 33 may be configured to determine the data volume of the sample common service data under each of the cluster categories respectively, and based on the data volume, determine the sampling quantity of the sample common service data in each of the cluster categories corresponding to each of the classification categories respectively.

[0094] The fine-tuning unit 34 can be configured to sample the sample general business data in the sample general business data of the corresponding cluster category based on the sampling quantity, expand the sample vertical business data set based on the sampled sample general business data, fine-tune the base large model based on the expanded sample vertical business data set, and use the fine-tuned base large model as the large language model suitable for the target business scenario.

[0095] In a specific application scenario, in order to determine the sampling quantity of the sample general business data, as shown in the following formula (1), the determination unit 33 includes a construction module 331 and a prediction module 332. Figure 4

[0096] The construction module 331 can be configured to construct a preset sampling quantity prediction function.

[0097] The prediction module 332 can be configured to respectively substitute the data quantity of the sample general business data in each cluster category into the preset sampling quantity prediction function to predict the sampling quantity, and obtain the sampling quantity of the sample general business data in each cluster category corresponding to each cluster category.

[0098] In a specific application scenario, in order to construct the preset sampling quantity prediction function, the construction module 331 can be specifically configured to construct a preset initial sampling quantity prediction function, wherein the preset initial sampling quantity prediction function contains a to-be-solved sampling growth coefficient and a to-be-solved sampling offset coefficient; obtain an initial sample data set, wherein the initial sample data set contains sample data under different cluster categories with labeled information, and the labeled information is the sample data sampling quantity corresponding to different cluster categories; determine a first cluster category with the largest data quantity of sample data and a second cluster category with the smallest data quantity of sample data corresponding to different cluster categories in the initial sample data set, and form a sample data set from the sample data under the first cluster category with labeled information and the sample data under the second cluster category; solve the to-be-solved sampling growth coefficient and the to-be-solved sampling offset coefficient in the preset initial sampling quantity prediction function by using the sample data set, obtain the sampling growth coefficient and the sampling offset coefficient, and use the preset initial sampling quantity prediction function with the sampling growth coefficient and the sampling offset coefficient as the preset sampling quantity prediction function.

[0099] In a specific application scenario, in order to classify the data in the sample general business data set, the clustering unit 32 includes a determination module 321, a word segmentation module 322, and a matching module 323.

[0100] The determination module 321 can be configured to determine a plurality of candidate categories and determine candidate category keywords in each candidate category. ​

[0101] The word segmentation module 322 can be configured to perform word segmentation on each data in the sample general business data set respectively to obtain respective words segmented from each data, and determine classification keywords affecting classification in the respective words.

[0102] The matching module 323 can be configured to perform similarity matching between the classification keywords in each data and the candidate classification keywords in each candidate classification respectively, and divide each data into each candidate classification based on the similarity matching result to obtain sample general business data under different classification categories.

[0103] In a specific application scenario, in order to cluster the sample general business data under each classification category respectively, the clustering unit 32 further includes a first division module 324.

[0104] The determination module 321 can be further configured to determine a data feature vector of the sample general business data.

[0105] The first division module 324 can be configured to initialize a centroid vector corresponding to each cluster, calculate a distance between the data feature vector and the centroid vector corresponding to each cluster respectively, and divide the sample general business data into each cluster based on the distance.

[0106] The first division module 324 can be specifically configured to determine an updated centroid vector corresponding to each cluster based on the data feature vector corresponding to the sample general business data in each cluster, re-divide the sample general business data into each cluster based on the updated centroid vector, until the updated centroid vector does not change, and determine the sample general business data finally divided into each cluster as sample general business data under different clustering categories.

[0107] In a specific application scenario, in order to perform data sampling, the fine-tuning unit 34 includes an allocation module 341 and a parallel sampling module 342.

[0108] The allocation module 341 can be configured to take any clustering category in any classification category in each classification category as a target clustering category respectively, and allocate a sampling processing node to each target clustering category respectively.

[0109] The parallel sampling module 342 can be configured to generate a random index for each target clustering category based on the sampling quantity, and perform parallel sampling of sample general business data in the sample general business data corresponding to the target clustering category under the corresponding sampling processing node based on the random index to obtain sample general business data sampled in the target clustering category.

[0110] In a specific application scenario, in order to fine-tune the base large model, the fine-tuning unit 34 further comprises a second division module 343 and a fine-tuning module 344.

[0111] The second division module 343 can be used to divide the expanded sample vertical business data set into a training set and a test set.

[0112] The fine-tuning module 344 can be used to train the base large model using the training set, and test the trained base large model using the test set, and take the trained base large model that meets the test condition as the large language model.

[0113] It should be noted that other corresponding descriptions of the functions of the fine-tuning device for a large language model provided by the embodiments of the present application can be referred to the corresponding descriptions of the method shown in Figure 1 , which will not be repeated here.

[0114] Based on the above method shown in Figure 1 , accordingly, the embodiments of the present application also provide a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the following steps: obtaining a base large model, and obtaining a sample vertical business data set in a target business scenario and a sample general business data set in multiple business scenarios; classifying the data in the sample general business data set to obtain sample general business data in different classification categories, and respectively clustering the sample general business data in each classification category to obtain sample general business data in multiple clustering categories corresponding to each classification category; respectively determining the data amount of the sample general business data in each clustering category, and respectively determining the sampling number of sample general business data in each clustering category corresponding to each classification category based on the data amount; based on the sampling number, sampling sample general business data in the corresponding clustering category, and expanding the sample vertical business data set based on the sampled sample general business data, fine-tuning the base large model using the expanded sample vertical business data set, and taking the fine-tuned base large model as a large language model applicable to the target business scenario.

[0115] Based on the above method shown in Figure 1 and the device shown in Figure 3 , the embodiments of the present application also provide an entity structure diagram of a computer device, as shown in Figure 5As shown, the computer device comprises a processor 41, a memory 42, and a computer program stored on the memory 42 and executable on the processor, wherein the memory 42 and the processor 41 are both arranged on a bus 43, and the processor 41 implements the following steps when executing the program: obtaining a base large model, and obtaining a sample vertical business data set in a target business scenario and a sample general business data set in multiple business scenarios; classifying the data in the sample general business data set to obtain sample general business data in different classification categories, and clustering the sample general business data in each classification category respectively to obtain sample general business data in multiple clustering categories corresponding to each classification category; determining the data amount of the sample general business data in each clustering category respectively, and based on the data amount, determining the sampling number of sample general business data in each clustering category corresponding to each classification category; based on the sampling number, sampling sample general business data in the corresponding clustering category, and based on the sampled sample general business data, expanding the sample vertical business data set, and using the expanded sample vertical business data set to fine-tune the base large model, and using the fine-tuned base large model as a large language model applicable to the target business scenario.

[0116] Through the technical scheme of the present application, the present application first classifies the data in the sample general business data set, and then clusters the data in different classification categories, and then determines the data sampling number corresponding to different clustering categories, samples sample general business data in different clustering categories based on the data sampling number, adds the sampled sample general business data to the sample vertical business data set, obtains the expanded sample vertical business data set, and finally uses the expanded sample vertical business data set to fine-tune the base large model. By classifying and clustering the sample general business data set, the present application can ensure that the data in each category can be sampled, and by determining the data sampling number of each clustering category based on the data amount of the sample general business data in each clustering category, the present application can ensure that each clustering category can be reasonably sampled, avoiding the problem of over-sampling or ignoring some categories, thereby ensuring that the base large model is exposed to more diverse data during fine-tuning, which helps the model to learn more stable feature representations, improves the generalization ability and robustness of the model, and further improves the fine-tuning effect of the model.

[0117] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with a general purpose computing device, which can be centralized on a single computing device or distributed over a network of multiple computing devices, and optionally implemented with program code executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be performed in a different order than shown, or made into individual integrated circuit modules, or multiple modules or steps made into a single integrated circuit module. Thus, the application is not limited to any particular combination of hardware and software.

[0118] The preferred embodiments of the application described above are intended to be merely illustrative, and numerous modifications and adaptations will be apparent to those skilled in the art. Such modifications and adaptations do not depart from the spirit and scope of the application as set forth in the following claims.

Claims

1. A fine-tuning method for a large language model, characterized in that: include: Obtain the large base model, as well as sample vertical business datasets for the target business scenario and sample general business datasets for multiple business scenarios; Classifying the data in the sample general service data set to obtain sample general service data under different classification categories, and clustering the sample general service data under each classification category to obtain sample general service data under multiple clustering categories corresponding to each classification category; Determining the data volume of the sample common service data under each of the cluster categories respectively, and determining the sampling quantity of the sample common service data in each of the cluster categories corresponding to each of the classification categories based on the data volume; Based on the sampling quantity, sample general business data is sampled in the sample general business data of the corresponding cluster category, and the sample vertical business data set is expanded based on the sample general business data obtained by sampling, and the expanded sample vertical business data set is used to fine-tune the base large model, and the fine-tuned base large model is used as the large language model suitable for the target business scenario.

2. The method according to claim 1, characterized in that Determining the sampling quantity of the sample common service data in each cluster category corresponding to each classification category based on the data volume includes: Construct a preset sampling quantity prediction function; The data volume of the sample common business data in each cluster category is respectively substituted into the preset sampling quantity prediction function to perform sampling quantity prediction, and the sampling quantity of the sample common business data in each cluster category corresponding to each classification category is obtained.

3. The method according to claim 2, characterized in that Construct a preset sampling quantity prediction function, including: Constructing a preset initial sampling number prediction function, wherein the preset initial sampling number prediction function includes a sampling growth coefficient to be solved and a sampling offset coefficient to be solved; Acquire an initial sample data set, wherein the initial sample data set includes sample data of different cluster categories with annotation information, and the annotation information is the number of sample data samples corresponding to different cluster categories; Determining, in the initial sample data set, a first clustering class with the largest amount of sample data corresponding to different clustering classes and a second clustering class with the smallest amount of sample data, and forming a sample data set from the sample data of the first clustering class and the sample data of the second clustering class with labeled information; The sample data set is used to solve the sampling growth coefficient and the sampling offset coefficient to be solved in the preset initial sampling quantity prediction function to obtain the sampling growth coefficient and the sampling offset coefficient, and the preset initial sampling quantity prediction function with the sampling growth coefficient and the sampling offset coefficient is used as the preset sampling quantity prediction function.

4. The method according to claim 1, wherein Classify the data in the sample general business data set to obtain sample general business data under different classification categories, including: determining a plurality of candidate categories, and determining a candidate category keyword in each of the candidate categories; Perform word segmentation processing on each data in the sample general business data set to obtain each word segment corresponding to each data, and determine the classification keywords that affect the classification in each word segment; The classification keywords in each data are respectively matched with the candidate category keywords in each candidate category for similarity, and each data is divided into each candidate category based on the similarity matching result to obtain sample general business data under different classification categories.

5. The method according to claim 1, wherein Clustering the sample common service data under each of the classification categories respectively to obtain sample common service data under multiple cluster categories corresponding to each of the classification categories includes: Determining a data feature vector of the sample general service data; Initializing the centroid vectors corresponding to different clusters, respectively calculating the distance between the data feature vector and the centroid vector corresponding to each cluster, and dividing the sample general service data into each cluster based on the distance; Based on the data feature vector corresponding to the sample common business data in each of the clusters, the updated centroid vector corresponding to each of the clusters is determined; based on the updated centroid vector, the sample common business data is re-divided into each of the clusters until the updated centroid vector does not change; and the sample common business data finally divided into each of the clusters is determined as sample common business data under different clustering categories.

6. The method according to claim 1, characterized in that Based on the sampling quantity, sampling the sample general service data from the sample general service data of the corresponding cluster category includes: Any cluster category in any classification category in each classification category is respectively used as a target cluster category, and a sampling processing node is respectively allocated to each target cluster category; Based on the sampling quantity, a random index is generated for each target cluster category, and based on the random index, parallel sampling of the sample general business data corresponding to the target cluster category under the corresponding sampling processing node is performed to obtain the sample general business data sampled in the target cluster category.

7. The method according to claim 1, characterized in that Utilize the expanded sample vertical business data set with annotation information of different business scenarios; Fine-tuning the base large model and using the fine-tuned base large model as a large language model applicable to the target business scenario includes: Divide the expanded sample vertical business data set into a training set and a test set; The base large model is trained using a training set, and the trained base large model is tested using a test set, and the trained base large model that meets the test conditions is used as the large language model.

8. A fine-tuning device for a large language model, characterized in that: include: An acquisition unit, used to acquire a large base model, and acquire a sample vertical business data set under a target business scenario and a sample general business data set under multiple business scenarios; a clustering unit, configured to classify the data in the sample general service data set to obtain sample general service data under different classification categories, and cluster the sample general service data under each classification category to obtain sample general service data under multiple clustering categories corresponding to each classification category; a determining unit, configured to respectively determine the data volume of the sample common service data under each of the cluster categories, and based on the data volume, respectively determine the sampling quantity of the sample common service data in each of the cluster categories corresponding to each of the classification categories; A fine-tuning unit is used to sample sample general business data from the sample general business data of the corresponding cluster category based on the sampling quantity, and expand the sample vertical business data set based on the sample general business data obtained by the sampling, and use the expanded sample vertical business data set to fine-tune the base large model, and use the fine-tuned base large model as the large language model suitable for the target business scenario.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images