Model fine tuning method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a model fine tuning method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a knowledge document of a target domain; extracting question and answer pairs in the knowledge document to obtain a question and answer pair set in the knowledge document; determining a quality score of each question and answer pair according to the text information of each question and answer pair in the question and answer pair set; according to the quality score of each question and answer pair, screening a target question and answer pair from the question and answer pair set, and taking the target question and answer pair as a fine tuning training sample of the to-be-fine-tuned model; and based on the fine tuning training sample, performing fine tuning optimization on the to-be-fine-tuned model to obtain a target model. According to the method provided by the scheme, the domain classification is performed on the knowledge document, and the target question and answer pair which can be used as the fine tuning training sample is screened according to the quality score of the question and answer pair, so that the quality of the fine tuning training sample is improved, deviation of the model in the fine tuning process is avoided, and the fine tuning precision of the model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model fine-tuning method, device, electronic device and storage medium. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, large models with strong understanding and generation capabilities have been gradually applied to various fields. The performance of large models mainly depends on the training samples used in their training process. Therefore, how to provide large model training samples has become a key research topic.

[0003] In related technologies, a random generation method is usually adopted to randomly generate training samples that conform to the distribution of the original training data set.

[0004] However, during the fine-tuning stage of large models, high-quality training samples become the key factor in determining whether the model can accurately adapt to the needs of specific industries or scenarios. The quality of training samples generated by random generation cannot be guaranteed, which will lead to problems such as deviations in the fine-tuning process of large models. Summary of the invention

[0005] The present application provides a model fine-tuning method, device, electronic device and storage medium to solve the problem that related technologies cannot guarantee the quality of generated training samples, which will lead to defects such as deviations in large models during the fine-tuning process.

[0006] The first aspect of the present application provides a model fine-tuning method, comprising:

[0007] Acquire knowledge documents in the target domain;

[0008] Extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document;

[0009] Determining a quality score of each question-answer pair according to text information of each question-answer pair in the question-answer pair set;

[0010] According to the quality scores of each question-answer pair, a target question-answer pair is selected from the question-answer pair set, and the target question-answer pair is used as a fine-tuning training sample of the model to be fine-tuned;

[0011] Based on the fine-tuning training samples, the model to be fine-tuned is fine-tuned and optimized to obtain a target model.

[0012] In an optional implementation, extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document includes:

[0013] Splitting the knowledge document into natural segments to obtain a number of initial knowledge text segments;

[0014] According to the paragraph length of each initial paragraph of the knowledge text, determining whether each initial paragraph of the knowledge text is a paragraph to be merged;

[0015] If any of the initial sections of the knowledge text is a section to be merged, then the section to be merged is merged into the next initial section of the knowledge text;

[0016] In the case where the paragraph length of the initial segment of the knowledge text is not less than the preset condition value, the initial segment of the knowledge text is used as the knowledge text segment, so that the paragraph lengths of the several knowledge text segments finally obtained are not less than the preset condition value; wherein the paragraph length of the paragraph to be merged is less than the preset condition value;

[0017] For any of the knowledge text segments, obtaining a question statement in the knowledge text segment to obtain a question statement corresponding to each knowledge text segment;

[0018] For any of the question statements, a preset number of answer text segments corresponding to the question statement are selected from the plurality of knowledge text segments;

[0019] taking the preset number of answer text segments as a knowledge set;

[0020] Inputting the question statement and the knowledge set into the model to be fine-tuned, so as to answer the question statement based on the model to be fine-tuned according to the knowledge set, so as to generate a corresponding answer statement;

[0021] A question-answer pair is generated according to each of the question statements and the answer statements to obtain a set of question-answer pairs in the knowledge document.

[0022] In an optional implementation, determining the quality score of each question-answer pair according to the text information of each question-answer pair in the question-answer pair set includes:

[0023] Determining a perplexity score for each question-answer pair in the question-answer pair set according to the sentence coherence represented by the text information of each question-answer pair;

[0024] Determining an explainability score for each question-answer pair according to the question-answer quality represented by the text information of each question-answer pair in the question-answer pair set;

[0025] Determining a word frequency score for each question-answer pair in the question-answer pair set according to a word frequency feature represented by the text information of each question-answer pair;

[0026] The quality score of each question-answer pair is determined based on the perplexity score, interpretability score and word frequency score of each question-answer pair.

[0027] In an optional implementation, determining the perplexity score of each question-answer pair in the question-answer pair set according to the sentence coherence represented by the text information of each question-answer pair includes:

[0028] For any question-answer pair in the question-answer pair set, performing word splitting on an answer sentence in the question-answer pair to convert the answer sentence into a word sequence of a target length;

[0029] Determining the perplexity of the answer statement according to the probability of occurrence of each word in the word sequence;

[0030] Determine a perplexity score of the answer sentence according to the perplexity of the answer sentence;

[0031] The text information of the question-answer pair includes the probability of occurrence of each word in the word sequence.

[0032] In an optional implementation, determining the perplexity of the answer statement according to the probability of occurrence of each word in the word sequence includes:

[0033] Based on the following formula, the confusion of the answer statement is determined:

[0034]

[0035] Where PP(W) represents the perplexity of the answer sentence, N represents the target length of the word sequence, and P(w i |w1,w2,…,w i-1 ) represents the probability of the i-th word appearing in the word sequence.

[0036] In an optional implementation, determining the word frequency score of each question-answer pair in the question-answer pair set according to the word frequency feature represented by the text information of each question-answer pair includes:

[0037] For any question-answer pair in the question-answer pair set, performing word segmentation processing on an answer sentence in the question-answer pair to obtain a word segmentation result of the answer sentence;

[0038] Performing word segmentation processing on the knowledge set corresponding to the answer sentence to obtain a knowledge set word segmentation result; wherein the answer sentence is generated based on the knowledge set;

[0039] Performing word segmentation processing on the knowledge document to obtain a knowledge document word segmentation result;

[0040] Determine a word frequency vector of the answer sentence according to the frequency of occurrence of each answer word represented by the word segmentation result of the answer sentence in the answer sentence;

[0041] Determine a knowledge set word frequency vector according to the occurrence frequency of each knowledge word represented by the knowledge set word segmentation result in the knowledge set;

[0042] Obtaining the total number of knowledge text segments of the knowledge document, the number of first target knowledge text segments containing answer words, and the number of second target knowledge text segments containing knowledge words;

[0043] Determine an answer sentence weight vector according to the answer sentence word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of first target knowledge text segments containing answer words;

[0044] Determining a knowledge set weight vector according to the knowledge set word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of second target knowledge text segments containing knowledge words;

[0045] The word frequency score of each question-answer pair is determined based on the answer sentence weight vector and the knowledge set weight vector.

[0046] In an optional implementation, determining the word frequency score of each question-answer pair according to the answer sentence weight vector and the knowledge set weight vector includes:

[0047] The word frequency score of each question-answer pair is determined based on the following formula:

[0048]

[0049] Among them, Sc3 is the word frequency score of the question and answer pair, Va i Vk represents the answer sentence weight vector of the i-th question-answer pair in the question-answer pair set, i represents the knowledge set weight vector corresponding to the i-th question-answer pair, and n represents the total number of question-answer pairs in the question-answer pair set.

[0050] A second aspect of the present application provides a model fine-tuning device, comprising:

[0051] The acquisition module is used to obtain knowledge documents in the target field;

[0052] An extraction module, used for extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document;

[0053] A scoring module, used to determine a quality score of each question-answer pair according to the text information of each question-answer pair in the question-answer pair set;

[0054] A screening module, used to screen target question-answer pairs from the question-answer pair set according to the quality scores of each question-answer pair, and use the target question-answer pairs as fine-tuning training samples for the model to be fine-tuned;

[0055] The fine-tuning module is used to fine-tune and optimize the model to be fine-tuned based on the fine-tuning training samples to obtain a target model.

[0056] A third aspect of the present application provides an electronic device, comprising: at least one processor and a memory;

[0057] The memory stores computer-executable instructions;

[0058] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0059] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect are implemented.

[0060] A fifth aspect of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the method described in the first aspect and various possible designs of the first aspect.

[0061] The technical solution of this application has the following advantages:

[0062] The present application provides a model fine-tuning method, device, electronic device and storage medium, the method comprising: obtaining knowledge documents in a target domain; extracting question-answer pairs from the knowledge documents to obtain a set of question-answer pairs in the knowledge documents; determining the quality score of each question-answer pair based on the text information of each question-answer pair in the set of question-answer pairs; screening target question-answer pairs in the set of question-answer pairs based on the quality scores of each question-answer pair, and using the target question-answer pairs as fine-tuning training samples for the model to be fine-tuned; based on the fine-tuning training samples, fine-tuning and optimizing the model to be fine-tuned to obtain the target model. The method provided by the above scheme improves the quality of the fine-tuning training samples by classifying the knowledge documents by domains, and screening the target question-answer pairs that can be used as fine-tuning training samples based on the quality scores of the question-answer pairs, thereby avoiding deviations in the model during the fine-tuning process and improving the fine-tuning accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technical descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0064] Figure 1This is a schematic diagram of the structure of the model fine-tuning system based on the embodiments of the present application;

[0065] Figure 2 A schematic diagram of a flow chart of a model fine-tuning method provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of the overall process of the model fine-tuning method provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the structure of a model fine-tuning device provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of the structure of an exemplary model fine-tuning device provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0070] The above drawings have shown clear embodiments of the present application, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0073] With the rapid development of artificial intelligence technology, big models, as the core driving force, are gradually penetrating into all walks of life, significantly improving the ability of data processing, analysis and decision-making. The performance and effect of big models depend to a large extent on the quality of the data used in their training process. Especially in the fine-tuning stage of big models, high-quality data has become a key factor in determining whether the model can accurately adapt to the needs of specific industries or scenarios.

[0074] Large model fine-tuning is to conduct targeted data training on pre-trained large models to optimize their performance in specific tasks or scenarios. In this process, data quality directly determines the effect of fine-tuning and the final performance of the model. High-quality data can not only improve the accuracy of the model, but also enhance the robustness and generalization ability of the model, so that the model can still maintain stable performance in complex and changing real-world environments. On the contrary, if the data quality is not high, it will cause deviations in the model during fine-tuning, and even cause hallucinations, impaired conversation ability, and catastrophic forgetting, which will seriously affect the practicality and reliability of the model.

[0075] In order to meet the demand for high-quality data for fine-tuning large models, researchers and developers are constantly exploring various data generation methods. Among them, common methods include random sample generation based on statistical distribution, synthetic data generation based on machine learning models, and the use of deep generative models such as variational autoencoders (VAE) and generative adversarial networks (GAN) to generate highly relevant synthetic data. These methods have their own advantages and disadvantages. For example, the statistical distribution generation method is simple but lacks flexibility, the machine learning model can fit complex distributions but has the risk of overfitting, and the deep generative model can generate high-quality synthetic data, but the model training is difficult and the computational cost is high.

[0076] Although there are many data generation methods, they still face many challenges in practical applications, the most prominent of which are the low quality of generated data and unclear field classification.

[0077] 1. Low quality of generated data: Related data generation methods often fail to fully simulate the complexity and diversity of real-world data, resulting in deficiencies in the accuracy, completeness, and consistency of the generated data. This not only affects the effect of fine-tuning large models, but may also introduce new biases and noise, reducing the performance of the model.

[0078] 2. Unclear domain classification: When applying large models across domains, data from different domains need to be accurately classified and labeled. However, related data generation methods often have difficulty accurately capturing the subtle differences between domains, resulting in unclear domain classification of the generated data. This not only increases the difficulty of model training, but may also cause the model to perform poorly in specific domains.

[0079] In summary, the importance of data quality in the process of fine-tuning large models is self-evident. However, the problems of low quality of generated data and unclear field classification in current data generation methods have seriously restricted the wide application and in-depth development of large models in various fields. Therefore, it is an important issue to study more efficient and accurate data generation methods and improve data quality and field classification accuracy.

[0080] In response to the above problems, an embodiment of the present application provides a model fine-tuning method, device, electronic device and storage medium, the method comprising: obtaining knowledge documents in the target domain; extracting question-answer pairs from the knowledge documents to obtain a set of question-answer pairs in the knowledge documents; determining the quality score of each question-answer pair based on the text information of each question-answer pair in the set of question-answer pairs; screening target question-answer pairs in the set of question-answer pairs based on the quality scores of each question-answer pair, and using the target question-answer pairs as fine-tuning training samples for the model to be fine-tuned; based on the fine-tuning training samples, fine-tuning and optimizing the model to be fine-tuned to obtain the target model. The method provided by the above scheme improves the quality of the fine-tuning training samples by classifying the knowledge documents by domains, and screening the target question-answer pairs that can be used as fine-tuning training samples based on the quality scores of the question-answer pairs, thereby avoiding deviations in the model during the fine-tuning process and improving the fine-tuning accuracy of the model.

[0081] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0082] First, the structure of the model fine-tuning system on which this application is based is described:

[0083] The model fine-tuning method, device, electronic device and storage medium provided in the embodiments of the present application are suitable for generating training samples in the fine-tuning stage of a large model to perform corresponding fine-tuning optimization on the large model. Figure 1 The figure is a schematic diagram of the structure of the model fine-tuning system based on the embodiment of the present application, which mainly includes a model to be fine-tuned, a data acquisition device and a model fine-tuning device. The data acquisition device is used to collect knowledge documents in the target field and send the collected knowledge documents to the model fine-tuning device. The model fine-tuning device generates fine-tuning training samples based on the obtained knowledge documents, and finally fine-tunes and optimizes the model to be fine-tuned based on the fine-tuning training samples.

[0084] The embodiment of the present application provides a model fine-tuning method for generating training samples in the fine-tuning stage of a large model to perform corresponding fine-tuning optimization on the large model. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used for model fine-tuning.

[0085] like Figure 2 FIG. 1 is a flow chart of a model fine-tuning method provided in an embodiment of the present application, and the method includes:

[0086] Step 201, obtaining knowledge documents in the target domain.

[0087] Specifically, the target domain can be determined according to the specific application domain of the model to be fine-tuned, and then the knowledge documents of the target domain can be collected and obtained.

[0088] Step 202: extract question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document.

[0089] Specifically, we can first extract several question statements from the knowledge document, and then answer these question statements based on the knowledge information provided by the knowledge document to obtain answer statements, associate the corresponding question statements and answer statements to obtain question-answer pairs, and thus obtain a question-answer pair set consisting of several question-answer pairs.

[0090] Step 203, determining the quality score of each question-answer pair according to the text information of each question-answer pair in the question-answer pair set.

[0091] Specifically, the quality score of the question and answer pair can be determined based on information such as the fluency of the answer sentence, the accuracy of the answer, and the logic of the answer represented by the text information of the question and answer pair.

[0092] Step 204, based on the quality scores of each question-answer pair, select the target question-answer pair from the question-answer pair set, and use the target question-answer pair as a fine-tuning training sample for the model to be fine-tuned.

[0093] Specifically, according to the preset fine-tuning training requirements, based on the quality scores of each question and answer pair, target question and answer pairs whose quality scores meet the preset requirements can be screened in the question and answer pair set, and then the target question and answer pairs can be used as fine-tuning training samples for the model to be fine-tuned.

[0094] Step 205: fine-tune and optimize the model to be fine-tuned based on the fine-tuning training samples to obtain a target model.

[0095] It should be noted that the model to be fine-tuned in the embodiment of the present application is a large model with strong comprehension and generation capabilities. Before fine-tuning and optimization, a dataset file (dataset.json) containing all question statements Q_ij and answer statements A_ij, and the corresponding dataset description file (dataset_info.json) can be output according to the input and output key attributes specified by the model user and stored in the location specified by the model user for use in the model fine-tuning and optimization stage.

[0096] Specifically, the fine-tuning training samples obtained in the embodiment of the present application can be put into the original fine-tuning training process of the model to be fine-tuned, so as to fine-tune and optimize the model to be fine-tuned based on the fine-tuning training samples. Since the fine-tuning training samples obtained in the embodiment of the present application are of high quality, the target model obtained by fine-tuning the method provided in the embodiment of the present application will have higher performance.

[0097] Based on the above embodiment, as an implementable manner, in one embodiment, question-answer pairs are extracted from a knowledge document to obtain a set of question-answer pairs in the knowledge document, including:

[0098] Step 2021, splitting the knowledge document into natural segments to obtain a number of initial knowledge text segments;

[0099] Step 2022, judging whether each initial segment of the knowledge text is a segment to be merged according to the length of the initial segment of each knowledge text;

[0100] Step 2023, if any initial segment of the knowledge text is a segment to be merged, then merge the segment to be merged into the initial segment of the next knowledge text;

[0101] Step 2024, when the paragraph length of the initial knowledge text segment is not less than the preset condition value, the initial knowledge text segment is used as the knowledge text segment, so that the paragraph lengths of the several knowledge text segments finally obtained are not less than the preset condition value; wherein the paragraph length of the paragraph to be merged is less than the preset condition value;

[0102] Step 2025, for any knowledge text segment, obtain a question statement in the knowledge text segment to obtain a question statement corresponding to each knowledge text segment;

[0103] Step 2026, for any question statement, select a preset number of answer text segments corresponding to the question statement from a plurality of knowledge text segments;

[0104] Step 2027, taking a preset number of answer text segments as a knowledge set;

[0105] Step 2028, inputting the question statement and the knowledge set into the model to be fine-tuned, so as to answer the question statement based on the model to be fine-tuned according to the knowledge set, so as to generate a corresponding answer statement;

[0106] Step 2029, generating question-answer pairs based on each question statement and answer statement to obtain a set of question-answer pairs in the knowledge document.

[0107] It should be noted that if the length of the split paragraph is too small, the knowledge document will be too fragmented. If the split is performed according to a fixed paragraph length, the original longer natural paragraph in the knowledge document will be divided into multiple slices, which is not conducive to the extraction of subsequent question statements and answer statements. In order to solve this technical problem, the embodiment of the present application first splits the knowledge document into natural paragraphs. Each time an initial segment of the knowledge text is obtained by splitting, it is determined whether the paragraph length of the initial segment of the knowledge text is less than the preset condition value (rec_len). If so, it is used as a paragraph to be merged and merged into the next initial segment of the knowledge text. The pseudo code for knowledge document splitting provided in the embodiment of the present application is as follows:

[0108]

[0109] Among them, after the above document segmentation, multiple knowledge text segments (slices) are obtained, and their length may exceed the rec_len constraint; this design aims to find the shortest segment with a length longer than rec_len. Overall, the slice length is relatively long and the original natural segment is retained, which is more conducive to extracting questions and answers.

[0110] Specifically, for the extraction of question sentences, the model to be fine-tuned can be used to obtain the question sentences Q_ij contained in the corresponding knowledge text segment fi_i, where j is the preset question number hyperparameter, that is, j question sentences are obtained in each knowledge text segment.

[0111] Furthermore, for the answer to any question statement, 2k-1 (preset number) answer text segments can be extracted from all knowledge text segments as the associated document range, that is, the knowledge set corresponding to the question statement is obtained, k is the preset associated document range hyperparameter, and the pseudo code for extracting the answer text segment is as follows:

[0112] Kn=[];

[0113] for tmp in range(k):

[0114] Kn.append(fi[mod(tmp+k,size(fi))],fi[mod(tmp-k,size(fi))])

[0115] return Kn

[0116] Specifically, the embodiment of the present application screens the answer text segments by performing a modulo calculation on the index number of the knowledge text segment, that is, in addition to returning the knowledge text segment fi_i itself, it also returns 2k-2 more forward and backward. The remainder is taken to avoid over-limit. In this way, it is possible to obtain a segment that is far away from fi_i. The design is such that if the introduced data is relevant, the generation effect can be enhanced, and if it is irrelevant, it will not affect the generation effect. Among them, for the questions generated from the same segment fi_i, the knowledge set Kn_i selected at the same time should be reused.

[0117] Furthermore, the question statement Q_ij and the knowledge set Kn_i are input together into the large model to be fine-tuned to obtain the answer statement A_ij.

[0118] Based on the above embodiment, as an implementable manner, in one embodiment, the quality score of each question-answer pair is determined according to the text information of each question-answer pair in the question-answer pair set, including:

[0119] Step 2031, determining a perplexity score for each question-answer pair in the question-answer pair set according to the sentence coherence represented by the text information of each question-answer pair;

[0120] Step 2032, determining an interpretability score of each question-answer pair according to the question-answer quality represented by the text information of each question-answer pair in the question-answer pair set;

[0121] Step 2033, determining a word frequency score for each question-answer pair according to the word frequency features represented by the text information of each question-answer pair in the question-answer pair set;

[0122] Step 2034, determine the quality score of each question and answer pair based on the perplexity score, interpretability score and word frequency score of each question and answer pair.

[0123] It should be noted that the perplexity score is used to evaluate whether the answer statement in the question and answer pair is fluent, the explainability score is used to evaluate the accuracy, logic and proportion of redundant information of the answer statement, and the word frequency score is used to evaluate the degree of correlation between the answer statement and the knowledge set, that is, to rate the accuracy of the answer statement.

[0124] Specifically, in one embodiment, for the calculation of the perplexity score, for any question-answer pair in the question-answer pair set, the answer sentence in the question-answer pair can be split into words to convert the answer sentence into a word sequence of a target length; the perplexity of the answer sentence is determined based on the probability of occurrence of each word in the word sequence; and the perplexity score of the answer sentence is determined based on the perplexity of the answer sentence.

[0125] The text information of the question-answer pair includes the probability of each word appearing in the word sequence.

[0126] Specifically, the answer statement in the question-answer pair can be word-segmented according to the target length of the word sequence to convert the answer statement into a word sequence W = w1, w2, ..., w with a length of N. N .

[0127] Specifically, in one embodiment, the perplexity of the answer sentence may be determined based on the following formula:

[0128]

[0129] Among them, PP(W) represents the perplexity of the answer sentence, N represents the target length of the word sequence, and P(w i |w1,w2,…,w i-1 ) represents the probability of the i-th word appearing in the word sequence.

[0130] Among them, P(w i |w1,w2,…,w i-1) can be calculated by the large model to be trained based on the answer content. The higher the perplexity PP(W) of the answer sentence, the more incoherent the answer sentence is. The perplexity score Sc1 of the answer sentence is proportional to the perplexity PP(W).

[0131] Furthermore, for the calculation of the explainability score, the large model to be fine-tuned can be used to give an explainability score Sc2 for the answer content A_ij from the perspectives of accuracy, logic, and redundant information based on the semantic information contained in the question statement, answer statement, and knowledge set.

[0132] Among them, the accuracy of the answer content, that is, whether the information in the answer content is correct based on the given references; the logic of the answer content, that is, whether the answer is logically self-consistent and consistent with the question; the proportion of redundant information in the answer content, that is, the answer should not contain too much irrelevant content. Finally, an interpretability score Sc2 in the range of 0 to 10 is output. The higher the answer quality, the higher the interpretability score Sc2.

[0133] Furthermore, in one embodiment, for the calculation of word frequency scores, for any question-answer pair in the question-answer pair set, the answer sentence in the question-answer pair can be segmented to obtain the answer sentence segmentation result; the knowledge set corresponding to the answer sentence can be segmented to obtain the knowledge set segmentation result; wherein the answer sentence is generated based on the knowledge set; the knowledge document is segmented to obtain the knowledge document segmentation result; the answer sentence word frequency vector is determined based on the frequency of occurrence of each answer word represented by the answer sentence segmentation result in the answer sentence; the knowledge set segmentation result is used to determine the word frequency vector of the answer sentence; the knowledge set segmentation result is used to determine the frequency of occurrence of each knowledge word in the knowledge set represented by the knowledge set segmentation result frequency, determine the knowledge set word frequency vector; obtain the total number of knowledge text segments of the knowledge document, the number of first target knowledge text segments containing answer words, and the number of second target knowledge text segments containing knowledge words; determine the answer sentence weight vector according to the answer sentence word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of first target knowledge text segments containing answer words; determine the knowledge set weight vector according to the knowledge set word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of second target knowledge text segments containing knowledge words; determine the word frequency score of each question and answer pair according to the answer sentence weight vector and the knowledge set weight vector.

[0134] Specifically, the answer sentence A_ij, the knowledge set Kn_i and the knowledge document fi can be segmented based on the preset word segmentation library. For each word in A_ij and Kn_i, the frequency of the word in A_ij and Kn_i is calculated as tf. Calculating tf for A_ij and Kn_i will get two vectors, which are recorded as the answer sentence word frequency vector tf A And the knowledge set word frequency vector tf k .

[0135] Based on the following formula, the answer sentence weight vector V is calculated a :

[0136]

[0137] V at =tf A ×idf A

[0138] Among them, V at Represents the intermediate vector of the answer, V at1 ~V atn Indicates the n elements in the answer intermediate vector, idf A Represents the answer sentence feature vector, idf A Determined based on the following formula:

[0139]

[0140] Where D represents the total number of knowledge text segments in the knowledge document, D A It indicates the number of first target knowledge text segments containing answer words. The number of first target knowledge text segments containing answer words is determined according to the knowledge document segmentation result.

[0141] Similarly, the knowledge set weight vector V k The calculation formula is as follows:

[0142]

[0143] V kt =tf k ×idf k

[0144] Among them, V kt represents the intermediate vector of the knowledge set, V kt1 ~V ktn Represents the n elements in the intermediate vector of the knowledge set, idf k Represents the knowledge set feature vector, idf k Determined based on the following formula:

[0145]

[0146] Where D represents the total number of knowledge text segments in the knowledge document, D k It indicates the number of second target knowledge text segments containing knowledge words. The number of second target knowledge text segments containing knowledge words is also determined according to the knowledge document segmentation result.

[0147] Specifically, in one embodiment, the word frequency score of each question-answer pair may be determined based on the following formula:

[0148]

[0149] Among them, Sc3 is the word frequency score of the question and answer pair, Va i Vk represents the answer sentence weight vector of the i-th question-answer pair in the question-answer pair set, i represents the knowledge set weight vector corresponding to the i-th question-answer pair, and n represents the total number of question-answer pairs in the question-answer pair set.

[0150] It should be noted that Va i and Vk i Actually it is the TF-IDF weight vector, Sc3 is actually Va i and Vk i The cosine similarity ranges from [-1,1]. The larger the value, the more similar they are. The more similar they are, the higher the quality of the answer.

[0151] Furthermore, in one embodiment, the perplexity score, interpretability score, and word frequency score of the question-answer pair may be integrated based on the following formula to obtain a quality score of the question-answer pair:

[0152]

[0153] Wherein, Sc represents the quality score of the question-answer pair, and the full score of Sc is T, for example, T = 10. The embodiment of the present application can finally output a data result score file (score.jsonl) containing Q_ij, A_ij, Sc1, Sc2, Sc3 and Sc for model users to consult and use.

[0154] For example, Figure 3 As shown, it is a schematic diagram of the overall process of the model fine-tuning method provided in the embodiment of the present application, such as Figure 3 The method shown is as follows Figure 2 The method shown is an exemplary implementation method, and the implementation principles of the two are the same, which will not be repeated. The entire process can be encapsulated as a container I, calling the external large model service API to achieve parallel data generation.

[0155] Specifically, multiple instances of container I can be started to fine-tune data generation for different document sets or the same document set; through container scheduling platforms such as K8S, docker, AIStation, etc., parallel fine-tuning data generation task creation and execution can be supported. Furthermore, through the container scheduling platform and the large model service platform, load balancing of large model reasoning services can be achieved (based on the container scheduling platform, Kubernetes' affinity schedule mechanism or approximate mechanism, pod-level load balancing and multi-instance construction), which can further improve the efficiency of parallel fine-tuning data generation tasks. From the perspective of the supporting system, it is necessary to store input documents, storage devices for output data sets, processors for running algorithms, caches for storing temporary data, and related computer equipment that provides large model services.

[0156] The model fine-tuning method provided in the embodiment of the present application obtains knowledge documents in the target domain; extracts question-answer pairs from the knowledge documents to obtain a set of question-answer pairs in the knowledge documents; determines the quality score of each question-answer pair based on the text information of each question-answer pair in the set of question-answer pairs; screens the target question-answer pairs in the set of question-answer pairs based on the quality scores of each question-answer pair, and uses the target question-answer pairs as fine-tuning training samples for the model to be fine-tuned; based on the fine-tuning training samples, fine-tunes and optimizes the model to be fine-tuned to obtain the target model. The method provided by the above scheme improves the quality of the fine-tuning training samples by classifying the knowledge documents by domain, and screens the target question-answer pairs that can be used as fine-tuning training samples based on the quality scores of the question-answer pairs, thereby avoiding deviations in the model during the fine-tuning process, thereby improving the fine-tuning accuracy of the model. In addition, by extracting the problem part of the fine-tuning data in the form of a summary after segmenting the document, irrelevant problems that may arise when a large model generates question-answer pair training data are avoided, and the accuracy of sample data generation can be effectively improved.

[0157] An embodiment of the present application provides a model fine-tuning device for executing the model fine-tuning method provided in the above embodiment.

[0158] like Figure 4 , which is a schematic diagram of the structure of a model fine-tuning device provided in an embodiment of the present application. The model fine-tuning device 40 comprises: an acquisition module 401 , an extraction module 402 , a scoring module 403 , a screening module 404 and a fine-tuning module 405 .

[0159] Among them, the acquisition module is used to acquire knowledge documents in the target field; the extraction module is used to extract question-answer pairs in the knowledge documents to obtain a question-answer pair set in the knowledge documents; the scoring module is used to determine the quality score of each question-answer pair based on the text information of each question-answer pair in the question-answer pair set; the screening module is used to screen the target question-answer pairs in the question-answer pair set according to the quality scores of each question-answer pair, and use the target question-answer pairs as fine-tuning training samples of the model to be fine-tuned; the fine-tuning module is used to fine-tune and optimize the model to be fine-tuned based on the fine-tuning training samples to obtain the target model.

[0160] For example, as another logical function division of the model fine-tuning device, Figure 5 As shown, it is a schematic diagram of the structure of an exemplary model fine-tuning device provided in an embodiment of the present application, and the device includes: a segmentation module, a question extraction module, a fragment selection module, a large model question and answer module, a perplexity calculation module, an explainability scoring module, a word frequency scoring module and a result integration output module.

[0161] Regarding the model fine-tuning device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0162] The model fine-tuning device provided in the embodiment of the present application is used to execute the model fine-tuning method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0163] An embodiment of the present application provides an electronic device for executing the model fine-tuning method provided in the above embodiment.

[0164] like Figure 6 FIG. 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 60 includes: at least one processor 61 and a memory 62 .

[0165] The memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the model fine-tuning method provided in the above embodiment.

[0166] The electronic device provided in the embodiment of the present application is used to execute the model fine-tuning method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0167] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the model fine-tuning method provided in any of the above embodiments is implemented.

[0168] The storage medium containing computer executable instructions provided in the embodiment of the present application can be used to store computer executable instructions of the model fine-tuning method provided in the aforementioned embodiment. Its implementation method and principle are the same and will not be repeated here.

[0169] An embodiment of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the model fine-tuning method provided in the aforementioned embodiment.

[0170] The embodiments of the present application provide a computer program product that can be used to execute computer instructions of the model fine-tuning method provided in the aforementioned embodiments. The implementation method and principle are the same and will not be repeated here.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0174] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0175] A part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0176] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A model fine-tuning method, characterized in that: include: Acquire knowledge documents in the target domain; Extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document; Determining a quality score of each question-answer pair according to text information of each question-answer pair in the question-answer pair set; According to the quality scores of each question-answer pair, a target question-answer pair is selected from the question-answer pair set, and the target question-answer pair is used as a fine-tuning training sample of the model to be fine-tuned; Based on the fine-tuning training samples, the model to be fine-tuned is fine-tuned and optimized to obtain a target model.

2. The method according to claim 1, characterized in that The step of extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document includes: Splitting the knowledge document into natural segments to obtain a number of initial knowledge text segments; According to the paragraph length of each initial paragraph of the knowledge text, determining whether each initial paragraph of the knowledge text is a paragraph to be merged; If any of the initial sections of the knowledge text is a section to be merged, then the section to be merged is merged into the next initial section of the knowledge text; In the case where the paragraph length of the initial segment of the knowledge text is not less than the preset condition value, the initial segment of the knowledge text is used as the knowledge text segment, so that the paragraph lengths of the several knowledge text segments finally obtained are not less than the preset condition value; wherein the paragraph length of the paragraph to be merged is less than the preset condition value; For any of the knowledge text segments, obtaining a question statement in the knowledge text segment to obtain a question statement corresponding to each knowledge text segment; For any of the question statements, a preset number of answer text segments corresponding to the question statement are selected from the plurality of knowledge text segments; taking the preset number of answer text segments as a knowledge set; Inputting the question statement and the knowledge set into the model to be fine-tuned, so as to answer the question statement based on the model to be fine-tuned according to the knowledge set, so as to generate a corresponding answer statement; A question-answer pair is generated according to each of the question statements and the answer statements to obtain a set of question-answer pairs in the knowledge document.

3. The method according to claim 1, characterized in that Determining the quality score of each question-answer pair according to the text information of each question-answer pair in the question-answer pair set includes: Determining a perplexity score for each question-answer pair in the question-answer pair set according to the sentence coherence represented by the text information of each question-answer pair; Determining an explainability score for each question-answer pair according to the question-answer quality represented by the text information of each question-answer pair in the question-answer pair set; Determining a word frequency score for each question-answer pair in the question-answer pair set according to a word frequency feature represented by the text information of each question-answer pair; The quality score of each question-answer pair is determined based on the perplexity score, interpretability score and word frequency score of each question-answer pair.

4. The method according to claim 3, characterized in that Determining the perplexity score of each question-answer pair according to the sentence coherence represented by the text information of each question-answer pair in the question-answer pair set includes: For any question-answer pair in the question-answer pair set, performing word splitting on an answer sentence in the question-answer pair to convert the answer sentence into a word sequence of a target length; Determining the perplexity of the answer statement according to the probability of occurrence of each word in the word sequence; Determine a perplexity score of the answer sentence according to the perplexity of the answer sentence; The text information of the question-answer pair includes the probability of occurrence of each word in the word sequence.

5. The method according to claim 4, characterized in that Determining the perplexity of the answer statement according to the probability of occurrence of each word in the word sequence includes: Based on the following formula, the confusion of the answer statement is determined: Where PP(W) represents the perplexity of the answer sentence, N represents the target length of the word sequence, and P(w i |w1,w2,…,w i-1 ) represents the probability of the i-th word appearing in the word sequence.

6. The method according to claim 3, characterized in that Determining the word frequency score of each question-answer pair according to the word frequency feature represented by the text information of each question-answer pair in the question-answer pair set includes: For any question-answer pair in the question-answer pair set, performing word segmentation processing on an answer sentence in the question-answer pair to obtain a word segmentation result of the answer sentence; Performing word segmentation processing on the knowledge set corresponding to the answer sentence to obtain a knowledge set word segmentation result; wherein the answer sentence is generated based on the knowledge set; Performing word segmentation processing on the knowledge document to obtain a knowledge document word segmentation result; Determine a word frequency vector of the answer sentence according to the frequency of occurrence of each answer word represented by the word segmentation result of the answer sentence in the answer sentence; Determine a knowledge set word frequency vector according to the occurrence frequency of each knowledge word represented by the knowledge set word segmentation result in the knowledge set; Obtaining the total number of knowledge text segments of the knowledge document, the number of first target knowledge text segments containing answer words, and the number of second target knowledge text segments containing knowledge words; Determine an answer sentence weight vector according to the answer sentence word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of first target knowledge text segments containing answer words; Determining a knowledge set weight vector according to the knowledge set word frequency vector, the total number of knowledge text segments of the knowledge document, and the number of second target knowledge text segments containing knowledge words; The word frequency score of each question-answer pair is determined based on the answer sentence weight vector and the knowledge set weight vector.

7. The method according to claim 6, characterized in that Determining the word frequency score of each question-answer pair according to the answer sentence weight vector and the knowledge set weight vector includes: The word frequency score of each question-answer pair is determined based on the following formula: Among them, Sc3 is the word frequency score of the question and answer pair, Va i Vk represents the answer sentence weight vector of the i-th question-answer pair in the question-answer pair set, i represents the knowledge set weight vector corresponding to the i-th question-answer pair, and n represents the total number of question-answer pairs in the question-answer pair set.

8. A model fine-tuning device, characterized in that: include: The acquisition module is used to obtain knowledge documents in the target field; An extraction module, used for extracting question-answer pairs from the knowledge document to obtain a set of question-answer pairs in the knowledge document; A scoring module, used to determine a quality score of each question-answer pair according to the text information of each question-answer pair in the question-answer pair set; A screening module, used to screen target question-answer pairs from the question-answer pair set according to the quality scores of each question-answer pair, and use the target question-answer pairs as fine-tuning training samples for the model to be fine-tuned; The fine-tuning module is used to fine-tune and optimize the model to be fine-tuned based on the fine-tuning training samples to obtain a target model.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 7 is implemented.

Images