Method, device and equipment for obtaining fine adjustment data of large model and storage medium

Abstract

The invention discloses a large model fine tuning data acquisition method and device, equipment and a storage medium. The method comprises the following steps: generating an input instruction based on an application field, a training question, an original training data document and a document cue word of a to-be-adjusted model, and inputting the input instruction into a language generation model with a reflection mechanism to generate a plurality of question and answer results; wherein the document prompt word is used for indicating the generation requirement. Then, candidate results are determined from the question and answer results, and target vector clauses are extracted from the original training data document; and by calculating the similarity between the candidate result and the target vector clause and carrying out threshold judgment, screening out a target result meeting a condition. And finally, combining the target result and the training question into a question-answer pair to form large model fine tuning data. The answer generation is guided through the document cue word, the multiple question and answer results are generated in combination with the reflection mechanism, the optimal answer is screened through similarity matching, and therefore it is ensured that the question and answer result serving as the target result is comprehensive and accurate and meets the requirement.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method, device, equipment and storage medium for obtaining large model fine-tuning data. Background Art

[0002] In order to improve the application effect of the big model for specific financial scenarios (such as customer service assistant, wealth assistant, operation and maintenance assistant, credit assistant, compliance assistant, etc.), it is usually necessary to fine-tune the big financial model. Before this, a certain amount of fine-tuning data (usually question-answer pairs) needs to be prepared.

[0003] In order to obtain fine-tuning data, the existing technical solution is to directly input unstructured original documents in the financial field into the open source big model, and then ask questions to the open source big model to obtain generated answers. Although this method can quickly obtain financial knowledge answer data, when it comes to complex and comprehensive financial foreign regulatory knowledge fields, the answer data generated by the open source big model often encounters incomplete answers. For example, suppose that the big model is required to generate answer data about "the main role of financial institutions in promoting event A" based on financial knowledge. In theory, a complete answer should include multiple aspects, such as institutional settings, service focus, market positioning, etc. However, in the actual generated data, only a detailed answer about "institutional settings" may be obtained, while other important aspects are ignored. As a result, the fine-tuning data used for fine-tuning the financial big model has the problem of incomplete information, which affects the fine-tuning effect and final application performance of the financial big model.

[0004] Therefore, how to improve the integrity of fine-tuning data to enhance the application effect and performance of large financial models in complex financial scenarios is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0005] Based on the above problems, the present application provides a method, device, equipment and storage medium for obtaining fine-tuning data of a large model, which can improve the integrity of the fine-tuning data to enhance the application effect and performance of the financial large model in complex financial scenarios.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] A method for obtaining large model fine-tuning data, the method comprising:

[0008] Generate input instructions based on the application field of the model to be adjusted, the training problem, the original training data document and the document prompt words of the training problem, and input the input instructions into the language generation model with a reflection mechanism to obtain multiple question and answer results of the training problem; the document prompt words are used to indicate the generation requirements of the question and answer results; the question and answer results are the answers to the training problems;

[0009] Determine a candidate result from the plurality of question-answer results, and obtain a target vector sentence of the candidate result from the plurality of original sentences of the original training data document;

[0010] Calculating the similarity between the candidate result and the target vector sentence to obtain a first similarity;

[0011] If the first similarity is greater than or equal to a first similarity threshold, determining the candidate result as a target result;

[0012] The target result and the training question are combined into a question-answer pair to obtain large model fine-tuning data.

[0013] In a possible implementation, the step of obtaining the target vector sentence of the candidate result from the plurality of original sentences of the original training data document includes:

[0014] Each original sentence of the original training data document is vectorized and segmented to obtain a plurality of vectorized sentences;

[0015] Calculate similarity between the candidate result and the multiple vector sentences respectively to obtain multiple second similarities;

[0016] Based on the screening condition and the plurality of second similarities, determining N candidate vector clauses from the plurality of vector clauses; N is an integer greater than 1;

[0017] A reranker model is used to determine a target vector sentence from the N candidate vector sentences.

[0018] In a possible implementation, the determining a target vector sentence from the N candidate vector sentences using a reranker model includes:

[0019] Using the reranker model, respectively calculate the similarity between the candidate result and the N candidate vector sentences to obtain N third similarities;

[0020] Based on the N third similarities, the target vector sentence is screened out from the N candidate vector sentences; the third similarity corresponding to the target vector sentence is the largest among the N third similarities.

[0021] In a possible implementation, each original sentence of the original training data document is vectorized and segmented to obtain a plurality of vectorized sentences, including:

[0022] The original training data document is converted into a vector representation using a text embedded adaptive model 002text-embedding-ada-002 to obtain a vector document;

[0023] defining a sliding window of a fixed size, and using the sliding window to segment the vector document into a plurality of text blocks;

[0024] For any adjacent text blocks, calculate the semantic similarity between the adjacent text blocks;

[0025] Adjacent text blocks whose semantic similarity is greater than or equal to a second similarity threshold are merged to obtain the vector sentence.

[0026] In a possible implementation, the generating of the input instruction based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt word of the training problem includes:

[0027] Determining the application field of the model to be adjusted, the training problem, and the document prompt words of the training problem;

[0028] Acquire the original training data document according to the application field of the model to be adjusted;

[0029] An input instruction is generated based on the training question, the document prompt word, and the original training data document.

[0030] In a possible implementation, the method further includes:

[0031] If the first similarity is less than the first similarity threshold, the candidate result is marked as unavailable, and a new candidate result is re-determined from the multiple question and answer results to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

[0032] In a possible implementation, the method further includes:

[0033] If no candidate result whose first similarity is greater than or equal to the first similarity threshold is obtained after traversing multiple question and answer results, the multiple question and answer results are adjusted through the thinking chain to obtain multiple new question and answer results, and the candidate results are re-determined from the multiple new question and answer results to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

[0034] A device for acquiring large model fine-tuning data, the device comprising:

[0035] A first input instruction generating unit, configured to generate an input instruction based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt word of the training problem;

[0036] An instruction input unit, used for inputting the input instruction into a language generation model with a reflection mechanism to obtain a plurality of question and answer results of the training question; the document prompt word is used to indicate the generation requirements of the question and answer result; the question and answer result is the answer to the training question;

[0037] A candidate result determination unit, used to determine a candidate result from a plurality of the question and answer results;

[0038] A target vector sentence acquisition unit, configured to acquire the target vector sentence of the candidate result from a plurality of original sentences in the original training data document;

[0039] A first calculation unit, configured to calculate a similarity between the candidate result and the target vector sentence to obtain a first similarity;

[0040] a target result determining unit, configured to determine the candidate result as a target result if the first similarity is greater than or equal to a first similarity threshold;

[0041] A combination unit is used to combine the target result and the training question into a question-answer pair to obtain large model fine-tuning data.

[0042] A device for acquiring large model fine-tuning data includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for acquiring large model fine-tuning data as described above is implemented.

[0043] A computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method for obtaining large model fine-tuning data as described above.

[0044] Compared with the prior art, this application has the following beneficial effects:

[0045] The present application provides a method, device, equipment and storage medium for obtaining large model fine-tuning data. Specifically, when executing the method for obtaining large model fine-tuning data provided in the embodiment of the present application, first, based on the application field of the model to be adjusted, the training problem, the original training data document and the document prompt words related to the training problem, an input instruction is generated. This input instruction is input into a language generation model with a reflection mechanism to generate multiple possible question and answer results as answers to the training questions. Among them, the document prompt words are used to clearly guide the generation requirements of the question and answer results. Next, the target vector clauses are extracted from multiple sentences in the original training data document, and the candidate answers are screened out from the generated question and answer results. By calculating the similarity between the candidate answer and the target vector clause, a first similarity is obtained. If the similarity is greater than or equal to the set threshold, the candidate result is determined as the target answer. Finally, the target answer is combined with the training question into a question and answer pair to constitute the final large model fine-tuning data. This application uses document prompts to indicate the generation requirements of question-answering results, which can ensure that the generated answers cover multiple aspects. At the same time, multiple question-answering results are generated based on a language generation model with a reflection mechanism, which helps to think deeply and integrate various aspects of information during the generation of question-answering results, thereby improving the completeness and accuracy of the question-answering results. In addition, by calculating the similarity between the candidate results and the target vector sentence to determine the target result, this matching mechanism can help screen out the best question-answering results that meet the requirements and avoid the selection of incomplete or irrelevant answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A method flow chart of a method for obtaining large model fine-tuning data provided in an embodiment of the present application;

[0048] Figure 2 A method flow chart of a method for generating an input instruction provided in an embodiment of the present application;

[0049] Figure 3 A method flow chart of a target vector sentence determination method provided in an embodiment of the present application;

[0050] Figure 4 A method flow chart of a vector sentence acquisition method provided in an embodiment of the present application;

[0051] Figure 5A schematic diagram of the structure of a device for acquiring fine-tuning data of a large model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0053] In order to improve the application effect of large models in specific financial scenarios, it is usually necessary to fine-tune these large models. The preparation work before fine-tuning mainly includes preparing a certain scale of fine-tuning data, which usually exists in the form of question-answer pairs. The relevant technical solution is to obtain these fine-tuning data by inputting unstructured original documents in the financial field into the open source large model and obtaining the generated answers by asking questions. Although this method can quickly obtain financial knowledge answer data, when dealing with complex financial foreign regulatory knowledge fields that require comprehensive coverage, the generated answer data often has incomplete answers. This incomplete information leads to problems with the fine-tuning data used for fine-tuning, which in turn affects the performance and application performance of the fine-tuned financial large model.

[0054] In order to solve this problem, a method, device, equipment and storage medium for obtaining fine-tuning data for a large model are provided in an embodiment of the present application. First, input instructions are generated based on the application field of the model to be adjusted, the training problem, the original training data document and the document prompt words of the training problem, and multiple question and answer results are generated in combination with a language generation model with a reflection mechanism. Then, the document prompt words are used to indicate the generation requirements of the question and answer results to ensure that the generated answers meet the requirements. Further, the target vector sentence is extracted from the original training data document and the candidate results are determined from multiple question and answer results, and the first similarity is obtained by calculating the similarity between the candidate results and the target vector sentence. If the first similarity is higher than or equal to the set threshold, the candidate result is confirmed as the target result, and the target result is finally combined with the training problem to form a question and answer pair as data for model adjustment. The present application uses document prompt words to indicate the generation requirements of the question and answer results, and generates multiple question and answer results in combination with a language generation model with a reflection mechanism, so that in the process of generating the question and answer results, in-depth thinking and comprehensive information from all aspects are carried out to ensure that the answers cover many aspects and improve completeness and accuracy. In addition, by calculating the similarity between the candidate results and the target vector sentence to determine the target result, the best question-answering result can be effectively screened, avoiding the selection of incomplete or irrelevant answers, and improving the quality and relevance of the question-answering results.

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] See also Figure 1 , which is a flow chart of a method for obtaining large model fine-tuning data provided by an embodiment of the present application, such as Figure 1 As shown, the method for obtaining the large model fine-tuning data may include steps S101-S105:

[0057] S101: Generate input instructions based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt words of the training problem, and input the input instructions into a language generation model with a reflection mechanism to obtain multiple question and answer results of the training problem.

[0058] To obtain data for fine-tuning a large model, you first need to generate input instructions based on the application domain of the model to be adjusted, the content of the training problem, the original training data document, and prompt information related to the training problem. These instructions will be sent to a language generation model with a reflection mechanism, and the model will generate multiple possible question and answer results to answer the training question. The document prompt words play a guiding role in this process, clarifying the requirements that need to be followed when generating question and answer results, and these generated question and answer results are the answers to the training problems.

[0059] For example, assume that the training question is "What is the main role of financial institutions in event A?"; the document prompt is "explain in combination with institutional settings, service focus and market positioning"; the original training data is "x", then the input instruction can be: find the answer to "What is the main role of financial institutions in event A?" in "x", and output it in the form of "explain in combination with institutional settings, service focus and market positioning".

[0060] Assume that the training question is "What is diabetes?"; the document prompt words are "definition of diabetes" and "symptoms of diabetes"; the original training data is "y", then the input instruction can be: find the answer to "What is diabetes?" in "y" and output it in the form of "definition of diabetes" and "symptoms of diabetes".

[0061] It should be noted that the reflection mechanism of the model refers to the ability of the model to continuously review and evaluate the results produced through internal introspection and logical reasoning when generating output. This mechanism enables the model to think and analyze problems more deeply, improve its understanding and processing of input information, and thus improve the quality and accuracy of the generated results. Through the reflection mechanism, the model can evaluate the previous decision when generating each answer or result, consider various possibilities, and make more reasonable and contextual outputs. This mechanism helps the model understand and answer complex questions more comprehensively and accurately.

[0062] For example, suppose the user inputs the training question "Who is the president of Country A?" The model first generates the answer: "Zhang San". Then, the model evaluates this answer and finds that it is too brief and does not mention Zhang San's importance in the history of Country A. Next, the model automatically adds more details, such as "Zhang San is the first president of Country A. He was appointed president in the late period of the War of Independence of City A and took office in year z."

[0063] During this process, the model identified the brevity of the original answer through a "self-assessment" mechanism (i.e., reflection mechanism) and automatically generated a more detailed and complete answer.

[0064] This approach combines application domains, training questions, original data, and document prompts to ensure that the multiple answers generated not only cover all aspects of the training questions, but also further optimize the quality and completeness of these answers through a reflection mechanism.

[0065] In one possible implementation, the language generation model may be, but is not limited to, Generative Pre-trained Transformer 4 (GPT4).

[0066] See also Figure 2 , Figure 2 A method flow chart of an input instruction generation method provided in an embodiment of the present application. Accordingly, step S101 generates an input instruction based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt word of the training problem, which can be specifically implemented by steps A1-A3:

[0067] A1: Determine the application field of the model to be adjusted, the training problem, and the document prompt words of the training problem.

[0068] Determining the application domain, training problem, and document prompts for the model to be adjusted means that before fine-tuning the model, it is necessary to clarify the specific domain in which the model will be applied, the specific problem it is targeting, and the required document prompts. This helps ensure that the fine-tuned model can better adapt to the needs of the specific domain and generate expected results.

[0069] Application domain refers to the scenario or task type that the user expects the model to be applied to. For example, the model can be applied to fields such as medicine, finance, and education. The purpose of determining the application domain is to focus on specific tasks, making the subsequent data collection, model design, and training directions clearer.

[0070] The training problem is the question that the model needs to solve or answer. For example, in the field of medical diagnosis, the training question might be "How to diagnose heart disease?"

[0071] Document prompts are a set of guiding words or phrases that are used to ensure that the generated answers cover multiple aspects and meet specific generation requirements. For example, for the question "How to diagnose heart disease?", document prompts may include "symptoms", "examination methods", "treatment recommendations", etc. to ensure that the generated answers cover these aspects.

[0072] By clearly defining these key information, the fine-tuning process can be effectively guided so that the adjusted model better meets the needs of specific domains and generates more accurate and appropriate output results.

[0073] A2: Acquire the original training data document according to the application field of the model to be adjusted.

[0074] Obtaining original training data documents according to the application domain of the model to be adjusted means collecting original data from data sources related to the specific domain where the model will be applied. These original training data documents contain information that the model needs to learn and refer to during the fine-tuning process, which helps to improve the performance and adaptability of the model on tasks in specific domains. By utilizing the original data related to the application domain, it can ensure that the model can better understand, learn and capture the characteristics and laws of the domain during the fine-tuning process, thereby improving the effectiveness and accuracy of the model in practical applications. This process is a key step in fine-tuning the model, which aims to provide sufficient and effective data support for the model to meet specific domain needs and optimize model performance.

[0075] A3: Generate input instructions based on the training question, the document prompt word and the original training data document.

[0076] Generating input instructions based on training questions, document prompts, and original training data documents means combining given training questions, prompts, and original data to create specific guidance instructions suitable for model operation or training, thereby guiding the model on how to learn from the data and complete the task.

[0077] S102: Determine candidate results from the multiple question-answer results, and obtain target vector sentences of the candidate results from the multiple original sentences of the original training data document.

[0078] The system selects one from multiple possible question-and-answer results as a candidate result. The "determined candidate result" here means that the system randomly selects a question-and-answer result as a candidate answer from multiple question-and-answer results, or selects a question-and-answer result as a candidate answer based on specific rules or ranking order. At the same time, the system also needs to convert multiple sentences in the original training data document into multiple vector sentences, and determine the target vector sentence from the multiple vector sentences. The target vector sentence is a mathematical representation that can represent the original sentence information after vectorization processing.

[0079] join Figure 3 , Figure 3 A method flow chart of a method for determining a target vector sentence provided in an embodiment of the present application. Accordingly, in step S102, obtaining the target vector sentence of the candidate result from the multiple original sentences of the original training data document can be specifically implemented through steps B1-B4:

[0080] B1: Each original sentence in the original training data document is vectorized and segmented to obtain a plurality of vectorized sentences.

[0081] Vectorizing and segmenting each original sentence of the original training data document means converting each original sentence into a vector representation and segmenting it to generate multiple vector sentences.

[0082] Vectorization refers to converting each original sentence into a numerical vector form that can be understood by a computer so that the model can process and analyze this text information.

[0083] Sentence processing refers to segmenting each vectorized sentence according to semantics or structure to generate multiple independent vector sentences. This helps the model better understand and process each part of the sentence and improves the performance of the model in the task.

[0084] By vectorizing the original sentences and performing sentence processing, the text data can be effectively converted into a machine-operable form so that the model can learn and understand semantic information more accurately, thereby improving its performance and efficiency in the training or application stage.

[0085] join Figure 4 , Figure 4 A method flow chart of a method for obtaining vector sentences provided in an embodiment of the present application. Accordingly, step B1 vectorizes and sentences each original sentence of the original training data document to obtain multiple vector sentences. Specifically, you can implement it through steps C1-C4:

[0086] C1: Use text-embedding-ada-002 to convert the original training data document into a vector representation to obtain a vector document.

[0087] Text-embedding-ada-002 is text embedded adaptive model 002 (text-embedding-ada-002).

[0088] The text-embedding-ada-002 model can be used to convert the original training data document into a vector representation. The "vector representation" here means that the text content is converted into a vector in a high-dimensional space through the model, so that semantically similar texts are close in the vector space, which is convenient for subsequent processing and analysis. Through this step, a "vector document" containing multiple text fragments can be obtained.

[0089] C2: Define a sliding window of a fixed size, and use the sliding window to segment the vector document into multiple text blocks.

[0090] Define a fixed-size sliding window and use it to split the entire "vector document" into multiple small text blocks. The size of the sliding window determines the content range of each text block. This step splits the vector document, which may have been a long text, into multiple relatively independent fragments, providing a basis for subsequent similarity calculations.

[0091] C3: For any adjacent text blocks, calculate the semantic similarity between adjacent text blocks.

[0092] For each pair of adjacent text blocks (such as text block 1 and text block 2, text block 2 and text block 3, etc.), the semantic similarity between them is calculated. Semantic similarity measures the semantic similarity between two text blocks by calculating the similarity between the corresponding vectors of the text blocks (such as cosine similarity, etc.).

[0093] C4: merging adjacent text blocks whose semantic similarity is greater than or equal to a second similarity threshold to obtain the vector sentence.

[0094] If the semantic similarity between two adjacent text blocks is greater than or equal to a preset second similarity threshold, the two text blocks are merged into a "vector sentence". This merging process is determined based on the semantic relationship between the text blocks, aiming to merge semantically closely related text content into a more coherent semantic unit.

[0095] It should be noted that the user can set the size of the second similarity threshold according to actual needs, and this application does not make any specific restrictions on the size of the second similarity threshold.

[0096] For example, suppose there are 10 adjacent text blocks, numbered 1 to 10. It is necessary to decide which text blocks can be merged based on the semantic similarity between them. Assume that the second similarity threshold is 0.85, and the similarity between adjacent text blocks is calculated as follows:

[0097] The similarity between text block 1 and text block 2 is 0.9;

[0098] The similarity between text block 2 and text block 3 is 0.88;

[0099] The similarity between text block 3 and text block 4 is 0.42;

[0100] The similarity between text block 4 and text block 5 is 0.92;

[0101] The similarity between text block 5 and text block 6 is 0.87;

[0102] The similarity between text block 6 and text block 7 is 0.89;

[0103] The similarity between text block 7 and text block 8 is 0.31;

[0104] The similarity between text block 8 and text block 9 is 0.91;

[0105] The similarity between text block 9 and text block 10 is 0.86.

[0106] Based on these similarity values, the merge can be performed as follows:

[0107] Combine 1, 2, and 3:

[0108] The similarity between text block 1 and text block 2 is 0.9 (greater than 0.85);

[0109] The similarity between text block 2 and text block 3 is 0.88 (greater than 0.85);

[0110] Therefore, text blocks 1, 2, and 3 are merged into a new vector sentence.

[0111] Combine 4, 5, 6, and 7:

[0112] The similarity between text block 4 and text block 5 is 0.92 (greater than 0.85);

[0113] The similarity between text block 5 and text block 6 is 0.87 (greater than 0.85);

[0114] The similarity between text block 6 and text block 7 is 0.89 (greater than 0.85);

[0115] Therefore, text blocks 4, 5, 6, and 7 are merged into a new vector sentence.

[0116] Combine 8, 9, and 10:

[0117] The similarity between text block 8 and text block 9 is 0.91 (greater than 0.85);

[0118] The similarity between text block 9 and text block 10 is 0.86 (greater than 0.85);

[0119] Therefore, text blocks 8, 9, and 10 are merged into a new vector sentence.

[0120] The result after merging:

[0121] Vector sentence 1: It is formed by merging text blocks 1, 2, and 3;

[0122] Vector sentence 2: It is formed by merging text blocks 4, 5, 6, and 7;

[0123] Vector sentence 3: It is formed by merging text blocks 8, 9, and 10.

[0124] The goal of steps C1-C4 is to segment a long text into multiple small blocks through a sliding window, and by calculating the semantic similarity of adjacent text blocks, merge those text blocks with similar semantics, and finally obtain "vector sentences". These "vector sentences" are vector representations of larger semantic units, which can express information more accurately in semantic analysis, information retrieval or question-answering systems.

[0125] In a possible implementation, a sequence model may be used to vectorize and segment each original sentence in the original training data document to obtain a plurality of vectorized sentences.

[0126] B2: Calculate similarity between the candidate result and the multiple vector sentences respectively to obtain multiple second similarities.

[0127] First, a candidate result is vectorized, and then a vectorized candidate result is compared with multiple vectorized sentences, and a new set of similarity values ​​is obtained by calculating the similarity between them. By calculating the similarity between the candidate result and these vectorized sentences, multiple "second similarity" values ​​can be obtained. These values ​​represent the degree of relevance of each candidate result to each vectorized sentence. Through similarity calculation, the matching degree between the candidate result and multiple vectorized sentences is quantified, thereby obtaining a set of similarity scores as the basis for subsequent sorting and screening.

[0128] For example, assuming there are three vector clauses and one candidate result, the calculation process can be expressed as:

[0129] The candidate result is similar to the vector sentence 1 to obtain the second similarity 1;

[0130] The candidate result is similar to the vector sentence 2 to obtain the second similarity 2;

[0131] The candidate result is similar to the vector sentence 3 to obtain a second similarity 3.

[0132] Finally, three second similarities are obtained, which respectively reflect the matching degree between the candidate result and each vector sentence.

[0133] B3: Determine N candidate vector sentences from the multiple vector sentences based on the screening condition and the multiple second similarities.

[0134] According to some preset screening criteria (such as sorting by similarity from high to low, selecting the top 5 most relevant sentences), and the multiple similarity values ​​calculated in the previous step (i.e., the second similarity), select N most relevant sentences from multiple vectorized sentences as candidates. Here N is an integer greater than 1, indicating the number of sentences finally selected from multiple candidate sentences.

[0135] For example, suppose there are 10 candidate vectorized sentences, and the screening criteria are: sort by similarity from high to low, and select the top 5 most relevant sentences. After screening, the 5 sentences with the highest similarity will be obtained, which will be used as the final candidate sentences for further processing (such as re-ranking or selecting the final answer).

[0136] This step can effectively narrow down the range of sentences to choose from, making subsequent processing more focused and efficient. The selected candidate sentences will be used for further analysis, sorting or other processing to meet the needs of specific tasks or goals.

[0137] In a possible implementation, step B3 can be implemented by, but not limited to, using a Langchain framework to determine N candidate vector sentences from multiple vector sentences based on the screening condition and the multiple second similarities.

[0138] The LangChain framework is an open source library for building language model applications.

[0139] B4: Determine the target vector sentence from the N candidate vector sentences using the reranker model.

[0140] After N candidate vectorized sentences are obtained through screening in the previous step, a reranking model (i.e., reranker model) is used to further rank these candidate sentences to determine the final target vector sentence.

[0141] The specific process is as follows:

[0142] First, input the candidate results and N candidate vector sentences: input the candidate results and N candidate vector sentences into the reranker model.

[0143] Then calculate the third similarity: the reranker model calculates the similarity between the candidate result and each candidate vector sentence respectively, and generates N third similarity values.

[0144] Finally, the target vector sentence is determined: according to the N third similarity values, the candidate vector sentence with the largest similarity value is selected as the target vector sentence.

[0145] For example, suppose there are 5 candidate vector clauses and 1 candidate result. The five third similarity values ​​calculated by the reranker model are 0.85, 0.92, 0.78, 0.95, and 0.89. Based on these similarity values, we select the vector clause with the largest similarity value (i.e., the vector clause corresponding to 0.95) as the target vector clause.

[0146] In short, the role of the reranker model is to re-rank the candidate sentences and find the sentence that best meets the actual needs as the target sentence. It improves the accuracy and rationality of the selection by introducing more features or complex models.

[0147] S103: Calculate the similarity between the candidate result and the target vector sentence to obtain a first similarity.

[0148] Calculating the similarity between the candidate result and the target vector sentence to obtain the first similarity means obtaining an initial similarity value for characterizing the degree of association between the candidate result and the target vector sentence by comparing the similarity between them.

[0149] It should be noted that in the process of calculating the similarity between the candidate results and the target vector sentence, the candidate results need to be vectorized, and then the vectorized candidate results and the target vector sentence are used to calculate the similarity.

[0150] S104: If the first similarity is greater than or equal to a first similarity threshold, determining the candidate result as a target result.

[0151] If the calculated first similarity is greater than or equal to a preset threshold (i.e., the "first similarity threshold"), then the current candidate result can be regarded as the final target result. In other words, the first similarity threshold is a standard used to determine whether the candidate result is sufficiently similar to the target vector sentence. If the similarity between a candidate result and the target vector reaches this standard, the candidate result can be considered correct or appropriate, and is therefore determined as the final target result.

[0152] In one possible implementation, if the calculated first similarity is less than a preset similarity threshold, the current candidate result does not meet the requirements and needs to be marked as "unavailable". Next, the system will reselect a new candidate result from multiple available question and answer results, and recalculate the similarity between it and the target vector sentence until a candidate result that meets the similarity requirements is found (i.e., step S103 and subsequent steps are re-executed using the new candidate result). This process ensures that only those candidate results that are sufficiently similar to the target vector will be selected, and inappropriate candidate results will be excluded to avoid wrong answers or choices. In other words, this is an iterative process until a target result that meets the standards is found.

[0153] In a possible implementation, if after traversing multiple candidate results, no candidate result is found whose similarity with the target vector sentence reaches the first similarity threshold, then the system will adjust these question-and-answer results through a thought chain (i.e., a reasoning or adjustment mechanism) to generate multiple new question-and-answer results. Then, the system will re-screen these new question-and-answer results to obtain the latest candidate results, calculate their similarity with the target vector sentence, and sort them by similarity until a candidate result that meets the conditions (i.e., the similarity is greater than or equal to the first similarity threshold) is found, and then continue to execute subsequent processing steps (i.e., re-execute step S103 and subsequent steps using the latest candidate result).

[0154] In a possible implementation, the first similarity threshold may be, but is not limited to, set to 0.7. The user may also adjust the first similarity threshold according to actual needs. This application does not impose any specific limitation on the size of the first similarity threshold.

[0155] S105: Combine the target result and the training question into a question-answer pair to obtain large model fine-tuning data.

[0156] Finally, the determined target result is paired with the corresponding training question to form a question-answer pair as data for model adjustment. Specifically, the target result refers to the candidate result selected as the correct answer after the aforementioned similarity calculation and screening, and the training question is the question or task related to the target result. These two parts are combined to form a complete question-answer pair, which can be used as large model fine-tuning data to help optimize and adjust the performance of the model. By continuously using this question-answer pair, the model can better understand and answer similar questions, thereby improving its accuracy and efficiency.

[0157] Based on the contents of S101-S105, it can be known that an input instruction is first generated based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt words of the training problem, and the instruction is input into a language generation model with a reflection mechanism to generate multiple question-and-answer results. These document prompt words are used to guide the generation requirements of the question-and-answer results to ensure the quality of the answers. Next, the target vector sentence is extracted from multiple original sentences of the original training data document, and the candidate result is selected from the generated question-and-answer results. The similarity value is determined by calculating the similarity between the candidate result and the target vector sentence. If the similarity value is greater than or equal to the set threshold, the candidate result is selected as the target result. Finally, the selected target result is combined with the training question into a question-and-answer pair to obtain large model fine-tuning data. This application uses document prompt words to indicate the generation requirements of the question-and-answer results to ensure that the generated answers cover multiple aspects. Multiple question-and-answer results are generated based on a language generation model with a reflection mechanism, which helps to think deeply and integrate various aspects of information during the generation process, thereby improving the completeness and accuracy of the question-and-answer results. In addition, the target result is determined by calculating the similarity between the candidate result and the target vector sentence. This matching mechanism helps to screen out the best question-and-answer result that meets the requirements and avoids the selection of incomplete or irrelevant answers.

[0158] See also Figure 5 , Figure 5 A schematic diagram of a device for acquiring fine-tuning data of a large model provided in an embodiment of the present application. Figure 5 As shown, the device for acquiring the fine-tuning data of the large model includes:

[0159] A first input instruction generating unit 501 is used to generate an input instruction based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt word of the training problem;

[0160] The instruction input unit 502 is used to input the input instruction into the language generation model with a reflection mechanism to obtain a plurality of question and answer results of the training question; the document prompt word is used to indicate the generation requirements of the question and answer result; the question and answer result is the answer to the training question;

[0161] A candidate result determination unit 503, configured to determine a candidate result from the plurality of question and answer results;

[0162] A target vector sentence acquisition unit 504 is used to acquire the target vector sentence of the candidate result from the multiple original sentences of the original training data document;

[0163] A first calculation unit 505 is used to calculate the similarity between the candidate result and the target vector sentence to obtain a first similarity;

[0164] a target result determining unit 506, configured to determine the candidate result as a target result if the first similarity is greater than or equal to a first similarity threshold;

[0165] The combining unit 507 is used to combine the target result and the training question into a question-answer pair to obtain large model fine-tuning data.

[0166] In a possible implementation, the target vector sentence obtaining unit 503 specifically includes:

[0167] A vectorization and sentence processing unit, used for performing vectorization and sentence processing on each original sentence of the original training data document to obtain a plurality of vector sentences;

[0168] A second calculation unit, configured to calculate similarities between the candidate result and the plurality of vector sentences respectively to obtain a plurality of second similarities;

[0169] a candidate vector clause determining unit, configured to determine N candidate vector clauses from the plurality of vector clauses based on a screening condition and the plurality of second similarities; N being an integer greater than 1;

[0170] The target vector sentence determining unit is used to determine the target vector sentence from the N candidate vector sentences by using the reranker model.

[0171] In a possible implementation manner, the target vector sentence determination unit specifically includes:

[0172] A third calculation unit is used to calculate the similarity between the candidate result and the N candidate vector sentences respectively by using the reranker model to obtain N third similarities;

[0173] The target vector sentence screening unit is used to screen the target vector sentence from the N candidate vector sentences based on the N third similarities; the third similarity corresponding to the target vector sentence is the largest among the N third similarities.

[0174] In a possible implementation, the vectorization and sentence processing unit specifically includes:

[0175] A vector document acquisition unit, used to convert the original training data document into a vector representation using text-embedding-ada-002 to obtain a vector document;

[0176] Define a sliding window unit, which is used to define a sliding window of a fixed size;

[0177] A document segmentation unit, used for segmenting the vector document into a plurality of text blocks using the sliding window;

[0178] A fourth calculation unit, used for calculating the semantic similarity between any adjacent text blocks;

[0179] The adjacent text block merging unit is used to merge the adjacent text blocks whose semantic similarity is greater than or equal to the second similarity threshold to obtain the vector sentence.

[0180] In a possible implementation, the first input instruction generating unit 501 specifically includes:

[0181] A comprehensive determination unit, used to determine the application field of the model to be adjusted, the training problem, and the document prompt words of the training problem;

[0182] An original training data document acquisition unit, used to acquire the original training data document according to the application field of the model to be adjusted;

[0183] The second input instruction generating unit is used to generate an input instruction based on the training question, the document prompt word and the original training data document.

[0184] In a possible implementation manner, the device further includes:

[0185] The first integration unit is used to mark the candidate result as unavailable if the first similarity is less than the first similarity threshold, and to re-determine a new candidate result from the multiple question and answer results, so as to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

[0186] In a possible implementation manner, the device further includes:

[0187] The second integration unit is used to adjust the multiple question and answer results through the thinking chain to obtain multiple new question and answer results, and re-determine the candidate results from the multiple new question and answer results, if no candidate result with the first similarity greater than or equal to the first similarity threshold is obtained after traversing multiple question and answer results, so as to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

[0188] In addition, an embodiment of the present application also provides a device for acquiring large model fine-tuning data, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for acquiring large model fine-tuning data as described above is implemented.

[0189] In addition, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the method for obtaining large model fine-tuning data as described above.

[0190] The embodiment of the present application provides a device for acquiring data for fine-tuning a large model. First, the first input instruction generating unit 501 is used to generate input instructions based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt words of the training problem, and the instruction input unit 502 is used to input the input instructions into the language generation model with a reflection mechanism to obtain multiple question-answering results of the training problem. It is also necessary to obtain the target vector sentence from multiple original sentences in the original training data document through the target vector sentence acquiring unit 503, and determine the candidate result from multiple question-answering results using the candidate result determining unit 504. Then, the first calculation unit 505 is used to calculate the similarity between the candidate result and the target vector sentence to obtain the first similarity. Finally, the target result determining unit 506 will determine the candidate result as the target result when the first similarity is greater than or equal to the first similarity threshold, so that the combination unit 507 combines the target result with the training problem into a question-answering pair, thereby obtaining the data for fine-tuning a large model. The present application clarifies the generation requirements through the document prompt words to ensure that the answer covers multiple dimensions, and generates multiple question-answering results in combination with the language generation model of the reflection mechanism, which promotes the model to think deeply and integrate various aspects of information, thereby improving the completeness and accuracy of the answer. The best question-and-answer results are screened by calculating the similarity between candidate answers and target vectors. This matching mechanism effectively avoids incomplete or irrelevant answers, ensuring the accuracy of the results and meeting requirements.

[0191] The above is a detailed introduction to a method, device, equipment and storage medium for obtaining fine-tuning data of a large model provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0192] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0193] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for obtaining large model fine-tuning data, characterized in that: The method comprises: Generate input instructions based on the application field of the model to be adjusted, the training problem, the original training data document and the document prompt words of the training problem, and input the input instructions into the language generation model with a reflection mechanism to obtain multiple question and answer results of the training problem; the document prompt words are used to indicate the generation requirements of the question and answer results; the question and answer results are the answers to the training problems; Determine a candidate result from the plurality of question-answer results, and obtain a target vector sentence of the candidate result from the plurality of original sentences of the original training data document; Calculating the similarity between the candidate result and the target vector sentence to obtain a first similarity; If the first similarity is greater than or equal to a first similarity threshold, determining the candidate result as a target result; The target result and the training question are combined into a question-answer pair to obtain large model fine-tuning data.

2. The method according to claim 1, characterized in that The step of obtaining the target vector clause of the candidate result from the plurality of original sentences of the original training data document comprises: Each original sentence of the original training data document is vectorized and segmented to obtain a plurality of vectorized sentences; Calculate similarity between the candidate result and the multiple vector sentences respectively to obtain multiple second similarities; Based on the screening condition and the plurality of second similarities, determining N candidate vector clauses from the plurality of vector clauses; N is an integer greater than 1; A reranker model is used to determine a target vector sentence from the N candidate vector sentences.

3. The method according to claim 2, characterized in that The step of using a reranker model to determine a target vector sentence from the N candidate vector sentences includes: Using the reranker model, respectively calculate the similarity between the candidate result and the N candidate vector sentences to obtain N third similarities; Based on the N third similarities, the target vector sentence is screened out from the N candidate vector sentences; the third similarity corresponding to the target vector sentence is the largest among the N third similarities.

4. The method according to claim 2, characterized in that: Each original sentence of the original training data document is vectorized and segmented to obtain a plurality of vectorized sentences, including: The original training data document is converted into a vector representation using a text embedded adaptive model 002text-embedding-ada-002 to obtain a vector document; defining a sliding window of a fixed size, and using the sliding window to segment the vector document into a plurality of text blocks; For any adjacent text blocks, calculate the semantic similarity between the adjacent text blocks; Adjacent text blocks whose semantic similarity is greater than or equal to a second similarity threshold are merged to obtain the vector sentence.

5. The method according to claim 1, characterized in that The step of generating input instructions based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt words of the training problem includes: Determining the application field of the model to be adjusted, the training problem, and the document prompt words of the training problem; Acquire the original training data document according to the application field of the model to be adjusted; An input instruction is generated based on the training question, the document prompt word, and the original training data document.

6. The method according to claim 1, characterized in that The method further comprises: If the first similarity is less than the first similarity threshold, the candidate result is marked as unavailable, and a new candidate result is re-determined from the multiple question and answer results to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

7. The method according to claim 6, characterized in that The method further comprises: If no candidate result whose first similarity is greater than or equal to the first similarity threshold is obtained after traversing multiple question and answer results, the multiple question and answer results are adjusted through the thinking chain to obtain multiple new question and answer results, and the candidate results are re-determined from the multiple new question and answer results to execute the calculation of the similarity between the candidate result and the target vector sentence to obtain the first similarity and subsequent steps.

8. A device for acquiring fine-tuning data of a large model, characterized in that: The device comprises: A first input instruction generating unit, configured to generate an input instruction based on the application field of the model to be adjusted, the training problem, the original training data document, and the document prompt word of the training problem; An instruction input unit, used for inputting the input instruction into a language generation model with a reflection mechanism to obtain a plurality of question and answer results of the training question; the document prompt word is used to indicate the generation requirements of the question and answer result; the question and answer result is the answer to the training question; A candidate result determination unit, used to determine a candidate result from a plurality of the question and answer results; A target vector sentence acquisition unit, configured to acquire the target vector sentence of the candidate result from a plurality of original sentences in the original training data document; A first calculation unit, configured to calculate a similarity between the candidate result and the target vector sentence to obtain a first similarity; a target result determining unit, configured to determine the candidate result as a target result if the first similarity is greater than or equal to a first similarity threshold; A combination unit is used to combine the target result and the training question into a question-answer pair to obtain large model fine-tuning data.

9. A device for acquiring fine-tuning data of a large model, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for obtaining large model fine-tuning data as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method for obtaining large model fine-tuning data as described in any one of claims 1 to 7.

Images