Data set generation and model training method and device and storage medium

Abstract

The embodiment of the invention provides a data set generation method and device, a model training method and device and a storage medium. According to the data set generation method, the first type of question text can be extracted from the general question and answer data set, and the second type of question text can be acquired from the question and answer data in the target application scene; and inputting the obtained multiple question texts into a preset base model to obtain respective reply texts of the multiple question texts, and constructing a fine tuning data set corresponding to the base model according to the multiple question texts and the respective corresponding reply texts thereof. On one hand, the fine adjustment data set generated by the base model is matched with the data distribution characteristics of the pre-training data set of the base model, so that the risk of catastrophic forgetting of the model after fine adjustment can be reduced. On the other hand, the fine tuning data set covers the sample data in the target application scene, training of the model in the vertical industry / field is facilitated, and the service performance of the model after fine tuning can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data set generation and model training method, device and storage medium. Background Art

[0002] With the development of artificial intelligence, generative and human-aligned large language models (Human-Aligned LLM (Large Language Model)) represented by ChatGPT (Chat Generative Pre-trained Transformer) have gradually been widely used. This type of large language model is often obtained by fine-tuning and reinforcement learning optimization on a large-scale general instruction data set. This process is called the pre-training process of the model. However, when the pre-trained large language model is applied to different fields, it is often necessary to fine-tune the large language model based on specific data in the field to make it meet the business needs of the application field.

[0003] However, in the process of fine-tuning a large language, if the distribution of the data used to fine-tune the model is unreasonable, it is easy to cause catastrophic forgetting of the large language model, thereby reducing the model performance. Therefore, a new solution is needed. Summary of the invention

[0004] Multiple aspects of the present application provide a data set generation and model training method, device and storage medium for constructing a fine-tuning data set that is adapted to the pre-training data set of the base model in terms of data distribution, thereby helping to reduce the risk of catastrophic forgetting problems occurring in the base model during the fine-tuning process.

[0005] The embodiment of the present application also provides a data set generation method, including: obtaining a general question and answer data set, and extracting a first type of question text from the general question and answer data set; obtaining question and answer data in a target application scenario, and obtaining a second type of question text from the question and answer data in the target application scenario; inputting a plurality of question texts consisting of the first type of question text and the second type of question text into a preset base model respectively, and obtaining respective answer texts of the plurality of question texts output by the base model; constructing a fine-tuning data set corresponding to the base model according to the plurality of question texts and their respective corresponding answer texts; and matching the data distribution characteristics of the fine-tuning data set with the pre-training data set of the base model.

[0006] Optionally, before inputting the multiple question texts consisting of the first type of question text and the second type of question text into the preset base model respectively, it also includes: configuring the sampling strategy parameters and / or sampling randomness parameters of the base model; wherein the configured sampling strategy parameters and / or sampling randomness parameters are used to enable the base model to output multiple different answer texts for the same question text.

[0007] Optionally, the reply text corresponding to any question text among the multiple question texts includes: multiple different initial reply texts; constructing a fine-tuning dataset corresponding to the base model based on the multiple question texts and their respective corresponding reply texts, including: performing quality analysis on the multiple initial reply texts corresponding to the question text to obtain analysis results of each of the multiple initial reply texts; screening out a target reply text corresponding to the question text from the multiple initial reply texts based on the analysis results of each of the multiple initial reply texts; and adding a question-answer text pair formed by the question text and the target reply text to the fine-tuning dataset.

[0008] Optionally, quality analysis is performed on multiple initial reply texts corresponding to the question text to obtain analysis results of each of the multiple initial reply texts, including: generating prompt words based on the question text, the multiple initial reply texts and at least one quality analysis dimension; inputting the prompt words into a preset evaluation model, so that the evaluation model performs quality analysis on the multiple initial reply texts corresponding to the question text based on the prompt words to obtain analysis results of each of the multiple initial reply texts; the evaluation model includes: a large language model with a parameter amount greater than a set threshold.

[0009] Optionally, based on the analysis results of each of the multiple initial reply texts, a target reply text corresponding to the question text is filtered out from the multiple initial reply texts, including: displaying the analysis results corresponding to each of the multiple initial reply texts, so that a user can filter the multiple initial reply texts according to the analysis results; the analysis results corresponding to any initial reply text include: quality score and / or quality analysis process information; in response to the user's selection operation of the multiple initial reply texts, the selected initial reply text is used as the target reply text corresponding to the question text.

[0010] Optionally, it also includes: mixing the question and answer text pairs in the fine-tuning data set and the question and answer text pairs in the business data set according to a set ratio to obtain a mixed data set; and using the mixed data set to fine-tune the base model.

[0011] Optionally, the business data set includes: question-answer text pairs in the target application scenario corresponding to the base model.

[0012] The embodiment of the present application also provides a model training method, including: using the data set generation method provided in the embodiment of the present application to obtain a fine-tuning data set corresponding to a base model; obtaining question-answer text pairs in a target application scenario corresponding to a target model as a business data set; mixing the question-answer text pairs in the fine-tuning data set with the question-answer text pairs in the business data set according to a set ratio to obtain a mixed data set; and using the mixed data set to fine-tune the pre-trained target model.

[0013] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: execute the steps in the method provided in the embodiment of the present application.

[0014] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which can implement the steps in the method provided in the embodiment of the present application when the computer program is executed by a processor.

[0015] In the method provided in the embodiment of the present application, a first type of question text can be extracted from a general question and answer data set, and a second type of question text can be obtained from the question and answer data in the target application scenario; a plurality of question texts consisting of the first type of question text and the second type of question text are respectively input into a preset base model to obtain the respective answer texts of the plurality of question texts output by the base model, and a fine-tuning data set corresponding to the base model is constructed according to the plurality of question texts and their respective corresponding answer texts. On the one hand, by matching the data distribution characteristics of the pre-trained data set of the base model with the fine-tuning data set generated by the base model, the risk of catastrophic forgetting of the fine-tuned model can be reduced. On the other hand, the fine-tuning data set covers the sample data in the target application scenario, which is conducive to the training of the model in the vertical industry / field and can improve the business performance of the fine-tuned model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A flowchart of a method for generating a data set provided by an exemplary embodiment of the present application;

[0018] Figure 2 A flowchart of a method for generating a data set provided by another exemplary embodiment of the present application;

[0019] Figure 3A flowchart of a model training method provided for an exemplary embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0022] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0023] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0024] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0025] In the process of fine-tuning a large language, if the distribution of the data used to fine-tune the model is unreasonable, it is easy to cause catastrophic forgetting problems in the large language model, thereby reducing the model performance. In view of the above technical problems, some embodiments of the present application provide a solution. The following is a detailed description of the technical solutions provided by each embodiment of the present application in conjunction with the accompanying drawings.

[0026] Figure 1 A flowchart of a method for generating a data set provided by an exemplary embodiment of the present application is shown in FIG. Figure 1As shown, the method mainly includes the following steps:

[0027] Step 101: Obtain a general question and answer dataset, and extract a first type of question text from the general question and answer dataset.

[0028] Step 102: Obtain question and answer data in the target application scenario, and obtain a second type of question text from the question and answer data in the target application scenario.

[0029] Step 103: input a plurality of question texts consisting of the first type of question texts and the second type of question texts into a preset base model respectively, and obtain answer texts of the plurality of question texts output by the base model.

[0030] Step 104: construct a fine-tuning dataset corresponding to the base model based on the multiple question texts and their respective corresponding answer texts; the fine-tuning dataset matches the data distribution characteristics of the pre-training dataset of the base model.

[0031] This embodiment is used to generate a fine-tuning dataset to fine-tune the base model based on the fine-tuning dataset to obtain a vertical model for application in a specific vertical field.

[0032] Among them, the base model is a general model with basic capabilities and knowledge. The base model is usually obtained by pre-training a large amount of text and code data based on self-supervised training, so it is also called a pre-trained large model. The base model may include: a base model of LLM (Large Language Model), such as GPT3 (Generative Pre-Trained Transformer 3, the third generation of generative pre-trained transformation model), LLaMA (LargeLanguage Model Meta AI, a large language model developed by Meta), etc. The base model may also include: a large language model aligned with human instructions (Human-Aligned LLM). Among them, the large language model aligned with human instructions is based on LLM, using fine-tuning and reinforcement learning optimization on a large-scale general instruction data set, which can generate text more accurately according to human instructions, such as ChatGPT (Chat Generative Pre-trained Transformer, Chat Generative Pre-trained Transformer), ChatGLM (Chat generalize linear model, Chat model based on generalized linear model), etc. Among them, the base model is applicable to a wide range of tasks in multiple fields, but lacks certain specialization in specific fields.

[0033] Among them, the vertical model refers to a special model that uses tuning methods to strengthen and adjust knowledge on the base model to compress certain industries or private data and knowledge. It is also called an industry model, vertical model, etc. By training the basic model with data sets in different industry fields, vertical models suitable for different industry fields can be obtained. The vertical model has a stronger language understanding ability in a specific field, so that it can perform tasks in the corresponding industry field more professionally.

[0034] In this embodiment, a general question-answering dataset may be obtained, and question text may be extracted from the general question-answering dataset. For ease of description and distinction, the question text extracted from the general question-answering dataset is described as a first-category question text. The first-category question text extracted from the general question-answering dataset may include multiple question texts.

[0035] Among them, the general question-answering dataset can be obtained from an open source dataset, for example, it can be at least one of the GPT4LLM dataset, the Alpaca dataset, and the Belle dataset. The general question-answering dataset is a dataset with high knowledge diversity and knowledge coverage. In the general question-answering dataset, a single question-answering data includes the instruction input of the model (i.e., the question text) and the reply output (the reply text). Among them, each question-answering data can be described in JSON (JavaScript Object Notation, JS object notation) format. For example, a question-answering data can be described as:

[0036] {

[0037] "Command": "Generate a creative birthday greeting for a friend."

[0038] "Reply": "Dear (friend's name), happy birthday! On this special day, may you have endless laughter and joy, may your wishes come true, and may your dreams become reality. I hope that in your future, there will be more surprises and more gains, and may every day of your life be radiant. Let us raise a toast to celebrate for you, and wish you an extremely happy birthday, forever young and happy!"

[0039] }

[0040] Specifically, after obtaining multiple question and answer data from the general question and answer data set, the instruction part in each question and answer data can be extracted to obtain the first type of question text. Optionally, after extracting multiple question texts from the general question and answer data set, the multiple question texts can be aggregated and the aggregated question texts can be deduplicated to reduce information redundancy.

[0041] In addition to the first type of question text, the question and answer data in the target application scenario can also be obtained, and the question text can be obtained from the question and answer data in the target application scenario as the second type of question text. Among them, the target application scenario refers to the application scenario corresponding to the vertical model obtained after fine-tuning the base model, that is, the application scenario in the vertical field. Among them, the target application scenarios may include: financial scenarios, medical scenarios, e-commerce scenarios, or teaching scenarios of designated subjects, etc. For example, when the fine-tuned vertical model is used to assist manual customer service in intelligent question and answer in a financial scenario, the question and answer data in the financial scenario can be obtained, and the question text can be obtained from the question and answer data in the financial scenario. For another example, when the fine-tuned vertical model is used to assist manual customer service in intelligent question and answer in a teaching scenario, the question and answer data in the teaching scenario can be obtained, and the question text can be obtained from the question and answer data in the teaching scenario.

[0042] Among them, an optional implementation method of obtaining question and answer data in the target application scenario may include: obtaining business data and / or question and answer knowledge graphs generated by the target application scenario, and obtaining question and answer data from the business data and / or question and answer knowledge graphs. For example, the historical chat records of manual customer service in the target application scenario can be obtained, and question and answer pairs can be separated from the historical chat records. From the question and answer pairs, the question text can be extracted.

[0043] Among them, another optional implementation method of obtaining question and answer data in the target application scenario may include: obtaining a large number of general question and answer data sets, and using keywords corresponding to the target application scenario to filter out question and answer data matching the target application scenario from the large number of question and answer data sets.

[0044] After obtaining the first type of question text and the second type of question text based on the above implementation, the first type of question text and the second type of question text can be mixed to obtain multiple mixed question texts. The multiple question texts can be respectively input into a preset base model to obtain the answer texts of the multiple question texts output by the base model. According to the multiple question texts and their corresponding answer texts, a fine-tuning data set corresponding to the base model can be constructed.

[0045] Among them, before fine-tuning, the parameters of the base model are learned in the pre-training stage, and then the fine-tuning data set generated by the base model matches the data distribution characteristics of the pre-training data set used by the pre-trained base model. Under the premise of the pre-training data set of the unknown base model, the operation of generating a fine-tuning data set based on the base model can be regarded as the operation of reversely inferring the reply text used by the base model during pre-training according to the known instructions (i.e., the question text), and then a fine-tuning data set with a high consistency with the distribution of the training data can be obtained. Furthermore, fine-tuning the base model based on the fine-tuning data set can enable the base model to be trained on a fine-tuning data set with a data distribution that is relatively consistent with the pre-training data set, thereby reducing the probability of catastrophic forgetting.

[0046] In some exemplary embodiments, in order to improve the quality of the reply text generated by the base model, the base model may be controlled to output multiple initial reply texts for the same instruction text. Figure 2 As shown in Figure 1, after a single first-category question text is input into the base model, multiple initial answer texts can be obtained. By screening these multiple initial answer texts, a target answer text that is more suitable for the instruction text can be obtained. Figure 2 An exemplary description is given.

[0047] Optionally, before the multiple acquired question texts are respectively input into the preset base model, the sampling strategy parameters (i.e., top-k or top-p parameters) and / or sampling randomness parameters (i.e., temperature parameters) of the base model can be configured. Among them, the top-k parameter is used to enable the base model to select the top k reply texts as the output result from the multiple reply texts generated. The top-p parameter is used to enable the base model to select the top p% reply texts as the output result from the multiple reply texts generated. Among them, the temperature parameter is used to adjust the output randomness of the base model. For the same input instruction, the smaller the temperature parameter, the smaller the randomness of the reply output by the base model, and there is even the possibility of self-repetition; the larger the temperature parameter, the higher the randomness of the reply output by the base model, the lower the repetition rate of different replies, and the base model can give more creative output results.

[0048] In this embodiment, the sampling strategy parameters and / or sampling randomness parameters can be configured according to empirical values. Based on the configured sampling strategy parameters and / or sampling randomness parameters, the base model can output a variety of different reply texts for the same question text, thereby increasing the probability of the base model outputting more reasonable reply texts.

[0049] After generating multiple initial answer texts corresponding to multiple question texts based on the above implementation, the multiple initial answer texts of each question text can be screened, so that the screened target answer text and the question text form a piece of training data with high rationality. An exemplary description will be given below.

[0050] Optionally, among the multiple question texts, the answer text corresponding to any question text includes: multiple different initial answer texts. Continuing to take any question text as an example, when constructing a fine-tuning dataset corresponding to the base model, quality analysis can be performed on the multiple initial answer texts corresponding to the question text to obtain analysis results of each of the multiple initial answer texts.

[0051] In some optional embodiments, an evaluation model may be used to perform quality analysis on the question text and its corresponding multiple initial answer texts.

[0052] Optionally, a prompt word may be generated based on the question text, the multiple initial answer texts, and at least one quality analysis dimension. The at least one quality analysis dimension may include: knowledge accuracy dimension and / or content diversity dimension. For example, the prompt word may be: "For question Q1, there are the following replies R1, R2, R3, and R4. Please score R1, R2, R3, and R4 from the two perspectives of knowledge accuracy and content information diversity, and give the analysis process, and finally output in JSON format."

[0053] The prompt word is input into a preset evaluation model, so that the evaluation model performs a quality analysis on multiple initial answer texts corresponding to the question text according to the prompt word, and obtains analysis results of each of the multiple initial answer texts. Optionally, the evaluation model may include: a large language model with a parameter quantity greater than a set threshold, such as a GPT4 model.

[0054] For any question text, after obtaining the analysis results of multiple initial reply texts of the question text based on the above-mentioned implementation method, the target reply text corresponding to the question text can be filtered out from the multiple initial reply texts according to the analysis results of the multiple initial reply texts.

[0055] In some optional embodiments, the analysis result corresponding to any initial reply text may include: a quality score. Furthermore, when screening the target reply text, for any question text, the evaluation model may select the reply text with the highest instruction score from the multiple initial reply texts according to the analysis results of each of the multiple initial reply texts corresponding to the question text as the target reply text corresponding to the question text.

[0056] In some other optional embodiments, the analysis result corresponding to any initial reply text may include: quality score and / or quality analysis process information. The quality analysis process information is the scoring basis given by the evaluation model. Figure 2 As shown, multiple initial reply texts corresponding to each question text can be provided to the quality analysis link of human-computer collaboration, and then the target reply text can be screened based on the human-computer collaboration. Continuing with any question text as an example, optionally, the analysis results corresponding to each of the multiple initial reply texts can be displayed so that the user can screen the multiple initial reply texts according to the analysis results. In response to the user's selection operation on the multiple initial reply texts, the selected initial reply text is used as the target reply text corresponding to the question text.

[0057] Of course, in addition to the methods described in the aforementioned embodiments, in some optional embodiments, for any question text, the correlation between multiple initial reply texts corresponding to the question text and one or more scene keywords corresponding to the target application scenario can be calculated respectively. According to the correlation calculation results, the target reply text is screened out from the multiple initial reply texts. For example, the target application scenario is a financial scenario, and the scene keywords corresponding to the financial scenario include: risk, budget, credit, etc. When a question text corresponds to multiple reply texts, the correlation between each reply text and each of the above-mentioned scene keywords can be calculated, and the correlation score corresponding to each reply text can be accumulated. According to the correlation scores of different reply texts, the reply text with the highest correlation with the target application scenario can be selected from multiple reply texts. Among them, when calculating the correlation between any reply text and any scene keyword, the reply text and the scene keyword can be mapped to the vector space, and the similarity of the mapped vectors is calculated as the correlation between the reply text and the scene keyword.

[0058] After filtering the target answer text based on the above implementation method, the question and answer text pair formed by the question text and the target answer text may be added to the fine-tuning dataset.

[0059] In some exemplary embodiments, after a fine-tuning dataset is generated based on the methods provided in the aforementioned embodiments, the base model may be fine-tuned based on the fine-tuning dataset.

[0060] Optionally, in order to improve the performance of the vertical model obtained after fine-tuning in the target application scenario, a business data set can be added to the data set for fine-tuning the base model. Optionally, the question-answer text pairs in the obtained fine-tuning data set and the question-answer text pairs in the business data set can be mixed according to a set ratio to obtain a mixed data set. Figure 2 As shown, question-answer text pairs in the business data set scenario can be obtained, and the obtained question-answer text pairs are mixed with the question-answer text pairs obtained in the quality analysis link of human-machine collaboration to obtain a mixed data set.

[0061] In some optional embodiments, in the process of extracting question-answer texts from the fine-tuning dataset and the business dataset for mixed processing, the similarity of question-answer text pairs extracted from different datasets can be calculated, and when the similarity is less than a set threshold, the extracted question-answer text pairs are added to the mixed dataset, so that the similarities of question-answer text pairs extracted from different datasets are as different as possible. Thus, the diversity of the mixed dataset is increased, and the robustness of the fine-tuned vertical model is improved.

[0062] The base model is fine-tuned using the mixed data set. Optionally, the business data set includes: question-answer text pairs in the target application scenario corresponding to the base model. Optionally, in the mixed data set, the number of question-answer texts from the fine-tuning data set can be 3 to 5 times the number of question-answer text pairs from the incremental training set.

[0063] Based on this implementation, the fine-tuning dataset can cover sample data in the vertical industry / field corresponding to the target application scenario, which is conducive to training in combination with the business scenario of the target model and improving the business performance of the fine-tuned target model. In the fine-tuning process, the base model can learn general question-answering knowledge on the fine-tuning dataset and learn question-answering knowledge in the target application scenario on the business dataset, thereby improving the performance of the fine-tuned vertical model in a specific application scenario without forgetting the knowledge learned in pre-training.

[0064] Figure 3 is a flow chart of a model training method provided by an exemplary embodiment of the present application, which method may include: Figure 3 Steps shown:

[0065] Step 301: Obtain a general question and answer dataset, and extract a first type of question text from the general question and answer dataset.

[0066] Step 302: Obtain question and answer data in the target application scenario, and obtain a second type of question text from the question and answer data in the target application scenario.

[0067] Step 303: input the multiple question texts consisting of the first type of question texts and the second type of question texts into a preset base model respectively, and obtain the answer texts of the multiple question texts output by the base model.

[0068] Step 304: construct a fine-tuning data set corresponding to the base model based on the multiple question texts and their respective corresponding answer texts; the fine-tuning data set matches the data distribution characteristics of the pre-training data set of the base model.

[0069] Step 305: Obtain question-answer text pairs in the target application scenario corresponding to the target model as a business data set.

[0070] Step 306: Mix the question-answer text pairs in the fine-tuning data set and the question-answer text pairs in the business data set according to a set ratio to obtain a mixed data set.

[0071] Step 307: Use the mixed data set to fine-tune the pre-trained target model.

[0072] The optional implementation methods of steps 301 to 304 may refer to the description of the aforementioned embodiment of data set generation, which will not be described in detail here.

[0073] In steps 305 to 307, the target model may be a base model or another model having the same or similar data distribution characteristics as the pre-trained data set of the base model, and this embodiment does not limit this. Wherein, when the question-answer text pairs in the fine-tuning data set and the question-answer text pairs in the business data set are mixed according to a set ratio, the question-answer text pairs in the fine-tuning data set and the question-answer text pairs in the business data set may be mixed according to a ratio of 3:1, 4:1, 5:1 or other ratios, and this embodiment includes but is not limited to this.

[0074] In this embodiment, on the one hand, the fine-tuning dataset is generated by the base model, so that the data distribution characteristics of the fine-tuning dataset match those of the pre-training dataset of the base model, thereby reducing the risk of catastrophic forgetting of the fine-tuned target model during the fine-tuning process. On the other hand, the fine-tuning dataset is generated based on the second type of question text in the target application scenario, so that the fine-tuning dataset covers the sample data corresponding to the target application scenario, thereby facilitating the training of the target model in the business scenarios in the vertical industry / field, and improving the business performance of the fine-tuned target model.

[0075] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 301 to 304 can be device A; for another example, the execution subject of steps 301 and 302 can be device A, and the execution subject of step 303 can be device B; and so on.

[0076] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 301, 302, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0078] Figure 4 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application is shown. Figure 4 As shown, the electronic device includes: a memory 401 , a processor 402 and a communication component 403 .

[0079] The memory 401 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method operating on the electronic device.

[0080] In some optional embodiments, the processor 402 is coupled to the memory 401 and is used to execute the computer program in the memory 401 to execute a data set generation method. The processor 402 is specifically used to: obtain a general question and answer data set, and extract a first type of question text from the general question and answer data set; obtain question and answer data in the target application scenario, and obtain a second type of question text from the question and answer data in the target application scenario; input multiple question texts consisting of the first type of question text and the second type of question text into a preset base model respectively, and obtain the answer texts of the multiple question texts output by the base model; construct a fine-tuning data set corresponding to the base model according to the multiple question texts and their corresponding answer texts; the fine-tuning data set matches the data distribution characteristics of the pre-training data set of the base model.

[0081] Optionally, before the processor 402 inputs the multiple question texts consisting of the first type of question text and the second type of question text into the preset base model respectively, it is also used to: configure the sampling strategy parameters and / or sampling randomness parameters of the base model; wherein the configured sampling strategy parameters and / or sampling randomness parameters are used to enable the base model to output multiple different answer texts for the same question text.

[0082] Optionally, the reply text corresponding to any question text among the multiple question texts includes: multiple different initial reply texts; when the processor 402 constructs the fine-tuning dataset corresponding to the base model according to the multiple question texts and their respective corresponding reply texts, it is specifically used to: perform quality analysis on the multiple initial reply texts corresponding to the question text to obtain analysis results of each of the multiple initial reply texts; based on the analysis results of each of the multiple initial reply texts, filter out the target reply text corresponding to the question text from the multiple initial reply texts; and add the question-answer text pair formed by the question text and the target reply text to the fine-tuning dataset.

[0083] Optionally, when the processor 402 performs quality analysis on the multiple initial reply texts corresponding to the question text and obtains analysis results of each of the multiple initial reply texts, it is specifically used to: generate prompt words based on the question text, the multiple initial reply texts and at least one quality analysis dimension; input the prompt words into a preset evaluation model, so that the evaluation model performs quality analysis on the multiple initial reply texts corresponding to the question text according to the prompt words, and obtains analysis results of each of the multiple initial reply texts; the evaluation model includes: a large language model with a parameter amount greater than a set threshold.

[0084] Optionally, when the processor 402 filters out the target reply text corresponding to the question text from the multiple initial reply texts based on the analysis results of each of the multiple initial reply texts, it is specifically used to: display the analysis results corresponding to each of the multiple initial reply texts, so that the user can filter the multiple initial reply texts according to the analysis results; the analysis results corresponding to any initial reply text include: quality score and / or quality analysis process information; in response to the user's selection operation on the multiple initial reply texts, the selected initial reply text is used as the target reply text corresponding to the question text.

[0085] Optionally, the processor 402 is also used to: mix the question and answer text pairs in the fine-tuning data set and the question and answer text pairs in the business data set according to a set ratio to obtain a mixed data set; and use the mixed data set to fine-tune the base model.

[0086] Optionally, the business data set includes: question-answer text pairs in the target application scenario corresponding to the base model.

[0087] In some optional embodiments, the processor 402 is coupled to the memory 401 and is used to execute the computer program in the memory 401 to execute a model training method. The processor 402 is specifically used to: obtain a fine-tuning data set corresponding to the base model using the generation method of the model training data set provided in the embodiment of the present application; obtain the question-answer text pairs in the target application scenario corresponding to the target model as a business data set; mix the question-answer text pairs in the fine-tuning data set and the question-answer text pairs in the business data set according to a set ratio to obtain a mixed data set; and use the mixed data set to fine-tune the pre-trained target model.

[0088] Further, if Figure 4 As shown, the electronic device also includes: a power supply component 404, a display component 405, an audio component 406 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 Components shown.

[0089] Among them, the memory 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0090] Among them, the communication component 403 is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0091] The power supply component 404 is used to provide power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power to the device where the power supply component is located.

[0092] The display component 405 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0093] The audio component 406 may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0094] In this embodiment, the first type of question text can be extracted from the general question and answer data set, and the second type of question text can be obtained from the question and answer data in the target application scenario; the multiple question texts consisting of the first type of question text and the second type of question text are respectively input into the preset base model to obtain the answer texts of the multiple question texts output by the base model, and the fine-tuning data set corresponding to the base model is constructed according to the multiple question texts and their corresponding answer texts. On the one hand, the fine-tuning data set is generated by the base model, so that the data distribution characteristics of the fine-tuning data set and the pre-training data set of the base model are matched, thereby reducing the risk of catastrophic forgetting of the fine-tuned target model during the fine-tuning process. On the other hand, the fine-tuning data set is generated based on the second type of question text in the target application scenario, so that the fine-tuning data set can cover the sample data corresponding to the target application scenario, thereby facilitating the training of the target model in the business scenario of the vertical industry / field, and improving the business performance of the fine-tuned target model.

[0095] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0096] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0101] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0102] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0103] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0104] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for generating a data set, wherein the data set is used for model training, characterized in that: include: Obtain a general question-answering dataset, and extract a first type of question text from the general question-answering dataset; Acquire question and answer data in a target application scenario, and acquire a second type of question text from the question and answer data in the target application scenario; Inputting a plurality of question texts consisting of the first type of question texts and the second type of question texts into a preset base model respectively, and obtaining answer texts of the plurality of question texts output by the base model; A fine-tuning dataset corresponding to the base model is constructed based on the multiple question texts and their respective corresponding answer texts; the fine-tuning dataset matches the data distribution characteristics of the pre-training dataset of the base model.

2. The method according to claim 1, characterized in that Before inputting the plurality of question texts consisting of the first type of question texts and the second type of question texts into the preset base model respectively, the method further includes: The sampling strategy parameters and / or sampling randomness parameters of the base model are configured; wherein the configured sampling strategy parameters and / or sampling randomness parameters are used to enable the base model to output a variety of different answer texts for the same question text.

3. The method according to claim 1, characterized in that Among the multiple question texts, the answer text corresponding to any question text includes: multiple different initial answer texts; According to the multiple question texts and their corresponding answer texts, constructing a fine-tuning dataset corresponding to the base model, including: Performing quality analysis on a plurality of initial answer texts corresponding to the question text to obtain analysis results of each of the plurality of initial answer texts; According to the analysis results of each of the multiple initial reply texts, a target reply text corresponding to the question text is screened out from the multiple initial reply texts; The question-answer text pair formed by the question text and the target answer text is added to the fine-tuning dataset.

4. The method according to claim 3, characterized in that Performing quality analysis on the multiple initial answer texts corresponding to the question text to obtain analysis results of the multiple initial answer texts, including: Generate prompt words according to the question text, the multiple initial answer texts and at least one quality analysis dimension; The prompt word is input into a preset evaluation model so that the evaluation model performs quality analysis on multiple initial reply texts corresponding to the question text according to the prompt word to obtain analysis results of each of the multiple initial reply texts; the evaluation model includes: a large language model with a parameter quantity greater than a set threshold.

5. The method according to claim 3, characterized in that: According to the analysis results of each of the multiple initial reply texts, a target reply text corresponding to the question text is screened out from the multiple initial reply texts, including: Displaying the analysis results corresponding to each of the multiple initial reply texts, so that the user can filter the multiple initial reply texts according to the analysis results; the analysis result corresponding to any initial reply text includes: quality score and / or quality analysis process information; In response to the user's selection operation on the multiple initial reply texts, the selected initial reply text is used as the target reply text corresponding to the question text.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: Mixing the question-answer text pairs in the fine-tuning dataset and the question-answer text pairs in the business dataset according to a set ratio to obtain a mixed dataset; The base model is fine-tuned using the mixed dataset.

7. The method according to claim 6, characterized in that The business data set includes: question-answer text pairs in the target application scenario corresponding to the base model.

8. A model training method, characterized in that: include: Acquire a fine-tuning data set corresponding to the base model by using the method described in any one of claims 1 to 7; Obtain question-answer text pairs in the target application scenario corresponding to the target model as a business data set; Mixing the question-answer text pairs in the fine-tuning dataset and the question-answer text pairs in the business dataset according to a set ratio to obtain a mixed dataset; The pre-trained target model is fine-tuned using the mixed dataset.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to: perform the steps in the method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the method described in any one of claims 1 to 8 can be implemented.

Images