Question and answer pair generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a question and answer pair generation method and device, electronic equipment and a storage medium, and belongs to the technical field of natural language processing. The method comprises the steps of preprocessing a target text; extracting a preset proportion of target knowledge points to construct a first prompt word, and generating a fine tuning data set by using the reference large language model; performing low-rank adaptation training on the local large language model through the fine tuning data set; constructing a second prompt word based on the target knowledge point, and generating a question-answer pair to be evaluated by using an available local large language model; evaluating the to-be-evaluated question and answer pair by using an available local large language model based on a set evaluation rule; constructing a third prompt word according to the available question and answer pairs, and generating similar question and answer pairs by using the available local large language model; and integrating the similar question-answer pairs and the available question-answer pairs to generate target question-answer pairs. The question and answer pair generation efficiency and quality can be improved, the suitability of the question and answer pair in an actual question and answer scene is enhanced, and the occurrence of abnormal question and answer conditions such as'answering not questions' is reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of natural language processing technology, and in particular, to a question-answer pair generation method, device, electronic device, and storage medium. Background Art

[0002] At present, intelligent customer service technology has been applied in many industries. Through intelligent human-machine collaboration system, it can improve online service efficiency, reduce labor costs and the burden of training new employees. However, the quality of existing intelligent customer service questions and answers is highly dependent on carefully maintained question and answer libraries, and the question and answer pairs in these question and answer libraries are usually manually compiled by experienced R&D personnel, which is time-consuming and labor-intensive. In addition, the manually compiled question and answer pairs are inconsistent with the actual question and answer scenarios, which easily leads to abnormal question and answer situations such as "answering the wrong question", which limits the expansion of intelligent customer service. Summary of the invention

[0003] Embodiments of the present invention provide a question-answer pair generation method, device, electronic device, and storage medium to improve the efficiency and quality of question-answer pair generation, enhance the adaptability of question-answer pairs in actual question-answer scenarios, and help reduce the occurrence of abnormal question-answer situations such as "irrelevant answers".

[0004] In a first aspect, an embodiment of the present invention provides a method for generating a question-answer pair, comprising at least the following steps:

[0005] Preprocess the target text to generate target knowledge points;

[0006] Extracting a preset proportion of the target knowledge points to construct a first prompt, and generating a fine-tuning dataset according to the first prompt using a reference large language model;

[0007] Performing low-rank adaptation training on the local large language model using the fine-tuning dataset to obtain a usable local large language model;

[0008] Constructing a second prompt based on the target knowledge point, and generating a question-answer pair to be evaluated according to the second prompt using the available local large language model;

[0009] Based on the set evaluation rules, the question-answer pair to be evaluated is evaluated using the available local large language model to obtain an available question-answer pair;

[0010] Constructing a third prompt according to the available question-answer pair, and generating a similar question-answer pair according to the third prompt using the available local large language model;

[0011] The similar question-answer pairs and the available question-answer pairs are integrated to generate target question-answer pairs for the question-answer subject to call.

[0012] Optionally, the preprocessing of the target text to generate target knowledge points at least includes:

[0013] Performing a first processing on the target text to at least remove knowledge points in the target text that are not applicable to the question-answer pair generation scenario, thereby obtaining first intermediate knowledge points;

[0014] Performing a second process on the first intermediate knowledge points to at least remove knowledge points whose values ​​are empty among the first intermediate knowledge points, thereby obtaining second intermediate knowledge points;

[0015] The second intermediate knowledge points are subjected to a third processing to at least eliminate the knowledge points in the second intermediate knowledge points that are not suitable for voice broadcasting, thereby generating the target knowledge points.

[0016] Optionally, extracting a preset proportion of the target knowledge points to construct a first prompt, and generating a fine-tuning dataset according to the first prompt using a reference large language model, at least includes:

[0017] Classifying the target knowledge points according to the application scenarios of the knowledge points;

[0018] According to the classification of the target knowledge points, extracting the preset proportion of the target knowledge points to construct the first prompt;

[0019] The fine-tuning dataset is generated according to the first prompt using the reference large language model.

[0020] Optionally, performing low-rank adaptive training on the local large language model by using the fine-tuning dataset to obtain a usable local large language model at least includes:

[0021] Selecting a low-rank adaptive training layer in the local large language model;

[0022] At least setting hyperparameters, ranks and training parameters related to the low-rank adaptive training;

[0023] Based on the low-rank adaptive training layer, the hyperparameter, the rank and the training parameter, the local large language model is trained through the fine-tuning dataset to obtain the usable local large language model.

[0024] Optionally, the step of evaluating the question-answer pair to be evaluated based on the set evaluation rule using the available local large language model to obtain an available question-answer pair at least includes:

[0025] constructing a fourth prompt based on the corpus relevance between each of the question-answer pairs to be evaluated and the corresponding target knowledge point, and generating a relevance score according to the fourth prompt using the available local large language model;

[0026] Performing vectorization processing on the questions and answers constituting each of the question-answer pairs to be evaluated to measure the similarity between the vectorized questions and the vectorized answers, and generating a matching score;

[0027] Calculate a comprehensive evaluation score for each of the question-answer pairs to be evaluated according to the relevance score and the matching score;

[0028] When the comprehensive evaluation score of any of the question-answer pairs to be evaluated is lower than a preset threshold, the score of the corresponding question-answer pair to be evaluated is deemed to be unsatisfactory and the question-answer pair to be evaluated is eliminated to obtain the available question-answer pairs;

[0029] The fourth prompt includes at least one of background setting text, text to be evaluated, task issuing text, evaluation point, few sample text and output control text.

[0030] Optionally, the comprehensive evaluation score is calculated at least in the following manner:

[0031] S = M1*A+M2*B;

[0032] Among them, S represents the comprehensive evaluation score, M1 represents the weight of the corpus relevance, M2 represents the weight of the matching degree, A represents the relevance score, and B represents the matching score.

[0033] Optionally, the first prompt includes at least one of background setting text, task issuance text, few sample text, knowledge point text and output control text; the second prompt includes at least one of background setting text, task issuance text and knowledge point text; the third prompt includes at least one of target text, background setting text, task issuance text, output control text and few sample text.

[0034] In a second aspect, an embodiment of the present invention further provides a question-answer pair generating device, comprising at least:

[0035] A preprocessing module, at least used for preprocessing the target text to generate target knowledge points;

[0036] A data set generation module, at least used to extract a preset proportion of the target knowledge points to construct a first prompt, and generate a fine-tuning data set according to the first prompt using a reference large language model;

[0037] A low-rank training module, at least used to perform low-rank adaptation training on the local large language model through the fine-tuning dataset to obtain a usable local large language model;

[0038] A module for generating question-answer pairs to be evaluated, at least for constructing a second prompt based on the target knowledge point, and generating a question-answer pair to be evaluated according to the second prompt using the available local large language model;

[0039] An available question-answer pair generating module, at least used to evaluate the question-answer pair to be evaluated using the available local large language model based on set evaluation rules to obtain an available question-answer pair;

[0040] A similar question-answer pair generation module, at least used to construct a third prompt based on the available question-answer pair, and generate a similar question-answer pair based on the third prompt using the available local large language model;

[0041] The question-answer pair integration module is at least used to integrate the similar question-answer pairs and the available question-answer pairs to generate target question-answer pairs for the question-answer subject to call.

[0042] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the question-answer pair generation method as described in any one of the first aspects are performed.

[0043] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the question-answer pair generation method described in any one of the first aspects.

[0044] The embodiment of the present invention provides a method, device, electronic device and storage medium for generating question-answer pairs. First, the target text is preprocessed to generate target knowledge points; second, a preset proportion of target knowledge points is extracted to construct a first prompt, and a reference large language model is used to generate a fine-tuning data set according to the first prompt; third, the local large language model is low-rank adaptively trained through the fine-tuning data set to obtain an available local large language model; third, a second prompt is constructed based on the target knowledge point, and the available local large language model is used to generate a question-answer pair to be evaluated according to the second prompt; third, based on the set evaluation rules, the available local large language model is used to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair; then, a third prompt is constructed according to the available question-answer pair, and the available local large language model is used to generate a similar question-answer pair according to the third prompt; finally, the similar question-answer pairs and the available question-answer pairs are integrated to generate a target question-answer pair for the question-answer subject to call.

[0045] In view of this, on the one hand, the embodiments of the present invention involve the optimization of the question-answer pair generation process. In the question-answer pair generation scenario based on the large language model, the step of automatically generating a fine-tuning data set with reference to the large language model can reduce the workload of manually annotating the data set, which is conducive to improving the efficiency of question-answer pair generation; and, based on the fine-tuning data set, low-rank adaptive training and fine-tuning of the local large language model can make the local large language model more suitable for the practical scenario of question-answer pairs. On the other hand, the embodiments of the present invention also involve the step of evaluating and integrating the question-answer pairs. Through this step, the degree of fit between the question-answer pairs and their actual application scenarios can be further determined, which is conducive to ensuring the quality of the question-answer pairs and avoiding the occurrence of abnormal question-answer situations such as "answering the wrong question" to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a flow chart of a method for generating question-answer pairs provided by an embodiment of the present invention;

[0048] Figure 2 is a flowchart of a preprocessing provided by an embodiment of the present invention;

[0049] Figure 3 is a flow chart of another question-answer pair generation method provided by an embodiment of the present invention;

[0050] Figure 4 is a structural schematic diagram of a question-answer pair generating device provided by an embodiment of the present invention;

[0051] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0053] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0054] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0055] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0056] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0057] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0058] It should be particularly noted that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0059] Figure 1This is a flowchart of a question-answer pair generation method provided by an embodiment of the present invention. This embodiment can be applied to any intelligent customer service application scenario with question-answering needs, such as insurance product consultation, national or local policy explanation, vehicle fault analysis, etc. The question-answer pair generation method can be, but is not limited to, executed by the question-answer pair generation device in the embodiment of the present invention as the execution subject, and the execution subject can be implemented in software and / or hardware. Figure 1 As shown, the question-answer pair generation method includes at least the following steps:

[0060] S1. Preprocess the target text to generate target knowledge points.

[0061] The target text may refer to relevant materials containing knowledge points required for constructing the target question-answer pair. For example, if the present embodiment is applied to vehicle fault analysis, the target text may be a vehicle operation manual, a user manual, a vehicle repair manual, etc.; if the present embodiment is applied to insurance product consultation, the target text may be an insurance consultation work order, an insurance product document, etc. It is understandable that the target knowledge point may refer to the aforementioned knowledge point required for constructing the target question-answer pair.

[0062] S2. Extract a preset proportion of target knowledge points to construct a first prompt, and use a reference large language model to generate a fine-tuning dataset based on the first prompt.

[0063] The preset ratio can be adaptively adjusted according to the characteristic parameters of the target knowledge point (such as the scene category of the target knowledge point, etc.), the training effect to be obtained by training the local large language model through fine-tuning the data set, and other factors, and the present invention does not limit this. In addition, the reference large language model can select any existing mature and commercial large language model, such as the GPT-4 application programming interface (Application Programming Interface, API) and other 100 billion-level large language models.

[0064] In a specific implementation, optionally, the first prompt includes at least one of a background setting text, a task delivery text, a few-shot text, a knowledge point text, and an output control text. Specifically, the background setting text may refer to a text that can provide a detailed context to help the reference large language model fully understand the task background, question-and-answer situation, and the intended goal; for example, the background setting text may explain the scene in which the reference large language model is located and the role to be played. Specifically, the task delivery text may be used to illustrate the specific tasks that need to be completed by the reference large language model; for example, the reference large language model is required to generate a question-and-answer pair in the fine-tuning data set based on the extracted text corresponding to each target knowledge point (that is, the fine-tuning data set may be composed of multiple question-and-answer pairs, each question-and-answer pair corresponding to an extracted target knowledge point). Specifically, the few-shot text may guide the reference large language model to understand how to perform and complete specific tasks by providing a small number of examples; for example, the few-shot text may give an example based on the practical scenario of each type of target knowledge point, in which both the target knowledge point and the corresponding question-and-answer pair are explained. Specifically, the knowledge point text can be used to pass the text information of each extracted target knowledge point to the reference large language model. Specifically, to facilitate the subsequent integration and processing of the fine-tuning dataset, the output control text can be used to require the reference large language model to uniformly output each question-answer pair in the fine-tuning dataset in a specific format; for example, the output format must be {"instruction": task instruction, "input": text information, "output": output question-answer pair}, etc.

[0065] S3. Perform low-rank adaptation training on the local large language model by fine-tuning the dataset to obtain a usable local large language model.

[0066] Among them, the local large language model can adopt any non-commercial or untrained large language model whose size, training degree, network complexity, etc. are lower than the reference large language model.

[0067] It can be known that Low-Rank Adaptation (LORA) training is a fine-tuning method that can be used for some parameters in a large language model; the available local large language model may refer to a local large language model trained with LORA.

[0068] It can be understood that since the fine-tuning dataset is generated by the reference large language model based on a preset proportion of target knowledge points, and the preset proportion can be related to the scenario category of the target knowledge points, the fine-tuning dataset can cover all scenarios involved in the target knowledge points. This also means that the available local large language model obtained by using the fine-tuning dataset as a training set and training with LORA can be adapted to any scenario involved in the target knowledge points, that is, the available local large language model has initially possessed the ability to adapt to the question and answer generation scenario.

[0069] S4. Construct a second prompt based on the target knowledge point, and use the available local large language model to generate a question-answer pair to be evaluated based on the second prompt.

[0070] Among them, because the available local large language model can adapt to any scenario involving the target knowledge points, after all the target knowledge points are input into the available local large language model, the available local large language model can generate all question-answer pairs corresponding to all the target knowledge points, that is, generate question-answer pairs to be evaluated.

[0071] Specifically, the way in which all target knowledge points are input into the available local large language model can be input in the form of a second prompt. In another specific embodiment, optionally, the second prompt includes at least one of a background setting text, a task issuing text, and a knowledge point text. It is understandable that the background setting text in the second prompt may refer to a text that can provide a detailed context to help the available local large language model fully understand the background of the task of generating the question and answer pair to be evaluated, the question and answer situation, the intended goal, etc.; for example, the background setting text can explain the scene of generating the question and answer pair to be evaluated in which the available local large language model is located and the role to be played. Specifically, the task issuing text in the second prompt can be used to illustrate the generation task of the specific question and answer pair to be evaluated that needs to be completed by the available local large language model; for example, the available local large language model is required to generate a question and answer pair to be evaluated based on the text corresponding to each target knowledge point. Specifically, the knowledge point text can be used to pass the text information of each target knowledge point to the available local large language model.

[0072] S5. Based on the set evaluation rules, use the available local large language model to evaluate the question-answer pairs to obtain available question-answer pairs.

[0073] Among them, setting evaluation rules may involve multiple aspects, for example, it may include the semantic coherence of questions and answers in the question-answer pair, the generation delay of the question-answer pair, etc.

[0074] It can be seen that if the local large language model can be used to determine that a certain question-answer pair to be evaluated meets the set evaluation rules, then the question-answer pair to be evaluated can be classified as an available question-answer pair; on the contrary, if the local large language model can be used to determine that a certain question-answer pair to be evaluated does not meet the set evaluation rules, then the question-answer pair to be evaluated cannot be classified as an available question-answer pair. It can be understood that the available question-answer pair can meet the calling requirements of the question-answer subject and can be called by the question-answer subject in a specific question-answer scenario to answer the questions raised by the user in the specific question-answer scenario.

[0075] Of course, the evaluation rules and the question-answer pairs to be evaluated can also be set by inputting the available local large language model in the form of prompts, which will not be explained in detail here.

[0076] S6. Construct a third prompt based on the available question-answer pairs, and use the available local large language model to generate similar question-answer pairs based on the third prompt.

[0077] Among them, the reason for setting similar question-answer pairs is that in a certain question-answer scenario, when a user asks the same question with similar meaning to the available question-answer pairs in different ways (such as sign language, etc.), using different vocabulary (such as synonyms, etc.), using different languages ​​or dialects (such as English, German, Tianjin dialect, Tangshan dialect, etc.), it is difficult for the question-answering subject to accurately identify what the user really wants to ask and give an accurate answer. In view of this, the difference between similar question-answer pairs and available question-answer pairs can be the specific form of the user's question, the vocabulary of the user's question, the language or dialect involved in the user's question, etc.

[0078] In another specific implementation, the third prompt may include at least one of a target text, a background setting text, a task issuing text, an output control text, and a few sample texts. Specifically, the target text may refer to a text that generates a series of similar questions for each question in an available question-answer pair.

[0079] Background setting text can refer to text that can provide detailed context to help the available local large language model fully understand the background of the similar question and answer pair generation task, the question and answer situation, the intended goal, etc.; for example, the background setting text can explain the scene in which the available local large language model is located and the role it needs to play in the similar question and answer pair generation process. Specifically, the task delivery text can be used to explain the specific similar question and answer pair generation task that the available local large language model needs to complete; for example, the available local large language model is required to generate similar question texts using synonyms based on the question text corresponding to each available question and answer pair. Specifically, the few sample text can guide the available local large language model to understand how to execute and complete the similar question and answer pair generation task by providing a small number of examples; for example, the few sample text can give an example of synonym replacement based on the question of a certain available question and answer pair, in which the question of the available question and answer pair should be explained, and the question after the synonym replacement should be given. Specifically, in order to facilitate the subsequent integration and processing of similar question and answer pairs, the output control text can be used to require the available local large language model to uniformly output each question and answer pair in the similar question and answer pair in a specific format.

[0080] S7. Integrate similar question-answer pairs and available question-answer pairs to generate target question-answer pairs for the question-answer subject to call.

[0081] The question-and-answer subject may refer to intelligent customer service, customer service robots, etc. It is understandable that the target question-and-answer pairs generated by integrating similar question-and-answer pairs and available question-and-answer pairs can meet the actual question-and-answer scenario requirements of various users in different regions, and further optimize the scenario adaptability of the question-and-answer subject.

[0082] This embodiment provides a method for generating question-answer pairs. First, the target text is preprocessed to generate target knowledge points; second, a preset proportion of target knowledge points is extracted to construct a first prompt, and a reference large language model is used to generate a fine-tuning data set according to the first prompt; third, the local large language model is low-rank adaptively trained through the fine-tuning data set to obtain an available local large language model; third, a second prompt is constructed based on the target knowledge point, and the available local large language model is used to generate a question-answer pair to be evaluated according to the second prompt; third, based on the set evaluation rules, the available local large language model is used to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair; then, a third prompt is constructed according to the available question-answer pair, and the available local large language model is used to generate a similar question-answer pair according to the third prompt; finally, the similar question-answer pairs and the available question-answer pairs are integrated to generate a target question-answer pair for the question-answer subject to call.

[0083] In summary, on the one hand, this embodiment involves the optimization of the question-answer pair generation process. In the question-answer pair generation scenario based on the large language model, the step of automatically generating a fine-tuning data set with reference to the large language model can reduce the workload of manually annotating the data set, which is conducive to improving the efficiency of question-answer pair generation; and, based on the fine-tuning data set, low-rank adaptive training and fine-tuning of the local large language model can make the local large language model more suitable for practical scenarios of question-answer pairs. On the other hand, this embodiment also involves the step of evaluating and integrating the execution of question-answer pairs, through which the degree of fit between the question-answer pair and its actual application scenario can be further determined, which is conducive to ensuring the quality of the question-answer pair and avoiding the occurrence of abnormal question-answer situations such as "answering the wrong question" to the greatest extent.

[0084] It should be noted that, in addition to the knowledge points required for constructing the question-answer pair, the target text may also contain knowledge points that are irrelevant to the question-answer pair construction. Therefore, the target knowledge points may refer to all the knowledge points remaining in the target text after preprocessing to remove the knowledge points that are irrelevant to the question-answer pair construction. The steps of preprocessing are described below, but are not intended to limit the present invention.

[0085] In a specific implementation, the target text is optionally preprocessed to generate target knowledge points, which at least includes: performing a first processing on the target text to at least eliminate knowledge points in the target text that are not suitable for the question-answer pair generation scenario, thereby obtaining first intermediate knowledge points; performing a second processing on the first intermediate knowledge points to at least eliminate knowledge points whose values ​​are empty in the first intermediate knowledge points, thereby obtaining second intermediate knowledge points; performing a third processing on the second intermediate knowledge points to at least eliminate knowledge points in the second intermediate knowledge points that are not suitable for voice broadcasting, thereby generating target knowledge points.

[0086] Based on the above implementation, Figure 2 is a flowchart of a preprocessing provided by an embodiment of the present invention, see Figure 2 , the preprocessing includes at least the following steps:

[0087] S11. Perform a first processing on the target text to at least remove knowledge points in the target text that are not suitable for the question-answer pair generation scenario, thereby obtaining first intermediate knowledge points.

[0088] The first process may refer to a manual process step. It is known that the scenario that is not applicable to question-answer generation may refer to a scenario that requires multiple rounds of interaction with the user.

[0089] S12: Perform a second process on the first intermediate knowledge points to at least remove knowledge points whose values ​​are empty among the first intermediate knowledge points, thereby obtaining second intermediate knowledge points.

[0090] The specific implementation method of the second processing may be to remove the knowledge points whose values ​​are empty in the first intermediate knowledge points through the None keyword in Python.

[0091] S13, performing a third process on the second intermediate knowledge points to at least eliminate knowledge points in the second intermediate knowledge points that are not suitable for voice broadcasting, thereby generating target knowledge points.

[0092] The knowledge points that are not suitable for voice broadcasting may refer to knowledge points containing information such as telephone numbers and websites. It is understandable that the third processing may be implemented by programs such as Python and C++.

[0093] In summary, this embodiment implements the preprocessing flow of the target text through three processing steps. Except for the first processing step, the other two processing steps do not require manual intervention, which reduces the workload of manual processing of the target text and helps improve the processing efficiency of the target text.

[0094] Based on the above embodiments or implementation methods, the specific method for generating a fine-tuning dataset, the specific training process of a local large language model, and the specific process for obtaining available question-answer pairs are described in detail below, which does not serve as a limitation on the embodiments of the present invention.

[0095] In a specific implementation, extracting a preset proportion of target knowledge points to construct a first prompt, and using a reference large language model to generate a fine-tuning dataset according to the first prompt, at least includes:

[0096] Classify target knowledge points according to their application scenarios;

[0097] According to the classification of the target knowledge points, a preset proportion of the target knowledge points are extracted to construct the first prompt;

[0098] Generate a fine-tuning dataset based on the first prompt using a reference large language model.

[0099] In another specific implementation, low-rank adaptive training is performed on the local large language model by fine-tuning the dataset to obtain a usable local large language model, which at least includes:

[0100] Select low-rank adaptive training layers in the local large language model;

[0101] At least set the hyperparameters, ranks and training parameters related to low-rank adaptive training;

[0102] Based on the low-rank adaptive training layer, hyperparameters, ranks and training parameters, the local large language model is trained by fine-tuning the dataset to obtain a usable local large language model.

[0103] In another specific implementation, based on the set evaluation rules, the available local large language model is used to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair, which at least includes:

[0104] Constructing a fourth prompt based on the corpus relevance between each question-answer pair to be evaluated and the corresponding target knowledge point, and generating a relevance score based on the fourth prompt using an available local large language model;

[0105] Performing vectorization processing on the question and answer constituting each question-answer pair to be evaluated to measure the similarity between the vectorized question and the vectorized answer and generate a matching score;

[0106] Calculate the comprehensive evaluation score of each question-answer pair to be evaluated based on the relevance score and the matching score;

[0107] When the comprehensive evaluation score of any question-answer pair to be evaluated is lower than the preset threshold, the score of the corresponding question-answer pair to be evaluated is deemed to be unsatisfactory and the question-answer pair to be evaluated is eliminated to obtain available question-answer pairs;

[0108] The fourth prompt includes at least one of background setting text, text to be evaluated, task issuing text, evaluation point, few sample text and output control text.

[0109] Based on the above implementation, Figure 3 is a flowchart of another method for generating question-answer pairs provided by an embodiment of the present invention, see Figure 3 , the question-answer pair generation method includes at least the following steps:

[0110] S1. Preprocess the target text to generate target knowledge points.

[0111] S21. Classify the target knowledge points according to their application scenarios.

[0112] Among them, the specific implementation method of step S21 can be to classify the target knowledge points according to the user's specific consulting scenario. For example, for the consulting scenario of automobile company customer service, the target knowledge points can be divided into major categories such as product information consulting, sales consulting, operating instructions, fault analysis, etc.

[0113] S22. According to the classification of the target knowledge points, a preset proportion of the target knowledge points are extracted to construct a first prompt.

[0114] Among them, continuing to take the aforementioned automobile enterprise customer service consulting scenario as an example, the target knowledge points are divided into four major categories: product information consulting, sales consulting, operating instructions and fault analysis. In order to enable the reference large language model to have the ability to generate question-answer pairs for the target knowledge points corresponding to each major category, it is necessary to reasonably allocate the target knowledge points of each major category when constructing the fine-tuning dataset. Specifically, the allocation can be based on the proportion of the target knowledge points.

[0115] Assume that it is necessary to construct a fine-tuning dataset with a data volume of 1,000 items, and the total data volume of the target knowledge points is 5,000 items (among which, the data volume of the target knowledge points corresponding to the product information consulting category is 2,000 items, accounting for 40% of the total data volume; the data volume of the target knowledge points corresponding to the sales consulting category is 2,000 items, accounting for 40% of the total data volume; the data volume of the target knowledge points corresponding to the operating instructions category is 500 items, accounting for 10% of the total data volume; the data volume of the target knowledge points corresponding to the fault analysis category is 500 items, accounting for 10% of the total data volume), then in the process of extracting a preset proportion of target knowledge points, the extracted data volume of the target knowledge points corresponding to the product information consulting category is 400 items (corresponding to 400 target knowledge points in the product information consulting category), the extracted data volume of the target knowledge points corresponding to the sales consulting category is 400 items, the extracted data volume of the target knowledge points corresponding to the operating instructions category is 100 items, and the extracted data volume of the target knowledge points corresponding to the fault analysis category is 100 items.

[0116] S23. Generate a fine-tuning dataset based on the first prompt using a reference large language model.

[0117] Among them, in order to ensure the generation effect of the fine-tuning dataset, it is preferred to call the aforementioned trillion-level large language model here. The trillion-level large language model has a deeper network structure and stronger language understanding ability, and can more accurately capture the complexity and nuances of the language.

[0118] S31. Select a low-rank adaptive training layer in the local large language model.

[0119] Among them, the essence of LORA training is to divide the parameters into two parts, one part is the frozen parameter W, and the other part is the fine-tuning incremental weight ΔW. Assuming that the input of the local large language model is x and the output is h, then:

[0120] h=Wx+ΔWx;

[0121] In LORA, singular value decomposition ΔW is used, that is, ΔW is approximated by low-rank matrices A and B:

[0122] h=Wx+BAx;

[0123] At the same time, LORA also uses the hyperparameters α and rank γ to adjust the above two low-rank matrices:

[0124] h=Wx+αBAx / γ;

[0125] Obviously, the basic idea of ​​LORA training is to fix the frozen parameters W (also called pre-trained weights) and only train the low-rank matrices A and B. When saving the weights, only the low-rank matrix part needs to be saved. This setting can greatly reduce the number of trainable parameters for downstream tasks.

[0126] Therefore, LoRa can embed a low-rank adapter in any layer of the local large language model (such as the Embedding layer, Attention layer, MLP layer, etc.). In actual application, the layer that contributes the most to the size and computational cost of the local large language model is preferred. Usually, the Attention layer is fine-tuned by LoRa, that is, the low-rank adaptive training layer is preferred to the Attention layer.

[0127] S32. At least set the hyperparameters, ranks and training parameters related to the low-rank adaptive training.

[0128] The parameter γ is a hyperparameter that needs to be determined experimentally. If γ is too large, it may lead to overfitting, and if γ is too small, it may not be enough to handle the diverse tasks in the fine-tuning dataset. Adjusting α helps to maintain a balance between fitting the data and preventing overfitting through regularization. Usually, it is necessary to try a variety of different combinations of γ and α and make experimental judgments. The inventors have found through experiments that it is more reasonable to select an α value that is twice as large as γ when fine-tuning the local large language model.

[0129] It can be understood that the hyperparameter related to low-rank adaptive training is α; the rank related to low-rank adaptive training is γ; the training parameters related to low-rank adaptive training can be, for example, dropout rate (Dropout), batch size (Batch size), training round (Epoch), etc. Specifically, the dropout rate can refer to temporarily setting the output of some neurons of the local large language model to 0 with a certain probability, so as to reduce the risk of overfitting; the range of the dropout rate is generally [0,1). If the dropout rate is set too large, it may cause the local large language model to lose some important information; conversely, if the dropout rate is set too small, the local large language model may not be able to fully learn the features of the fine-tuning data set, resulting in overfitting; the inventors have found through experiments that the dropout rate of the local large language model is preferably set to 0.1. In addition, the batch size is usually set according to the computing resources of the local large language model; the training round usually requires at least 3 rounds of training, which can be determined based on the task complexity, computing power conditions and experimental results, and the present invention is not limited to this.

[0130] S33. Based on the low-rank adaptive training layer, hyperparameters, rank and training parameters, the local large language model is trained by fine-tuning the data set to obtain a usable local large language model.

[0131] S4. Construct a second prompt based on the target knowledge point, and use the available local large language model to generate a question-answer pair to be evaluated based on the second prompt.

[0132] S51. Construct a fourth prompt based on the corpus relevance between each question-answer pair to be evaluated and the corresponding target knowledge point, and generate a relevance score according to the fourth prompt using an available local large language model.

[0133] Among them, corpus relevance can refer to the degree of relevance between the question and answer pair to be evaluated and the corresponding target knowledge point; setting the corpus relevance is mainly to avoid the situation where the local large language model can be used to answer based on its own general knowledge, thereby making the generation of the question and answer pair to be evaluated invalid.

[0134] For example, the specific form of the fourth prompt may be as follows:

[0135] [Background Setting Text] It is known that the question-answer pairs to be evaluated are obtained from the corresponding target knowledge points.

[0136] [Text to be evaluated] Among them: Question and answer pairs to be evaluated: {text}; Target knowledge points: {text}.

[0137] [Task delivery text] The task of the available local large language model is to analyze the content relevance between the above-mentioned question and answer pairs to be evaluated and the target knowledge points and give corresponding scores. The relevance score must range from 0 to 1.

[0138] [Evaluation point text] The higher the degree of correlation between the two contents, the higher the correlation score.

[0139]

Few sample texts

[0140] Here are three specific examples where a local large language model can learn the analysis process and scoring logic:

[0141] Example 1:

[0142] Target knowledge point 1:

[0143] A hot cylinder is a vehicle that is filled with liquefied natural gas (LNG) for the first time or is parked for more than 6 days after the cylinder is filled with LNG.

[0144] Question 1:

[0145] “Question”: “What is a hot bottle?”

[0146] "Answer": "A thermos is a thermos bottle that can keep water warm all day long, usually using vacuum insulation technology to keep the water at the same temperature for a long time."

[0147] Analysis process:

[0148] The content of question and answer pair 1 is not related to the content of target knowledge point 1.

[0149] Score: "0".

[0150] Example 2:

[0151] Target knowledge point 2:

[0152] [Summary] Mr. *, * model, the reason for the engine oil leakage;

[0153] [Handling process] Inform the customer: "Customer service cannot make a judgment, but can assist in contacting the service station for inspection and repair."

[0154] Question 2:

[0155] “Question”: “Consult the cause of engine oil leakage.”

[0156] "Answer": "The main causes of engine oil leakage include uneven tightening force of the fastening nut, broken or loose threads, poor product (accessory) quality, material or workmanship, too much lubricating oil, too high oil level or insufficient bonding at the cylinder joint, etc. You can go to the service station for inspection and repair."

[0157] Analysis process:

[0158] Some of the content in question and answer pair 2 is not reflected in target knowledge point 2.

[0159] Score: “0.5”.

[0160] Example three:

[0161] Target knowledge point 3:

[0162] [Summary] Mr. *, * model, I want to know which base produced the vehicle with chassis number *;

[0163] [Handling process] Inform the customer: "Chassis number ** represents ** car manufacturer, and he understands it."

[0164] Question 3:

[0165] “Question”: “For the model of **, I want to know which base produced the chassis number *.”

[0166] "Answer": "If the chassis number is **, it means it is from ** car factory."

[0167] Analysis process: Question and answer pair 3 is a summary of the content of target knowledge point 3.

[0168] Score: “1”.

[0169] [Output control text] The final output answer of the available local large language model must be a value between 0 and 1.

[0170] S52. Perform vectorization processing on the question and answer constituting each question-answer pair to be evaluated to measure the similarity between the vectorized question and the vectorized answer, and generate a matching score.

[0171] The similarity between the vectorized (embedding) problem and the vectorized answer may be indicative of whether the quantitative answer is directly related to the vectorized problem, whether it deviates from the subject of the vectorized problem, whether it covers all the key points of the vectorized problem, etc.

[0172] Vectorization processing can refer to converting the question and answer of each question-answer pair to be evaluated into numerical form. There are many specific methods for vectorization processing. First, the Embedding model can be obtained through special training of corresponding field data. Of course, existing mature open source Embedding models can also be used as needed, such as BCE of NetEase and BGE developed by Zhiyuan Research Institute; then the similarity between two vectors (i.e., vectorized answer and vectorized question) is measured through cosine similarity, Euclidean distance, Jaccard similarity and other methods. The similarity range is [0,1]. The higher the similarity, the higher the matching degree and the higher the matching score.

[0173] S53. Calculate the comprehensive evaluation score of each question-answer pair to be evaluated based on the relevance score and the matching score.

[0174] The comprehensive evaluation score may be a weighted sum of the relevance score and the matching score.

[0175] In a specific implementation, optionally, the comprehensive evaluation score is calculated at least in the following manner:

[0176] S = M1*A+M2*B;

[0177] In the formula, S represents the comprehensive evaluation score, M1 represents the weight of corpus relevance, M2 represents the weight of matching, A represents the relevance score, and B represents the matching score.

[0178] S54. When the comprehensive evaluation score of any question-answer pair to be evaluated is lower than a preset threshold, the score of the corresponding question-answer pair to be evaluated is deemed to be substandard and the question-answer pair to be evaluated is eliminated to obtain available question-answer pairs.

[0179] Among them, the preset threshold, the weight of the corpus relevance, and the weight of the matching degree can all be adaptively configured according to the evaluation effect of the question and answer to be obtained. After careful research, the inventors found that the preset threshold, the weight of the corpus relevance, and the weight of the matching degree can generally be set to 0.5.

[0180] S6. Construct a third prompt based on the available question-answer pairs, and use the available local large language model to generate similar question-answer pairs based on the third prompt.

[0181] S7. Integrate similar question-answer pairs and available question-answer pairs to generate target question-answer pairs for the question-answer subject to call.

[0182] Among them, step S7 can be specifically to organize the data format of available question and answer pairs and similar question and answer pairs to be consistent with the knowledge base template of the question and answer subject, so as to batch enter them into the knowledge base of the question and answer subject and provide them for use by the question and answer subject.

[0183] In summary, on the one hand, this embodiment involves the optimization of the question-answer pair generation process. In the question-answer pair generation scenario based on the large language model, the step of automatically generating a fine-tuning data set with reference to the large language model can reduce the workload of manually annotating the data set, which is conducive to improving the efficiency of question-answer pair generation; and, based on the fine-tuning data set, low-rank adaptive training and fine-tuning of the local large language model can make the local large language model more suitable for practical scenarios of question-answer pairs. On the other hand, this embodiment also involves the step of evaluating and integrating the execution of question-answer pairs, through which the degree of fit between the question-answer pair and its actual application scenario can be further determined, which is conducive to ensuring the quality of the question-answer pair and avoiding the occurrence of abnormal question-answer situations such as "answering the wrong question" to the greatest extent.

[0184] It should be noted that in addition to the above-mentioned solution of training the local large language model with LORA by fine-tuning the data set to obtain a usable local large language model, in some embodiments, other fine-tuning methods can be selected to train or adjust the local large language model according to the question-answer pair generation requirements, computing power and time cost. Exemplarily, all parameters of the local large language model can be adjusted by full fine-tuning, but the resource and time costs of this method are relatively high. In addition, a small number of parameters of the local large language model can be adjusted by partial parameter fine-tuning. This method usually requires pre-freezing some pre-trained parameters and only training parameters of certain layers; for example, prefix fine-tuning (including Prompt-tuning, P-tunning, Prefix-tuning, P-tuning v2, etc.), fine-tuning of all parameters of certain layers of transformer (including QLora, etc.), and fine-tuning based on reinforcement learning (including RHLF, DPO, etc.) will not be repeated.

[0185] The embodiment of the present invention also provides a question-answer pair generating device, Figure 4 is a schematic diagram of the structure of a question-answer pair generating device provided by an embodiment of the present invention. This embodiment can be applied to any intelligent customer service application scenario with question-answering requirements, such as insurance product consultation, national or local policy explanation, vehicle fault analysis, etc. The device can be implemented in software and / or hardware. Figure 4 As shown, the question-answer pair generating device provided in this embodiment at least includes:

[0186] A preprocessing module 110 is at least used to preprocess the target text to generate target knowledge points;

[0187] The data set generation module 120 is at least used to extract a preset proportion of target knowledge points to construct a first prompt, and generate a fine-tuning data set according to the first prompt using a reference large language model;

[0188] A low-rank training module 130 is at least used to perform low-rank adaptive training on the local large language model by fine-tuning the data set to obtain a usable local large language model;

[0189] The module 140 for generating question-answer pairs to be evaluated is at least used to construct a second prompt based on the target knowledge point, and to generate a question-answer pair to be evaluated according to the second prompt using an available local large language model;

[0190] An available question-answer pair generating module 150 is at least used to evaluate the question-answer pair to be evaluated using an available local large language model based on set evaluation rules to obtain an available question-answer pair;

[0191] A similar question-answer pair generating module 160, at least for constructing a third prompt based on the available question-answer pairs, and generating similar question-answer pairs based on the third prompt using the available local large language model;

[0192] The question-answer pair integration module 170 is at least used to integrate similar question-answer pairs and available question-answer pairs to generate target question-answer pairs for the question-answer subject to call.

[0193] Optionally, the preprocessing module 110 is at least specifically used to perform a first processing on the target text to at least eliminate knowledge points in the target text that are not suitable for the question-answer pair generation scenario, thereby obtaining first intermediate knowledge points; and, perform a second processing on the first intermediate knowledge points to at least eliminate knowledge points whose values ​​are empty in the first intermediate knowledge points, thereby obtaining second intermediate knowledge points; and, perform a third processing on the second intermediate knowledge points to at least eliminate knowledge points in the second intermediate knowledge points that are not suitable for voice broadcasting, thereby generating target knowledge points.

[0194] Optionally, the dataset generation module 120 is at least specifically used to classify the target knowledge points according to the application scenarios of the knowledge points; and, based on the classification of the target knowledge points, extract a preset proportion of the target knowledge points to construct a first prompt; and, using a reference large language model, generate a fine-tuning dataset based on the first prompt.

[0195] Optionally, the low-rank training module 130 is at least used to select a low-rank adaptive training layer in the local large language model; and, at least set the hyperparameters, rank and training parameters related to the low-rank adaptive training; and, based on the low-rank adaptive training layer, hyperparameters, rank and training parameters, train the local large language model by fine-tuning the data set to obtain a usable local large language model.

[0196] Optionally, the available question and answer pair generation module 150 is at least used to construct a fourth prompt based on the corpus relevance between each question and answer pair to be evaluated and the corresponding target knowledge point, and use the available local large language model to generate a relevance score based on the fourth prompt; and, perform vectorization processing on the questions and answers that constitute each question and answer pair to be evaluated to measure the similarity between the vectorized questions and the vectorized answers, and generate a matching score; and, calculate the comprehensive evaluation score of each question and answer pair to be evaluated based on the relevance score and the matching score; and, when the comprehensive evaluation score of any question and answer pair to be evaluated is lower than a preset threshold, it is determined that the score of the corresponding question and answer pair to be evaluated does not meet the standard and the question and answer pair to be evaluated is eliminated to obtain an available question and answer pair; wherein the fourth prompt includes at least one of background setting text, text to be evaluated, task issuance text, evaluation point, small sample text and output control text.

[0197] Optionally, the comprehensive evaluation score is calculated by at least the following methods:

[0198] S = M1*A+M2*B;

[0199] In the formula, S represents the comprehensive evaluation score, M1 represents the weight of corpus relevance, M2 represents the weight of matching, A represents the relevance score, and B represents the matching score.

[0200] Optionally, the first prompt includes at least one of background setting text, task issuance text, few sample text, knowledge point text and output control text; the second prompt includes at least one of background setting text, task issuance text and knowledge point text; the third prompt includes at least one of target text, background setting text, task issuance text, output control text and few sample text.

[0201] This embodiment provides a question-answer pair generation device. First, the target text is preprocessed by a preprocessing module to generate a target knowledge point; second, a preset proportion of the target knowledge points is extracted by a data set generation module to construct a first prompt, and a fine-tuning data set is generated according to the first prompt using a reference large language model; third, a low-rank training module is used to perform low-rank adaptation training on the local large language model through the fine-tuning data set to obtain an available local large language model; third, a second prompt is constructed based on the target knowledge point by a to-be-evaluated question-answer pair generation module, and a to-be-evaluated question-answer pair is generated according to the second prompt using an available local large language model; third, an available question-answer pair generation module is used to evaluate the to-be-evaluated question-answer pair based on a set evaluation rule using an available local large language model to obtain an available question-answer pair; then, a third prompt is constructed according to the available question-answer pair by a similar question-answer pair generation module, and a similar question-answer pair is generated according to the third prompt using an available local large language model; finally, a similar question-answer pair and an available question-answer pair are integrated by a question-answer pair integration module to generate a target question-answer pair for the question-answer subject to call.

[0202] In summary, on the one hand, this embodiment involves the optimization of the question-answer pair generation process. In the question-answer pair generation scenario based on the large language model, the step of automatically generating a fine-tuning data set with reference to the large language model can reduce the workload of manually annotating the data set, which is conducive to improving the efficiency of question-answer pair generation; and, based on the fine-tuning data set, low-rank adaptive training and fine-tuning of the local large language model can make the local large language model more suitable for practical scenarios of question-answer pairs. On the other hand, this embodiment also involves the step of evaluating and integrating the execution of question-answer pairs, through which the degree of fit between the question-answer pair and its actual application scenario can be further determined, which is conducive to ensuring the quality of the question-answer pair and avoiding the occurrence of abnormal question-answer situations such as "answering the wrong question" to the greatest extent.

[0203] An embodiment of the present invention further provides an electronic device, Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, see Figure 5, the electronic device 1000 includes a processor 1001 and a memory 1002, the memory 1002 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 1001, the steps in any one of the above question-answer pair generation methods are executed. Through the above technical solution, the processor 1001 and the memory 1002 are interconnected and communicate with each other through a communication bus and / or other forms of connection mechanisms (not marked), and the memory 1002 stores a computer program executable by the processor. When the electronic device 1000 is running, the processor 1001 executes the computer program to execute the question-answer pair generation method in any optional implementation of the above embodiment to achieve the following functions: preprocess the target text to generate target knowledge points; extract a preset proportion of target knowledge points to construct a first prompt, and use a reference large language model to generate a question-answer pair according to the first prompt. Generate a fine-tuning dataset; perform low-rank adaptive training on the local large language model through the fine-tuning dataset to obtain an available local large language model; construct a second prompt based on the target knowledge point, and use the available local large language model to generate a question-answer pair to be evaluated based on the second prompt; based on the set evaluation rules, use the available local large language model to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair; construct a third prompt based on the available question-answer pair, and use the available local large language model to generate a similar question-answer pair based on the third prompt; integrate the similar question-answer pairs and the available question-answer pairs to generate a target question-answer pair for the question-answer subject to call.

[0204] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a question-answer pair generation method as provided in all the inventive embodiments of the present application: preprocessing the target text to generate target knowledge points; extracting a preset proportion of the target knowledge points to construct a first prompt, and using a reference large language model to generate a fine-tuning data set according to the first prompt; performing low-rank adaptive training on the local large language model through the fine-tuning data set to obtain an available local large language model; constructing a second prompt based on the target knowledge point, and using the available local large language model to generate a question-answer pair to be evaluated based on the second prompt; based on set evaluation rules, using the available local large language model to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair; constructing a third prompt based on the available question-answer pair, and using the available local large language model to generate a similar question-answer pair based on the third prompt; integrating similar question-answer pairs and available question-answer pairs to generate a target question-answer pair for the question-answer subject to call.

[0205] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0206] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0207] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0208] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0209] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating question-answer pairs, characterized in that: At least the following steps are included: Preprocess the target text to generate target knowledge points; Extracting a preset proportion of the target knowledge points to construct a first prompt, and generating a fine-tuning dataset according to the first prompt using a reference large language model; Performing low-rank adaptation training on the local large language model using the fine-tuning dataset to obtain a usable local large language model; Constructing a second prompt based on the target knowledge point, and generating a question-answer pair to be evaluated according to the second prompt using the available local large language model; Based on the set evaluation rules, the available local large language model is used to evaluate the question-answer pair to be evaluated to obtain an available question-answer pair; Constructing a third prompt according to the available question-answer pair, and generating a similar question-answer pair according to the third prompt using the available local large language model; The similar question-answer pairs and the available question-answer pairs are integrated to generate target question-answer pairs for the question-answer subject to call.

2. The question-answer pair generation method according to claim 1, characterized in that: The preprocessing of the target text to generate target knowledge points at least includes: Performing a first processing on the target text to at least remove knowledge points in the target text that are not applicable to the question-answer pair generation scenario, thereby obtaining first intermediate knowledge points; Performing a second process on the first intermediate knowledge points to at least remove knowledge points whose values ​​are empty among the first intermediate knowledge points, thereby obtaining second intermediate knowledge points; The second intermediate knowledge points are subjected to a third processing to at least eliminate the knowledge points in the second intermediate knowledge points that are not suitable for voice broadcasting, thereby generating the target knowledge points.

3. The question-answer pair generation method according to claim 1, characterized in that: The step of extracting a preset proportion of the target knowledge points to construct a first prompt, and generating a fine-tuning dataset according to the first prompt using a reference large language model, at least includes: Classifying the target knowledge points according to the application scenarios of the knowledge points; According to the classification of the target knowledge points, extracting the preset proportion of the target knowledge points to construct the first prompt; The fine-tuning dataset is generated according to the first prompt using the reference large language model.

4. The question-answer pair generation method according to claim 1, characterized in that: The low-rank adaptive training of the local large language model by using the fine-tuning dataset to obtain a usable local large language model at least includes: Selecting a low-rank adaptive training layer in the local large language model; At least setting hyperparameters, ranks and training parameters related to the low-rank adaptive training; Based on the low-rank adaptive training layer, the hyperparameter, the rank and the training parameter, the local large language model is trained through the fine-tuning dataset to obtain the usable local large language model.

5. The question-answer pair generation method according to claim 1, characterized in that: The step of evaluating the question-answer pair to be evaluated by using the available local large language model based on the set evaluation rule to obtain an available question-answer pair at least includes: constructing a fourth prompt based on the corpus relevance between each of the question-answer pairs to be evaluated and the corresponding target knowledge point, and generating a relevance score according to the fourth prompt using the available local large language model; Performing vectorization processing on the questions and answers constituting each of the question-answer pairs to be evaluated to measure the similarity between the vectorized questions and the vectorized answers, and generating a matching score; Calculate a comprehensive evaluation score for each of the question-answer pairs to be evaluated according to the relevance score and the matching score; When the comprehensive evaluation score of any of the question-answer pairs to be evaluated is lower than a preset threshold, the score of the corresponding question-answer pair to be evaluated is deemed to be unsatisfactory and the question-answer pair to be evaluated is eliminated to obtain the available question-answer pairs; The fourth prompt includes at least one of background setting text, text to be evaluated, task issuing text, evaluation point, few sample text and output control text.

6. The question-answer pair generation method according to claim 5, characterized in that: The comprehensive evaluation score is calculated at least in the following ways: S = M1*A+M2*B; In the formula, S represents the comprehensive evaluation score, M1 represents the weight of the corpus relevance, M2 represents the weight of the matching degree, A represents the relevance score, and B represents the matching score.

7. The question-answer pair generation method according to any one of claims 1 to 6, characterized in that: The first prompt includes at least one of background setting text, task issuance text, few sample text, knowledge point text and output control text; the second prompt includes at least one of background setting text, task issuance text and knowledge point text; the third prompt includes at least one of target text, background setting text, task issuance text, output control text and few sample text.

8. A question-answer pair generating device, characterized in that: At least: A preprocessing module, at least used for preprocessing the target text to generate target knowledge points; A data set generation module, at least used to extract a preset proportion of the target knowledge points to construct a first prompt, and generate a fine-tuning data set according to the first prompt using a reference large language model; A low-rank training module, at least used to perform low-rank adaptation training on the local large language model through the fine-tuning dataset to obtain a usable local large language model; A module for generating question-answer pairs to be evaluated, at least for constructing a second prompt based on the target knowledge point, and generating a question-answer pair to be evaluated according to the second prompt using the available local large language model; An available question-answer pair generating module, at least used to evaluate the question-answer pair to be evaluated using the available local large language model based on set evaluation rules to obtain an available question-answer pair; A similar question-answer pair generation module, at least used to construct a third prompt based on the available question-answer pair, and generate a similar question-answer pair based on the third prompt using the available local large language model; The question-answer pair integration module is at least used to integrate the similar question-answer pairs and the available question-answer pairs to generate target question-answer pairs for the question-answer subject to call.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the question-answer pair generation method as described in any one of claims 1 to 7 are executed.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the question-answer pair generation method described in any one of claims 1 to 7 is implemented.

Images