Machine learning model training method and device and knowledge base generation method and device

Abstract

The invention provides a machine learning model training method and device and a knowledge base generation method and device, and relates to the technical field of computers. The machine learning model training method comprises the following steps: processing a specified text by utilizing a machine learning model to obtain a plurality of candidate question and answer pairs and an evaluation result of each candidate question and answer pair in the plurality of candidate question and answer pairs; according to the evaluation result, labeling the plurality of candidate question and answer pairs to generate a plurality of sample question and answer pairs, each sample question and answer pair in the plurality of sample question and answer pairs having first labeling information or second labeling information; and training the machine learning model according to the probability distribution information of the plurality of sample question and answer pairs and the plurality of candidate question and answer pairs for the quality of different question and answer pairs.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for training a machine learning model, and a method and device for generating a knowledge base. Background Art

[0002] With the rapid development of information technology, the amount of data and documents is growing exponentially. Governments and enterprises are faced with the challenge of how to efficiently manage and utilize this information. In government office scenarios, how to extract high-quality knowledge from a large number of accumulated documents for empowerment is a key technical issue in the construction of intelligent government affairs.

[0003] For example, by automatically extracting useful question and answer pairs (QA) from massive documents, it can not only improve the efficiency of information retrieval, but also provide more accurate answers, significantly improving the user experience. By converting unstructured text into structured question and answer pairs, users can quickly obtain the information they need and reduce the trouble caused by information overload. Summary of the invention

[0004] The inventors noticed that in the prior art, key information (key points) in the document is captured as the answer. Next, the question generation model is used to generate high-quality questions based on the key information, and supplemented by manual expert correction to form QA question-answer pairs. However, this method cannot adapt to the user question-answering needs in specific business scenarios. For example, for a policy document, the content focuses on the basis and objectives of the policy, but users are mainly concerned about some how-to questions.

[0005] Accordingly, the present disclosure provides a knowledge base generation method, which processes files using a trained machine learning model to obtain question-answer pairs that meet requirements.

[0006] In a first aspect of the present disclosure, a method for training a machine learning model is provided, comprising: using a machine learning model to process a specified text to obtain a plurality of candidate question-answer pairs, and an evaluation result of each of the plurality of candidate question-answer pairs; based on the evaluation result, annotating the plurality of candidate question-answer pairs to generate a plurality of sample question-answer pairs, each of the plurality of sample question-answer pairs having first annotation information or second annotation information; and training the machine learning model based on the plurality of sample question-answer pairs and probability distribution information of the plurality of candidate question-answer pairs for different question-answer pair qualities.

[0007] In some embodiments, the training of the machine learning model includes: dividing the plurality of sample question-answer pairs into a first sample set and a second sample set, wherein each sample question-answer pair in the first sample set has first labeling information, and each sample question-answer pair in the second sample set has second labeling information; using the first sample set to label the first sample set. sample question-answer pairs, the first sample question-answer pairs and the probability distribution information, determine the sample question-answer pairs and the The loss value associated with sample question-answer pairs, , is the total number of sample question-answer pairs in the first question-answer pair set, , is the total number of sample question-answer pairs in the second question-answer pair set; determining a loss function according to all loss values; and using the loss function to train the machine learning model.

[0008] In some embodiments, the determining and the sample question-answer pairs and the The loss value associated with the sample question-answer pair includes: calculating the loss value of the machine learning model under the target strategy to generate the first The first generation probability of a sample question-answer pair is calculated by the machine learning model under the target strategy according to the specified text. a second generation probability of a sample question-answer pair; determining, according to the probability distribution information, a first specified proportion of candidate question-answer pairs that can be labeled with the first labeling information among the multiple candidate question-answer pairs, and a second specified proportion of candidate question-answer pairs that can be labeled with the second labeling information among the multiple candidate question-answer pairs; and determining, according to the first generation probability, the first specified proportion, and the second specified proportion used to represent the first labeling information. The sample question-answer pairs are better than the The first winning rate of the sample question-answer pair is used to determine the first sub-loss value; according to the second generation probability, the second specified proportion, and the The sample question-answer pairs are inferior to the The second winning rate of the sample question-answer pair is used to determine the second sub-loss value; and the second sub-loss value is used to determine the second sub-loss value corresponding to the first sub-loss value. sample question-answer pairs and the The loss value associated with each sample question-answer pair.

[0009] In some embodiments, determining the first sub-loss value includes: determining a first intermediate value based on the first generation probability and the first specified proportion; determining a second intermediate value based on the specified learning rate and the first win rate; and determining the first sub-loss value based on the first intermediate value and the second intermediate value.

[0010] In some embodiments, determining the first intermediate value includes: calculating the ratio of the first generation probability and the first specified proportion to obtain a first calculated value; calculating the logarithm of the first calculated value to obtain the first intermediate value; determining the second intermediate value includes: calculating the difference between the first win rate and a predetermined threshold to obtain a second calculated value; calculating the product of the specified learning rate and the second calculated value to obtain the second intermediate value; determining the first sub-loss value includes: calculating the difference between the first intermediate value and the second intermediate value to obtain the first sub-loss value.

[0011] In some embodiments, determining the second sub-loss value includes: determining a third intermediate value based on the second generation probability and the second specified proportion; determining a fourth intermediate value based on the specified learning rate and the second win rate; and determining the second sub-loss value based on the third intermediate value and the fourth intermediate value.

[0012] In some embodiments, determining the third intermediate value includes: calculating the ratio of the second generation probability and the second specified proportion to obtain a third calculated value; calculating the logarithm of the third calculated value to obtain the third intermediate value; determining the fourth intermediate value includes: calculating the difference between the second win rate and a predetermined threshold to obtain a fourth calculated value; calculating the product of the specified learning rate and the fourth calculated value to obtain the fourth intermediate value; determining the second sub-loss value includes: calculating the difference between the third intermediate value and the fourth intermediate value to obtain the second sub-loss value.

[0013] In some embodiments, the determining and the sample question-answer pairs and the The loss value associated with the sample question-answer pair includes: determining the loss value associated with the first sub-loss value according to the square sum of the first sub-loss value and the second sub-loss value sample question-answer pairs and the The loss value associated with each sample question-answer pair.

[0014] In some embodiments, determining the loss function based on all loss values ​​includes: calculating the sum of all loss values ​​to obtain the loss function.

[0015] In some embodiments, the labeling of the multiple candidate question-answer pairs includes: selecting some question-answer pairs from the multiple candidate question-answer pairs as multiple sampled question-answer pairs; and labeling each sampled question-answer pair according to the evaluation result of each sampled question-answer pair in the multiple sampled question-answer pairs to obtain the multiple sample question-answer pairs.

[0016] In some embodiments, selecting some question-and-answer pairs from the multiple candidate question-and-answer pairs includes: performing stratified sampling on the multiple candidate question-and-answer pairs according to the evaluation value in the evaluation result of each candidate question-and-answer pair in the multiple candidate question-and-answer pairs to select the multiple sampled question-and-answer pairs.

[0017] In some embodiments, the evaluation value of each candidate question-answer pair includes the evaluation value of the question in each candidate question-answer pair and the evaluation value of the answer in each candidate question-answer pair.

[0018] In some embodiments, selecting some question-and-answer pairs from the multiple candidate question-and-answer pairs includes: clustering the multiple candidate question-and-answer pairs according to the questions in each candidate question-and-answer pair in the multiple candidate question-and-answer pairs and at least one item of evaluation reason information in the evaluation result of each candidate question-and-answer pair to obtain multiple clustering results; and sampling the multiple clustering results to select the multiple sampled question-and-answer pairs.

[0019] In some embodiments, selecting some question and answer pairs from the multiple candidate question and answer pairs includes: calculating the matching degree of each candidate question and answer pair from the multiple candidate question and answer pairs, wherein the matching degree of each candidate question and answer pair is the degree of matching between the question and the answer in each candidate question and answer pair; calculating the deviation between the matching degree of each candidate question and answer pair and the evaluation value in the evaluation result of each candidate question and answer pair; selecting the multiple sampled question and answer pairs according to at least one of the following sampling methods: sampling from the multiple candidate question and answer pairs whose matching degree is less than a matching degree threshold; sampling from the multiple candidate question and answer pairs whose deviation is less than a deviation degree threshold.

[0020] In some embodiments, labeling the sampled question and answer pairs includes: presenting the multiple sampled question and answer pairs and evaluation results of the multiple sampled question and answer pairs to a user; and receiving labeling information input by the user for each of the multiple sampled question and answer pairs.

[0021] In some embodiments, the probability distribution information input by the user is received.

[0022] In some embodiments, the use of a machine learning model to process a specified text to obtain a plurality of candidate question-answer pairs, and an evaluation result of each of the plurality of candidate question-answer pairs includes: using the machine learning model to process the specified text according to first prompt information for generating question-answer pairs to obtain the plurality of candidate question-answer pairs; and using the machine learning model to process each of the plurality of candidate question-answer pairs according to second prompt information for evaluating question-answer pairs to obtain an evaluation result of each candidate question-answer pair.

[0023] In a second aspect of the present disclosure, a machine learning model training device is provided, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute a machine learning model training method as described in any of the above embodiments based on instructions stored in the memory.

[0024] In a third aspect of the present disclosure, a knowledge base generation method is provided, comprising: processing a text to be processed using a machine learning model to generate a plurality of question-answer pairs, and an evaluation value of each of the plurality of question-answer pairs, wherein the machine learning model is trained by a machine learning model training method as described in any of the above embodiments; and storing question-answer pairs having an evaluation value greater than an evaluation value threshold into a knowledge base.

[0025] In a fourth aspect of the present disclosure, a machine learning model training device is provided, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute instructions stored in the memory to implement a knowledge base generation method as described in any of the above embodiments.

[0026] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0027] In a sixth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0028] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0030] Figure 1 A flowchart of a machine learning model training method according to an embodiment of the present disclosure;

[0031] Figure 2 This is a schematic diagram of prompt information according to an embodiment of the present disclosure;

[0032] Figure 3 This is a schematic diagram of prompt information according to another embodiment of the present disclosure;

[0033] Figure 4 A schematic diagram of the structure of a machine learning model training device according to an embodiment of the present disclosure;

[0034] Figure 5 A schematic diagram of a knowledge base generation method according to an embodiment of the present disclosure;

[0035] Figure 6 A schematic diagram of the structure of a knowledge base generating device according to an embodiment of the present disclosure;

[0036] Figure 7 A schematic diagram of the structure of model training and deployment according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0039] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0040] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0041] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0042] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0043] Figure 1 The following is a flow chart of a machine learning model training method according to an embodiment of the present disclosure. In some embodiments, the following machine learning model training method is performed by a machine learning model training device, including steps 11-13.

[0044] In step 11, the specified text is processed using a machine learning model to obtain multiple candidate question-answer pairs and evaluation results of each of the multiple candidate question-answer pairs.

[0045] For example, the evaluation result of each candidate question-answer pair includes the evaluation value of the candidate question-answer pair and the reason for giving the evaluation value.

[0046] It should be noted here that the machine learning module includes a large language model. For example, the large language model includes ChatGPT, Qwen and other large language models.

[0047] In some embodiments, the step of processing the specified text using a machine learning model includes the following.

[0048] 1) Using a machine learning model to process a specified text according to the first prompt information used to generate a question-answer pair, so as to obtain a plurality of candidate question-answer pairs.

[0049] It should be noted that the first prompt information describes the extraction task in detail according to the application scenario requirements. For example, in the government service scenario, the first prompt information is as follows: Figure 2 shown.

[0050] 2) Using the machine learning model to process each of the multiple candidate question-answer pairs according to the second prompt information used to evaluate the question-answer pair, so as to obtain an evaluation result of each candidate question-answer pair.

[0051] It should be noted that the second prompt information includes the dimensions and precautions for quality assessment in a specific scenario. Figure 3 shown.

[0052] In step 12, based on the evaluation results, multiple candidate question-answer pairs are labeled to generate multiple sample question-answer pairs, each of which has the first labeling information or the second labeling information.

[0053] Since the evaluation result includes the evaluation value of the candidate question-answer pair and the reason for giving the evaluation value, the evaluation result can be referred to when annotating the candidate question-answer pair so as to provide an accurate annotation result.

[0054] For example, if the quality of a sample question-answer pair is high, the sample question-answer pair may be annotated with the first annotation information. Conversely, if the quality of the sample question-answer pair is low, the sample question-answer pair may be annotated with the second annotation information.

[0055] It should be noted here that multiple candidate question-answer pairs can be labeled with the help of a trained machine learning model, or they can be labeled manually.

[0056] It should also be noted here that, according to computing power conditions, all candidate question-answer pairs can be labeled, or only some of the candidate question-answer pairs can be labeled.

[0057] In some embodiments, the step of labeling multiple candidate question-answer pairs includes the following contents.

[0058] 1) Select some question-answer pairs from multiple candidate question-answer pairs as multiple sampled question-answer pairs.

[0059] In some embodiments, some question-answer pairs may be selected in any of the following three ways.

[0060] A) According to the evaluation value in the evaluation result of each candidate question-answer pair in the multiple candidate question-answer pairs, stratified sampling is performed on the multiple candidate question-answer pairs to select multiple sampled question-answer pairs.

[0061] In some embodiments, the evaluation value of each candidate question-answer pair includes the evaluation value of the question in each candidate question-answer pair and the evaluation value of the answer in each candidate question-answer pair. Thus, the diversity of the sampling results can be ensured based on the evaluation value of the question and the evaluation value of the answer.

[0062] B) Clustering the multiple candidate question-answer pairs according to the questions in each candidate question-answer pair in the multiple candidate question-answer pairs and at least one item of the evaluation reason information in the evaluation result of each candidate question-answer pair to obtain multiple clustering results. Next, sampling the multiple clustering results to select multiple sampled question-answer pairs.

[0063] For example, K-means clustering is performed on multiple candidate question-answer pairs to obtain multiple clustering results, and then random sampling is performed based on the multiple clustering results, or outlier sampling is performed in the multiple clustering results.

[0064] C) Calculate the matching degree of each candidate question-answer pair among the multiple candidate question-answer pairs, where the matching degree of each candidate question-answer pair is the matching degree between the question and the answer in each candidate question-answer pair. Calculate the matching degree of each candidate question-answer pair and the deviation degree of the evaluation value in the evaluation result of each candidate question-answer pair. Then, select multiple sampled question-answer pairs according to at least one of the following sampling methods: sampling from multiple candidate question-answer pairs whose matching degree is less than a matching degree threshold; sampling from multiple candidate question-answer pairs whose deviation degree is less than a deviation degree threshold.

[0065] For example, through the above method, a set of question-answer pairs whose content and evaluation are inconsistent, a set of question-answer pairs whose questions and answers do not match, etc. are selected.

[0066] 2) According to the evaluation result of each sampled question-answer pair in the multiple sampled question-answer pairs, each sampled question-answer pair is labeled to obtain multiple sample question-answer pairs.

[0067] In some embodiments, multiple sampled question-answer pairs and evaluation results of the multiple sampled question-answer pairs are presented to a user so that the user can annotate each of the multiple sampled question-answer pairs according to the evaluation results. Next, annotating information for each of the multiple sampled question-answer pairs input by the user is received.

[0068] In some embodiments, probability distribution information for different question-answer pair qualities of a plurality of candidate question-answer pairs input by a user is also received.

[0069] It should be noted here that for question-answer pairs with better quality, there should be a larger generation probability. On the contrary, for question-answer pairs with poorer quality, there should be a smaller generation probability.

[0070] For example, the user may specify that the proportion of question-answer pairs among multiple candidate question-answer pairs that can be labeled as first labeling information corresponding to better quality exceeds 90%, and correspondingly, the proportion of question-answer pairs among multiple candidate question-answer pairs that can be labeled as second labeling information corresponding to poorer quality does not exceed 10%.

[0071] In step 13, the machine learning model is trained based on the probability distribution information of multiple sample question-answer pairs and multiple candidate question-answer pairs for different question-answer pair qualities.

[0072] It should be noted here that the probability distribution information of the qualities of different question-answer pairs of multiple candidate question-answer pairs can be pre-specified or set as needed during the training process.

[0073] In some embodiments, the step of training the machine learning model includes the following.

[0074] 1) Divide multiple sample question-answer pairs into a first sample set and a second sample set, wherein each sample question-answer pair in the first sample set has first labeling information, and each sample question-answer pair in the second sample set has second labeling information.

[0075] That is to say, each sample question-answer pair in the first sample set is a question-answer pair of better quality, and each sample question-answer pair in the second sample set is a question-answer pair of poorer quality.

[0076] 2) Using the first sample set sample question-answer pairs, the first Sample question-answer pairs, the above probability distribution information, determine the Sample question-answer pairs and The loss value associated with sample question-answer pairs, , is the total number of sample question-answer pairs in the first question-answer pair set, , is the total number of sample question-answer pairs in the second question-answer pair set.

[0077] In some embodiments, determining Sample question-answer pairs and The loss values ​​associated with each sample question-answer pair include the following.

[0078] 2.1) Compute the machine learning model to generate the first The first generation probability of a sample question-answer pair.

[0079] For example, if the specified text is , No. The sample question-answer pairs are , the target strategy is , then the first generation probability is .

[0080] 2.2) Compute the machine learning model to generate the first The second generation probability of a sample question-answer pair.

[0081] For example, if the specified text is , No. The sample question-answer pairs are , the target strategy is , then the second generation probability is .

[0082] 2.3) According to the probability distribution information, determine a first specified proportion of candidate question and answer pairs that can be labeled with the first labeling information among the multiple candidate question and answer pairs, and a second specified proportion of candidate question and answer pairs that can be labeled with the second labeling information among the multiple candidate question and answer pairs.

[0083] For example, according to the probability distribution information PY, the first specified proportion of the question-answer pairs with better quality among multiple candidate question-answer pairs can be determined. , and the second specified proportion of poor quality question-answer pairs in multiple candidate question-answer pairs .

[0084] 2.4) Based on the first generation probability, the first specified proportion, and the sample question-answer pairs are better than The first winning rate of sample question and answer pairs is used to determine the first sub-loss value.

[0085] In some embodiments, according to the first generated probability and the first designated proportion , determine the first intermediate value. According to the specified learning rate and first win rate Next, the first sub-loss value is determined based on the first intermediate value and the second intermediate value. .

[0086] For example, calculate the first generation probability and the first designated proportion The first calculation value is obtained by calculating the logarithm of the first calculation value and obtaining the first intermediate value. Calculating the first winning rate and a predetermined threshold The difference between , gets the second calculated value. Calculate the specified learning rate The product of the first intermediate value and the second intermediate value is calculated to obtain the second intermediate value. The difference between the first intermediate value and the second intermediate value is calculated to obtain the first sub-loss value .

[0087] For example, a predetermined threshold is 0.5.

[0088] For example, the first sub-loss value As shown in formula (1).

[0089] ) (1)

[0090] 2.5) Based on the second generation probability, the second specified proportion, and the Sample question-answer pairs are inferior to The second winning rate of sample question and answer pairs is used to determine the second sub-loss value.

[0091] In some embodiments, according to the second generation probability and the second designated proportion , determine the third intermediate value. According to the specified learning rate and second win rate , determine the fourth intermediate value. Determine the second sub-loss value based on the third intermediate value and the fourth intermediate value .

[0092] For example, calculate the second generation probability and the second designated proportion The third calculated value is obtained by calculating the logarithm of the third calculated value and obtaining the third intermediate value. Calculating the second winning rate and a predetermined threshold The difference between , and the fourth calculated value is obtained. Calculate the specified learning rate The product of the third intermediate value and the fourth calculated value is obtained to obtain the fourth intermediate value. The difference between the third intermediate value and the fourth intermediate value is calculated to obtain the second sub-loss value .

[0093] For example, the second sub-loss value As shown in formula (2).

[0094] (2)

[0095] For example, a predetermined threshold is 0.5.

[0096] It should be noted here that the first generation probability and second win rate Between 0 and 1. If the first generation probability is 1, then the second winning rate Is 0. If the first generation probability is 0.7, then the second winning rate is 0.3.

[0097] 2.6) According to the first sub-loss value and the second sub-loss value, determine Sample question-answer pairs and The loss value associated with each sample question-answer pair.

[0098] For example, according to the square sum of the first sub-loss value and the second sub-loss value, determine Sample question-answer pairs and The loss value associated with each sample question-answer pair.

[0099] For example, with Sample question-answer pairs and The loss value associated with the sample question-answer pair As shown in formula (3).

[0100] (3)

[0101] 3) Determine the loss function based on the total loss value.

[0102] For example, calculating the sum of all loss values ​​to get the loss function .

[0103] For example, the loss function As shown in formula (4).

[0104]

[0105] (4)

[0106] 4) Use the loss function to train the machine learning model.

[0107] It should be noted that during the training process, the prompt words can be updated based on the problems found in the annotation stage, mainly involving problems related to extraction rules, such as insufficient answers, missing extracted topics, etc., so as to iterate the prompt words for optimization.

[0108] In the machine learning model training method of the above-mentioned embodiment of the present disclosure, a specified text is processed by using a machine learning model to obtain multiple candidate question-answer pairs and corresponding evaluation results, and the multiple candidate question-answer pairs are labeled according to the evaluation results to generate multiple sample question-answer pairs. Then, based on the probability distribution information of the multiple sample question-answer pairs and the multiple candidate question-answer pairs for different question-answer pair qualities, the machine learning model is trained so that the machine learning model can output question-answer pairs that meet the requirements.

[0109] Figure 4 A schematic diagram of the structure of a machine learning model training device according to an embodiment of the present invention.

[0110] like Figure 4 As shown, the machine learning model training device 40 can be in the form of a general-purpose computing device. The machine learning model training device 40 includes a memory 41, a processor 42, and a bus 43 connecting different system components.

[0111] The memory 41 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one machine learning model training method being executed. The non-volatile storage medium includes, but is not limited to, a disk storage, an optical storage, a flash memory, and the like.

[0112] The processor 42 may be implemented by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, and other discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module may be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or may be implemented by a dedicated circuit that executes corresponding steps.

[0113] For example, the processor 42 is configured to execute instructions stored in the memory to implement the following Figure 1 The method of any one of the embodiments.

[0114] The bus 43 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0115] The interfaces 44, 45, 46 of the machine learning model training device 40 and the memory 41 and the processor 42 can be connected via a bus 43. The input and output interface 44 can provide a connection interface for input and output devices such as a display, a mouse, and a keyboard. The network interface 45 provides a connection interface for various networked devices. The storage interface 46 provides a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0116] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0117] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0118] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0119] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0120] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 1 The method of any one of the embodiments.

[0121] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 1 The method of any one of the embodiments.

[0122] Figure 5 The flowchart of the knowledge base generation method of one embodiment of the present disclosure is as follows. In some embodiments, the knowledge base generation method is executed by a knowledge base generation device, and includes steps 51-52.

[0123] In step 51, the text to be processed is processed using a machine learning model to generate a plurality of question-answer pairs and an evaluation value of each of the plurality of question-answer pairs.

[0124] It should be noted that the machine learning model is Figure 1 The model is obtained by training using the machine learning model training method involved in any of the embodiments.

[0125] In step 52, the question-answer pairs with evaluation values ​​greater than the evaluation value threshold are stored in the knowledge base.

[0126] Figure 6 The present invention is a schematic diagram of the structure of a knowledge base generating device according to an embodiment of the present invention.

[0127] like Figure 6 As shown, the knowledge base generating device includes a memory 61 , a processor 62 , a bus 63 , an input and output interface 64 , a network interface 65 and a storage interface 66 . Figure 6 and Figure 4 The difference is that in Figure 6 In the illustrated embodiment, the processor 62 is configured to execute instructions stored in the memory to implement the following Figure 5 The method of any one of the embodiments.

[0128] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 5 The method of any one of the embodiments.

[0129] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 5 The method of any one of the embodiments.

[0130] The present disclosure is described below by using a specific example.

[0131] like Figure 7 As shown, in the model training stage, the machine learning model is used to process the specified text according to the first prompt information for generating question-answer pairs to obtain multiple candidate question-answer pairs. Next, the machine learning model is used to process each of the multiple candidate question-answer pairs according to the second prompt information for evaluating the question-answer pairs to obtain an evaluation result for each candidate question-answer pair.

[0132] For example, if the file input into the machine learning model involves environmental protection, the candidate question-answer pairs output by the machine learning model and the corresponding evaluation results are shown in Table 1.

[0133]

[0134]

[0135] Table 1

[0136] Next, some question-answer pairs are selected from the multiple candidate question-answer pairs as multiple sampled question-answer pairs. The multiple sampled question-answer pairs and the evaluation results of the multiple sampled question-answer pairs are presented to the user so that the user can mark each sampled question-answer pair according to the evaluation results. For example, the user marks the question-answer pairs with better quality as the first marking information and the question-answer pairs with poorer quality as the second marking information. The user provides the training device with the marking information for each of the multiple sampled question-answer pairs.

[0137] It should be noted here that users can store high-quality question and answer pairs into the knowledge base.

[0138] In some embodiments, the user may also provide the training device with probability distribution information PY of different question-answer pair qualities for multiple candidate question-answer pairs.

[0139] It should be noted here that for question-answer pairs with better quality, there should be a larger generation probability. On the contrary, for question-answer pairs with poorer quality, there should be a smaller generation probability.

[0140] The training device constructs a label set based on user feedback. For example, the label set D = (T, Y [Q,A] , Wg, Lg, PY), where T is the file used for training, Y [Q,A] is a sample question-answer pair, Wg is a set of sample question-answer pairs with good quality, Lg is a set of sample question-answer pairs with poor quality, and PY is the generated distribution of manually annotated question-answer pairs. Based on the probability distribution information PY, the first specified proportion of the question-answer pairs with good quality among multiple candidate question-answer pairs can be determined. , and the second specified proportion of poor quality question-answer pairs in multiple candidate question-answer pairs .

[0141] For example, the user may specify that the proportion of question-answer pairs among multiple candidate question-answer pairs that can be labeled as first labeling information corresponding to better quality exceeds 90%, and correspondingly, the proportion of question-answer pairs among multiple candidate question-answer pairs that can be labeled as second labeling information corresponding to poorer quality does not exceed 10%.

[0142] Next, the loss function is calculated using the above (4), and the machine learning model is trained using the loss function.

[0143] In the model deployment stage, the specified text is processed using the machine learning model according to the first prompt information for generating question-answer pairs to obtain multiple question-answer pairs. Then, each of the multiple question-answer pairs is processed using the machine learning model according to the second prompt information for evaluating the question-answer pairs to obtain an evaluation value for each question-answer pair. Next, the question-answer pairs whose evaluation values ​​are greater than the evaluation value threshold are stored in the knowledge base. In this way, the knowledge base can be generated quickly and conveniently.

[0144] For example, a document related to environmental protection is input into a file of a trained machine learning model. The question-answer pairs output by the machine learning model and the corresponding evaluation results are shown in Table 2.

[0145]

[0146] Table 2

[0147] In some embodiments, the functional unit described above may be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0148] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0149] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A machine learning model training method, comprising: Processing the specified text using a machine learning model to obtain a plurality of candidate question-answer pairs and an evaluation result of each of the plurality of candidate question-answer pairs; According to the evaluation result, annotating the multiple candidate question-answer pairs to generate multiple sample question-answer pairs, each of the multiple sample question-answer pairs having the first annotation information or the second annotation information; The machine learning model is trained according to the probability distribution information of the multiple sample question-answer pairs and the multiple candidate question-answer pairs for different question-answer pair qualities.

2. The machine learning model training method according to claim 1, wherein: The training of the machine learning model comprises: Divide the multiple sample question-answer pairs into a first sample set and a second sample set, wherein each sample question-answer pair in the first sample set has first labeling information, and each sample question-answer pair in the second sample set has second labeling information; Using the first sample set sample question-answer pairs, the first sample question-answer pairs and the probability distribution information, determine the sample question-answer pairs and the The loss value associated with sample question-answer pairs, , is the total number of sample question-answer pairs in the first question-answer pair set, , is the total number of sample question-answer pairs in the second question-answer pair set; Determine the loss function based on all loss values; The machine learning model is trained using the loss function.

3. The machine learning model training method according to claim 2, wherein: The determination and the sample question-answer pairs and the The loss values ​​associated with sample question-answer pairs include: Calculate the machine learning model to generate the first The first generation probability of a sample question-answer pair; Calculate the machine learning model under the target strategy to generate the first The second generation probability of sample question-answer pairs; Determine, according to the probability distribution information, a first specified proportion of candidate question-answer pairs among the multiple candidate question-answer pairs that can be labeled with the first labeling information, and a second specified proportion of candidate question-answer pairs among the multiple candidate question-answer pairs that can be labeled with the second labeling information; According to the first generation probability, the first specified proportion, and the The sample question-answer pairs are better than the The first winning rate of the sample question and answer pairs determines the first sub-loss value; According to the second generation probability, the second specified proportion, and the Sample question-answer pairs are inferior to the The second winning rate of the sample question and answer pairs determines the second sub-loss value; According to the first sub-loss value and the second sub-loss value, determine sample question-answer pairs and the The loss value associated with each sample question-answer pair.

4. The machine learning model training method according to claim 3, wherein: Determining the first sub-loss value comprises: Determining a first intermediate value according to the first generation probability and the first specified proportion; Determining a second intermediate value according to a specified learning rate and the first win rate; The first sub-loss value is determined according to the first intermediate value and the second intermediate value.

5. The machine learning model training method according to claim 4, wherein: Determining the first intermediate value comprises: Calculating a ratio of the first generation probability to the first specified proportion to obtain a first calculated value; Calculating the logarithm of the first calculated value to obtain the first intermediate value; Determining the second intermediate value comprises: Calculating the difference between the first win rate and a predetermined threshold value to obtain a second calculated value; Calculate the product of the specified learning rate and the second calculated value to obtain the second intermediate value; Determining the first sub-loss value includes: The difference between the first intermediate value and the second intermediate value is calculated to obtain the first sub-loss value.

6. The machine learning model training method according to claim 3, wherein: Determining the second sub-loss value comprises: determining a third intermediate value according to the second generation probability and the second specified proportion; determining a fourth intermediate value according to a specified learning rate and the second win rate; The second sub-loss value is determined according to the third intermediate value and the fourth intermediate value.

7. The machine learning model training method according to claim 6, wherein: Determining the third intermediate value comprises: Calculating a ratio of the second generation probability to the second specified proportion to obtain a third calculated value; Calculating the logarithm of the third calculated value to obtain the third intermediate value; Determining the fourth intermediate value comprises: Calculating the difference between the second win rate and a predetermined threshold to obtain a fourth calculated value; Calculate the product of the specified learning rate and the fourth calculated value to obtain the fourth intermediate value; Determining the second sub-loss value comprises: The difference between the third intermediate value and the fourth intermediate value is calculated to obtain the second sub-loss value.

8. The machine learning model training method according to claim 3, wherein: The determination and the sample question-answer pairs and the The loss values ​​associated with sample question-answer pairs include: According to the square sum of the first sub-loss value and the second sub-loss value, determine sample question-answer pairs and the The loss value associated with each sample question-answer pair.

9. The machine learning model training method according to claim 2, wherein: Determining the loss function according to all loss values ​​includes: The sum of all loss values ​​is calculated to obtain the loss function.

10. The machine learning model training method according to claim 1, wherein: The labeling of the plurality of candidate question-answer pairs comprises: Selecting some question-answer pairs from the multiple candidate question-answer pairs as multiple sampled question-answer pairs; According to the evaluation result of each sampled question-answer pair in the multiple sampled question-answer pairs, each sampled question-answer pair is labeled to obtain the multiple sample question-answer pairs.

11. The machine learning model training method according to claim 10, wherein: The selecting some question-answer pairs from the plurality of candidate question-answer pairs comprises: According to the evaluation value in the evaluation result of each candidate question-answer pair in the multiple candidate question-answer pairs, stratified sampling is performed on the multiple candidate question-answer pairs to select the multiple sampled question-answer pairs.

12. The machine learning model training method according to claim 11, wherein: The evaluation value of each candidate question-answer pair includes the evaluation value of the question in each candidate question-answer pair and the evaluation value of the answer in each candidate question-answer pair.

13. The machine learning model training method according to claim 10, wherein: The selecting some question-answer pairs from the plurality of candidate question-answer pairs comprises: Clustering the multiple candidate question-answer pairs according to the question in each candidate question-answer pair in the multiple candidate question-answer pairs and at least one item of the evaluation reason information in the evaluation result of each candidate question-answer pair to obtain multiple clustering results; The plurality of clustering results are sampled to select the plurality of sampled question-answer pairs.

14. The machine learning model training method according to claim 10, wherein: The selecting some question-answer pairs from the plurality of candidate question-answer pairs comprises: Calculating the matching degree of each candidate question-answer pair in the plurality of candidate question-answer pairs, wherein the matching degree of each candidate question-answer pair is the matching degree between the question and the answer in each candidate question-answer pair; Calculating the matching degree of each candidate question-answer pair and the deviation degree of the evaluation value in the evaluation result of each candidate question-answer pair; The plurality of sampled question-answer pairs are selected according to at least one of the following sampling methods: Sampling from a plurality of candidate question-answer pairs whose matching degree is less than a matching degree threshold; Sampling is performed among a plurality of candidate question-answer pairs whose deviation is less than a deviation threshold.

15. The machine learning model training method according to claim 10, wherein: The labeling of the sampled question-answer pairs comprises: Presenting the plurality of sampled question-answer pairs and evaluation results of the plurality of sampled question-answer pairs to a user; Receive annotation information for each sampled question-answer pair in the plurality of sampled question-answer pairs input by the user.

16. The machine learning model training method according to claim 15, further comprising: The probability distribution information input by the user is received.

17. The machine learning model training method according to any one of claims 1 to 16, wherein: The specified text is processed by using a machine learning model to obtain multiple candidate question-answer pairs, and the evaluation result of each candidate question-answer pair in the multiple candidate question-answer pairs includes: Processing the specified text using the machine learning model according to the first prompt information for generating a question-answer pair to obtain the plurality of candidate question-answer pairs; The machine learning model is used to process each candidate question and answer pair in the multiple candidate question and answer pairs according to the second prompt information used to evaluate the question and answer pair, so as to obtain an evaluation result of each candidate question and answer pair.

18. A machine learning model training device, comprising: Memory; A processor, coupled to a memory, wherein the processor is configured to execute instructions stored in the memory to implement a machine learning model training method as described in any one of claims 1-17.

19. A method for generating a knowledge base, comprising: Processing the text to be processed by using a machine learning model to generate a plurality of question-answer pairs and an evaluation value of each of the plurality of question-answer pairs, wherein the machine learning model is trained by the machine learning model training method according to any one of claims 1 to 17; The question-answer pairs with evaluation values ​​greater than the evaluation value threshold are stored in the knowledge base.

20. A knowledge base generating device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the knowledge base generation method according to claim 19 based on instructions stored in the memory.

21. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method according to any one of claims 1-17 and 19 is implemented.

22. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1-17 and 19 is implemented.

Images