Medicine question answering model training method and apparatus, electronic device and storage medium

Abstract

A medicine question answering model training method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring training data comprising a plurality of medicine question answering data sets, the training data comprising labeled data and unlabeled data, and the labeled data comprising preceding information and subsequent information (S101); on the basis of a contextual relationship, configuring the unlabeled data as labeled data (S102); on the basis of the data volume of each data set, setting the extraction probability of data in the data set, the extraction probability of the data in the data set being inversely proportional to the data volume (S103); inputting preceding information in the extracted training data into a medicine question answering model, so as to output a subsequent information vector on an output layer of the medicine question answering model (S104); and, on the basis of the subsequent information vector and subsequent information corresponding to the input preceding information, adjusting parameters of the medicine question answering model (S105).

Description

A drug question-answering model training method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 26, 2023, with application number 202311251723.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of drug question-answering model training, for example, to a drug question-answering model training method, device, electronic device and storage medium. Background Art

[0003] The use of intelligent question-answering robots is a key means for businesses to save labor costs and improve efficiency. They can help companies improve the efficiency and quality of customer service, reduce costs, and enhance their competitiveness. Consequently, intelligent question-answering robots are increasingly being used in the medical sector, specifically pharmaceutical question-answering robots. Based on a large medical language model, these robots are trained through knowledge reinforcement and real-world data, providing services to medical staff and patients in a flexible and intelligent interactive manner.

[0004] The application of the pharmaceutical question-answering robot is based on the pharmaceutical question-answering model. However, when training the pharmaceutical question-answering model, not all tasks have sufficient labeled data in the training data, resulting in the pharmaceutical question-answering robot's low generalization ability in answering questions in various situations, making it difficult to provide high-quality services to patients.

[0005] Summary of the Invention

[0006] This application provides a method for training a pharmaceutical question-answering model to address the problem that not all tasks have sufficient labeled data, resulting in low generalization ability of the pharmaceutical question-answering model and difficulty in providing high-quality services to patients.

[0007] In a first aspect, the present application provides a method for training a drug question-answering model, comprising:

[0008] Acquire training data comprising a plurality of drug question-answering datasets, wherein the training data comprises labeled data and unlabeled data, wherein the labeled data comprises context information and context information;

[0009] Setting the unlabeled data as the labeled data based on the context relationship;

[0010] Setting a probability of extracting data from each data set based on the data volume of each data set, wherein the probability of extracting data from the data set is inversely proportional to the data volume;

[0011] Inputting the context information extracted from the training data into the pharmaceutical question-answering model to output a context information vector at the output layer of the pharmaceutical question-answering model;

[0012] The parameters of the drug question-answering model are adjusted according to the context information vector and the context information corresponding to the input context information.

[0013] In a second aspect, the present application provides a drug question-answering model training device, comprising:

[0014] a training data acquisition module configured to acquire training data comprising a plurality of drug question-answering datasets, wherein the training data comprises labeled data and unlabeled data, wherein the labeled data comprises context information and context information;

[0015] A marking module, configured to set the unmarked data as the marked data based on a context relationship;

[0016] a probability setting module configured to set a probability of extracting data from the data set based on the amount of data in each data set, wherein the probability of extracting data from the data set is inversely proportional to the amount of data;

[0017] a training data input module configured to input context information from the extracted training data into the pharmaceutical question-answering model, so as to output a context information vector at an output layer of the pharmaceutical question-answering model;

[0018] The parameter adjustment module is configured to adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the input context information.

[0019] In a third aspect, the present application provides an electronic device, comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drug question-answering model training method described in the first aspect of this application.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the drug question-answering model training method described in the first aspect of the present application when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG1 is a flow chart of a method for training a drug question-answering model provided in Example 1 of the present application;

[0025] FIG2 is a flow chart of a method for training a drug question-answering model provided in Example 2 of the present application;

[0026] FIG3 is a flow chart of a method for training a drug question-answering model provided in Example 3 of the present application;

[0027] FIG4 is a schematic diagram of the structure of a drug question-answering model training device provided in Example 4 of the present application;

[0028] FIG5 is a schematic structural diagram of an electronic device provided in Example 5 of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] Example 1

[0031] FIG1 is a flow chart of a method for training a drug question-answering model according to the first embodiment of the present application. This embodiment is applicable to drug question-answering model training. The method can be performed by a drug question-answering model training device. The drug question-answering model training device can be implemented in the form of hardware and / or software. The drug question-answering model training device can be configured in an electronic device. As shown in FIG1 , the drug question-answering model training method includes:

[0032] S101. Acquire training data of a data set including multiple drug question-answering categories, where the training data includes labeled data and unlabeled data, and the labeled data includes previous context information and following context information.

[0033] Among them, the amount of unlabeled data is greater than that of labeled data.

[0034] In terms of data type, the information types of training data include disease information and drug information, that is, information related to drug questions and answers.

[0035] A pharmaceutical question-and-answer model is a model that can answer questions posed, including questions about symptom diagnosis, treatment, or drug use. Therefore, the training data used to train the pharmaceutical question-and-answer model includes both symptom information and drug information. This information allows the pharmaceutical question-and-answer model to be infused with the knowledge contained in these information, allowing it to learn and "understand" this knowledge, effectively building a knowledge base for application in pharmaceutical question-and-answering. Labeled data, on the other hand, includes both contextual information and background information. This means that the labeled data is provided with tags (labels), and the contextual information and background information exist in pairs and are interrelated.

[0036] Example 1: The above information is "Weikaning Tablets", and the following information is "Weikaning Tablets have the effects of regulating qi and digesting food, softening the liver and relieving pain. It contains sulfcralfate and berberine hydrochloride (berberine hydrochloride), and has the effects of regulating qi and digesting food, softening the liver and relieving pain. It is mainly used for stomach pain caused by qi stagnation, belching and acid reflux, poor appetite and loss of food."

[0037] Example 2: The above information is "The rhizome of Polygonatum odoratum has certain medicinal value", and the following information is "The "Four-tone Materia Medica" records that Polygonatum odoratum can nourish the middle and replenish Qi, and can be used to treat dry heat cough, loss of body fluid and thirst, yin deficiency and exogenous infection, headache and fever, etc. Polygonatum odoratum slices can be boiled in water and drunk as tea to prevent and treat coronary heart disease, and stewed with meat slices in soup to replenish the body and strengthen the body."

[0038] Regarding datasets, when training a model, multiple datasets are usually used as training data sources, such as the Common Crawl dataset, the Wiki dataset, etc.

[0039] In an optional embodiment, after the training data is obtained, data cleaning may be performed on the training data, including checking data consistency, processing invalid values ​​and missing values, etc., to improve the quality of the training data.

[0040] S102: Setting the unlabeled data as labeled data based on the contextual relationship.

[0041] When training the model, a large amount of unlabeled data is used to assist a small amount of labeled data. For unlabeled data, since the text in the data has contextual relationships, the unlabeled data can be labeled according to the contextual relationships, which can also improve the generalization ability of the model.

[0042] It should be noted that contextual relationships are not limited to the relationship between upper and lower paragraphs, and can be set according to actual needs. For example, contextual relationships can be set based on identifiers, such as space characters (indicating the end of a sentence), paragraph separators (indicating the relationship between upper and lower paragraphs), start symbols, end symbols, etc.

[0043] Exemplary, <bos>The rhizome of Polygonatum odoratum has certain medicinal value <sep>",in, <bos>Represents the start symbol. <sep>Represents the end symbol, for "the rhizome of Polygonatum has certain medicinal value <sep>"For example," <bos>"It can be used as its identifier, which is equivalent to using the relationship between contexts to turn unsupervised data into supervised data. In this way, all unlabeled data can be turned into labeled data, and the data in the training data are all labeled data.

[0044] S103 : setting a probability of extracting data from a data set based on the data volume of each data set, wherein the probability of extracting data from a data set is inversely proportional to the data volume.

[0045] The amount of data in each dataset is different. For example, the Common Crawl dataset has 410 million tokens, accounting for 82% of the total 499 million tokens in the dataset, while the Wiki dataset has 30 million tokens, accounting for 0.6% of the total tokens.

[0046] When training a pharmaceutical Q&A model using training data, data is extracted from various datasets. Since the data volumes of various datasets vary significantly, the larger the dataset, the greater the probability of extracting data from it, leading to a concentration of training data in the larger dataset. The textual representations of each dataset are generally different, and the quality and quantity of pre-training data significantly impact the effectiveness of the pre-trained model. If the training data is concentrated in a large dataset, the trained pharmaceutical Q&A model will be biased towards a single dataset, which can negatively impact the content preference of the pharmaceutical Q&A model's output.

[0047] Therefore, in this application, the extraction probability of data in the data set is set based on the data volume of each data set. The extraction probability of data in the data set is inversely proportional to its own data volume, which can make the data of the data set evenly distributed. The larger the data volume of the data set, the smaller its extraction probability. This can avoid the data used for training being concentrated in data sets with large data volumes, which will have an adverse effect on the content preference output by the drug question-and-answer model. The even distribution of data in the data set also improves the generalization ability of the model, and can provide high-quality services to patients.

[0048] S104. Input the context information in the extracted training data into the drug question-answering model to output the context information vector at the output layer of the drug question-answering model.

[0049] This step is to train the drug question-answering model. Model training is one of the most basic steps in machine learning. It is used to train the model so that it can make accurate predictions about unknown data.

[0050] For example, self-supervised tasks can be constructed using Masked Language Modeling (MLM) or context prediction to improve task performance and computational efficiency. MLM is a natural language processing task that aims to predict the potential values ​​of "masked" (hidden) words in a sentence. To train an MLM model, words are typically randomly selected from the input sentence and replaced with a special "[MASK]" token. The model is then asked to predict the original values ​​of these "masked" words.

[0051] The context information in the extracted training data is input into the drug question-answering model to output the context information vector in the output layer of the drug question-answering model. In this way, the drug question-answering model can be trained to learn and understand disease information and drug information, and can also output the corresponding context after understanding the context. When answering drug questions, corresponding answers can be given based on questions such as disease treatment or drug use.

[0052] S105. Adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the inputted previous context information.

[0053] During the model training process, we need to constantly adjust the model parameters to make the model perform best.

[0054] The context information vector is the feature vector of the output context information. By comparing the output feature vector with the context information corresponding to the input context information, it can be determined whether the parameters of the drug question-answering model are adjusted in place. For example, the context information corresponding to the input context information can also be converted into a vector as a standard vector, and then the standard vector is compared with the output feature vector to obtain a comparison result. Based on the comparison result, the parameters of the drug question-answering model are adjusted until they meet the requirements. For example, the vector difference or loss value between the standard vector and the output feature vector can be calculated to see whether it is less than the corresponding preset vector difference threshold or loss threshold.

[0055] In an optional embodiment, the parameters of the drug question-answering model are adjusted according to the context information vector and the context information corresponding to the input context information, including: converting the context information corresponding to the input context information into a vector to obtain a standard vector; calculating the loss rate based on the context information vector, the standard vector and a preset loss function; judging whether the loss rate is less than a preset loss rate threshold; if so, stopping training to obtain a trained drug question-answering model; if not, adjusting the parameters of the drug question-answering model according to the loss rate, and returning to the step of inputting the context information in the training data into the drug question-answering model to output the context information vector at the output layer of the drug question-answering model. Based on this embodiment, in addition to judging whether the loss rate is less than the preset loss rate threshold, it is also possible to judge whether the decline rate of the loss rate for a preset number of consecutive times is less than the preset decline rate threshold, indicating that the model has been fully trained and that continuing training will have little effect on improving the accuracy of the model, that is, training can be stopped.

[0056] An embodiment of the present application provides a method for training a pharmaceutical question-and-answer model, comprising: obtaining training data from a data set including multiple pharmaceutical question-and-answer categories, the training data including labeled data and unlabeled data, the labeled data including previous context information and following context information, and setting the unlabeled data as labeled data based on the context relationship. The unlabeled data can then be converted into labeled data, thereby improving the generalization ability of the model.

[0057] Then, based on the data volume of each data set, the probability of extracting data from the data set is set. The probability of extracting data from a data set is inversely proportional to its own data volume. The larger the data volume of the data set, the smaller its extraction probability. This can prevent the data used for training from being concentrated in a large data set, which would have a negative impact on the content preference output by the drug question-answering model. The data in the data set is evenly distributed, which also improves the generalization ability of the model and can provide high-quality services to patients. The context information in the training data is input into the drug question-answering model to output a context information vector at the output layer of the drug question-answering model. The parameters of the drug question-answering model are adjusted based on the context information vector and the context information corresponding to the input context information. In this way, the drug question-answering model can be trained to learn and understand drug question-answering data, and can also output the corresponding context after understanding the context. When answering drug questions, it can provide corresponding answers based on questions such as disease treatment or drug use.

[0058] Example 2

[0059] FIG2 is a flow chart of a drug question-answering model training method provided in Example 2 of the present application. This embodiment of the present application is refined based on the above-mentioned Example 1. As shown in FIG2 , the drug question-answering model training method includes:

[0060] S201. Acquire training data including a plurality of drug question-answering datasets, where the training data includes labeled data and unlabeled data, and the labeled data includes previous context information and next context information.

[0061] S202: Set the unlabeled data as labeled data based on the contextual relationship.

[0062] S201 - S202 are similar to S101 - S102 of the first embodiment, and reference may be made to S101 - S102 , which will not be described here.

[0063] S203 : Determine the probability and value of the extraction probability of all data sets for which the extraction probability is to be set, where the initial value of the probability and value is 1.

[0064] S204 . In the data sets for which extraction probabilities are to be set, the data set with the largest amount of data is used as the first data, and the remaining data sets are used as the second data.

[0065] For each data set, an extraction probability is set separately to form the probability distribution of the data set, and the sum of the extraction probabilities of all data sets is 1. During the entire training process, data extraction from each data set is performed according to this probability distribution.

[0066] This embodiment uses a binary search method when setting the extraction probability. First, the data set with the largest data volume is separated out separately as the first data, and the other data sets are collectively used as the second data. The first data and the second data are assigned probabilities. At this time, the data set corresponding to the first data (only a single data set) has been assigned an extraction probability; then the data set in the second data is assigned an extraction probability, and this cycle is repeated, assigning extraction probabilities to the two subjects each time.

[0067] S205 : Setting extraction probabilities of the first data and the second data based on the data volumes of the first data and the second data, wherein the extraction probabilities of the first data and the second data are inversely proportional to the corresponding data volumes.

[0068] The extraction probability of data in the data set is set based on the data volume of each data set so that the data of each data set is evenly distributed. The extraction probability of data in a data set is inversely proportional to its own data volume. The larger the data volume of a data set, the smaller its extraction probability. This can avoid the data used for training being concentrated in a data set with a large data volume, which will have an adverse effect on the content preference output by the drug question-answering model. The even distribution of data in the data set also improves the generalization ability of the model, and can provide high-quality services for patients.

[0069] Exemplarily, setting the extraction probability of the first data and the second data based on the data volume of the first data and the second data includes: calculating the total data volume of the first data and the second data; calculating the first proportion of the data volume of the first data in the total data volume; calculating the difference between 1 and the first proportion to obtain the second proportion; taking the product of the second proportion and the probability sum value as the extraction probability of the first data, and taking the product of the first proportion and the probability sum value as the extraction probability of the second data.

[0070] S206: Determine whether the number of data sets in the second data is greater than 2.

[0071] If yes, execute S207, if no, execute S208.

[0072] S207: Taking the extraction probability of the second data as the probability sum of the extraction probabilities of all data sets for which extraction probabilities are to be set, and returning to the step of determining the probability sum of the extraction probabilities of all data sets for which extraction probabilities are to be set.

[0073] As described above, in the data set to be assigned extraction probabilities, the second data is a collection of other data sets except the data set with the largest data volume. Then the second data may include multiple data sets. If it includes multiple data sets, then these data sets have not yet been assigned extraction probabilities. In this case, the extraction probability of the second data will be used as the probability and value of the extraction probabilities of all data sets to be set for the extraction probabilities, and then the extraction probabilities will continue to be assigned to these data sets.

[0074] In order to clearly illustrate the process of allocating extraction probabilities, the following example is used for illustration:

[0075] The Common Crawl dataset (hereinafter referred to as the cc dataset) has 410 million tokens, accounting for 82% of the total 499 million tokens, that is, 4.1 / 4.99=82%. The Wiki dataset has 30 million tokens. Other datasets are not shown one by one for the time being.

[0076] (1) The cc dataset has the largest data volume, so the cc dataset is used as the first data A1, and other datasets including the Wiki dataset are used as the second data A2. The data volume of the second data A2 is 4.99-4.1=0.89 (unit: 100 million).

[0077] (2) For these two data, since the data volume of the cc data set is too large, the extraction probability of the cc data set is set to: 100%-82%=18%. The allocation of the extraction probability of the cc data set has been completed, and the extraction probability of the second data A2 is 82%.

[0078] (3) The second data A2 includes Wiki datasets and other datasets. If the number of datasets is greater than 2, these datasets are redistributed.

[0079] (4) The total extraction probability of the second data A2 is 82%. For the Wiki dataset, assuming that it is the dataset with the largest data volume in the second data A2, then at this time, the Wiki dataset is the first data B1, and the other datasets of the second data A2 except the Wiki dataset are the second data B2.

[0080] (5) The data volume of the Wiki dataset is 30 million, and the data volume of the second data A2 is 89 million. In the second data A2, the proportion of the Wiki dataset is 0.3 / 0.89=34%, so the extraction probability of the Wiki dataset in the second data A2 is 100%-34%=66%. The distribution of the extraction probability of the Wiki dataset has been completed, and the extraction probability of the second data B2 is 34.

[0081] (6) This cycle continues until all data sets are assigned extraction probabilities.

[0082] S208: Complete the setting of the extraction probability.

[0083] If the second data only includes one data set, then this data set is the second data itself, and the extraction probability of the second data is also the extraction probability of the data set. There is no need to perform probability allocation, and the setting of the extraction probability can be determined to be completed.

[0084] S209: For each text in the training data, number the paragraphs in the text in sequence.

[0085] The training data consists of multiple texts, each of which contains multiple paragraphs. When numbering the paragraphs in the text sequentially, some identifiers can be used to determine the position of the numbering. The numbering position can be the beginning or end of each paragraph. Identifiers can include space characters (indicating the end of a sentence), paragraph separators (indicating the relationship between previous and next paragraphs), start symbols, end symbols, and so on.

[0086] S210 : For each currently numbered paragraph, copy a preset number of bytes at the end of the currently numbered paragraph to the beginning of the next numbered paragraph.

[0087] The preset number of bytes may be 100 bytes. When all bytes in a paragraph are less than 100 bytes, all bytes in the paragraph are copied to the beginning of the next numbered paragraph.

[0088] In text training tasks, truncation is often used to truncate text that exceeds a threshold (usually 2048). However, this approach is wasteful in terms of knowledge infusion. For example, a book contains far more than 2048 tokens. If truncation is used at the beginning, only the first 2048 tokens of each book will be learned. Therefore, the best approach is to segment the article according to seq_len (usually 2048) and feed the segmented vectors to the model for training. To ensure coherence between paragraphs, allow the last 100 tokens of the previous paragraph to appear in the current paragraph.

[0089] For example: the bytes input in the first paragraph are (0-2048), and the last 100 bytes of the first paragraph are (1949-2048), then the bytes input in the second paragraph are (1949-3996), the last 100 bytes of the second paragraph are (3897-3996), and the bytes input in the third paragraph are (3897-5944).

[0090] Copy a preset number of bytes from the end of the currently numbered paragraph to the beginning of the next numbered paragraph. When this data is input into the pharmaceutical question-answering model for learning, the coherence and fluency of the sentences can be learned, thereby improving the quality of the sentences output by the pharmaceutical question-answering model.

[0091] S211. Input the context information in the extracted training data into the drug question-answering model to output the context information vector at the output layer of the drug question-answering model.

[0092] S212. Adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the inputted previous context information.

[0093] S211 - S212 are similar to S104 - S105 of the first embodiment, and reference may be made to S104 - S105 , which will not be described here.

[0094] In this embodiment, on the one hand, the probability of extracting data from a data set is set based on the data volume of each data set so that the data of each data set is evenly distributed. The probability of extracting data from a data set is inversely proportional to its own data volume. The larger the data volume of a data set, the smaller its extraction probability. This can prevent the data used for training from being concentrated in a data set with a large data volume, which would adversely affect the content preference output by the drug question-answering model. The even distribution of data in the data set also improves the generalization ability of the model, and can provide high-quality services to patients. On the other hand, by copying a preset number of bytes from the end of the currently numbered paragraph to the beginning of the next numbered paragraph, when these data are input into the drug question-answering model for learning, the coherence and fluency of the sentences can be learned, thereby improving the quality of the sentences output by the drug question-answering model.

[0095] Example 3

[0096] FIG3 is a flow chart of a drug question-answering model training method provided in Example 3 of the present application. This embodiment of the present application is refined based on the above-mentioned Example 1. As shown in FIG3 , the drug question-answering model training method includes:

[0097] S301. Acquire training data of a data set including multiple drug question-answering categories, where the training data includes labeled data and unlabeled data, and the labeled data includes previous context information and following context information.

[0098] S302: Set the unlabeled data as labeled data based on the contextual relationship.

[0099] S303: Setting a probability of extracting data from each data set based on the data volume of each data set. The probability of extracting data from the data set is inversely proportional to the data volume.

[0100] S304: Input the context information in the extracted training data into the drug question-answering model to output the context information vector at the output layer of the drug question-answering model.

[0101] S305. Adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the inputted previous context information.

[0102] S301 - S305 are similar to S101 - S105 of the first embodiment, and reference may be made to S101 - S105 , which will not be described here.

[0103] S306: Acquire test data, where the test data includes multiple-choice questions related to symptom information and drug information, and standard answers to the multiple-choice questions.

[0104] Model testing is the final step in machine learning, which is used to evaluate the model's performance on unknown data. During the model testing process, a test set is needed to evaluate the model's performance on unknown data to check whether the model meets expectations.

[0105] S307. Input the multiple-choice question into the drug question-answering model to output the multiple-choice answer in the output layer of the drug question-answering model.

[0106] The model's fit to the language template—that is, given a sentence, it predicts which legal, fluent words are likely to appear after it. However, simply generating fluent sentences is insufficient for a drug instruction Q&A bot. To address this, we can create a proficiency test dataset covering a wide range of drug knowledge, such as one containing 8,000 multiple-choice questions.

[0107] Since it is in the form of multiple-choice questions, we can write the question into the prompt and let the model continue to write a token to determine whether the answer to the continued token is the correct answer.

[0108] Example: The following is a multiple-choice question. Please choose the correct answer from the 4 options.

[0109] Question 1: My blood pressure is between 140 and 150, and I have a slight headache. Is this serious?

[0110] A: If your blood pressure is between 140 and 150 and you have a slight headache, we recommend that you visit a doctor for further examination and diagnosis. Your doctor will determine your specific condition based on your symptoms and test results and provide appropriate treatment and advice.

[0111] B: Blood pressure of 140-150 is normal and does not require much attention.

[0112] C: Blood pressure of 140-150 is considered high and is accompanied by a headache. It is recommended that you do not get excited and measure your blood pressure again after a while. If the condition does not improve, it is recommended to seek medical attention in time.

[0113] D: If you are a patient with hypertension, please take relevant medications on time.

[0114] The final answer output by the drug analysis model is the answer option.

[0115] The trained drug question-answering model is tested in the form of multiple-choice questions. Multiple-choice questions usually include multiple options, including at least one correct answer and at least one incorrect interference answer. The drug question-answering model needs to avoid the interference answers to select the correct answer. By judging whether the model can output multiple-choice answers with a certain accuracy rate, it can be judged whether the model has anti-interference ability. It can also be judged whether the model has truly learned and "understood" the knowledge contained in the disease information and drug information in the training data.

[0116] S308. Determine whether the drug question-answering model is overfitting by comparing the selected answers with the standard answers.

[0117] If yes, execute S309; ​​if no, execute S310.

[0118] Overfitting is “analytic results that correspond too closely or precisely to a specific dataset and therefore may not fit other data or reliably predict future observations.” In other words, if the model memorizes a large number of examples rather than learning attentive features, the pharmaceutical question-answering model may struggle to learn how to answer questions outside of the training data.

[0119] Exemplarily, whether the drug question-answering model is overfitting is determined by comparing the selected answer and the standard answer, including: calculating the semantic cosine value of the selected answer and the standard answer; determining whether the semantic cosine value is less than a preset semantic cosine threshold; if so, determining that the drug question-answering model is not overfitting; if not, determining that the drug question-answering model is overfitting.

[0120] S309. Retrain the drug question-answering model.

[0121] If the pharmaceutical question-answering model is determined to be overfitting, it can be retrained. For example, the model parameters can be modified and then retrained, or the amount of training data can be adjusted, such as by increasing the amount of training data before retraining. The retraining process can refer to the relevant content of model training above and will not be described here. The retrained pharmaceutical question-answering model that meets the requirements will then have anti-interference capabilities.

[0122] S310: Determine whether the drug question-answering model training is completed.

[0123] When it is determined that the drug question answering model is not overfitting, it means that the training effect of the drug question answering model is good and it has anti-interference ability. There is no need to retrain, and the training is determined to be completed.

[0124] This embodiment tests the trained drug question-answering model in the form of multiple-choice questions. Multiple-choice questions usually include multiple options, including at least one correct answer and at least one incorrect interference answer. The drug question-answering model needs to avoid the interference answers to select the correct answer. By judging whether the model can output multiple-choice answers with a certain accuracy rate, it can be judged whether the model has anti-interference ability. It can also be judged whether the model has truly learned and "understood" the disease information and drug information in the training data.

[0125] Example 4

[0126] FIG4 is a schematic diagram of the structure of a drug question-answering model training device provided in Example 4 of the present application. As shown in FIG4 , the drug question-answering model training device includes:

[0127] A training data acquisition module 401 is configured to acquire training data comprising a plurality of drug question-answering datasets, wherein the training data comprises labeled data and unlabeled data, wherein the labeled data comprises context information and context information;

[0128] A marking module 402 is configured to set the unmarked data as the marked data based on the context;

[0129] A probability setting module 403 is configured to set the probability of extracting data from each data set based on the data volume of each data set, wherein the probability of extracting data from the data set is inversely proportional to the data volume;

[0130] A training data input module 404 is configured to input the context information extracted from the training data into the pharmaceutical question-answering model to output a context information vector at the output layer of the pharmaceutical question-answering model;

[0131] The parameter adjustment module 405 is configured to adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the input context information.

[0132] In an optional embodiment, the probability setting module 403 includes:

[0133] A probability and value setting submodule is configured to determine the probability and value of the extraction probability of all the data sets for which the extraction probability is to be set, wherein the initial value of the probability and value is 1;

[0134] The data classification submodule is configured to use the data set with the largest amount of data as the first data and the remaining data sets as the second data in the data set for which the extraction probability is to be set.

[0135] a probability setting submodule, configured to set the extraction probability of the first data and the second data based on the data amount of the first data and the second data, wherein the extraction probability of the first data and the second data is inversely proportional to the corresponding data amount;

[0136] A data judgment submodule is configured to judge whether the number of the data sets in the second data is greater than 2; if so, the contents of the probability allocation submodule are executed; if not, the contents of the extraction probability completion submodule are executed;

[0137] a probability allocation submodule, configured to use the extraction probability of the second data as the probability sum of the extraction probabilities of all the data sets for which extraction probabilities are to be set, and return to the step of determining the probability sum of the extraction probabilities of all the data sets for which extraction probabilities are to be set;

[0138] The extraction probability completion submodule is set to complete the setting of the extraction probability.

[0139] In an optional embodiment, the probability setting submodule includes:

[0140] a data volume calculation unit, configured to calculate the total volume of the first data and the second data;

[0141] a first proportion calculation unit, configured to calculate a first proportion of the amount of the first data in the total amount of the data;

[0142] a second proportion calculation unit, configured to calculate a difference between 1 and the first proportion to obtain a second proportion;

[0143] The probability allocation unit is configured to use the product of the second proportion and the probability sum value as the extraction probability of the first data, and use the product of the first proportion and the probability sum value as the extraction probability of the second data.

[0144] In an optional embodiment, the training data includes a plurality of texts, each of which includes a plurality of paragraphs, and the drug question-answering model training device further includes:

[0145] a paragraph numbering module, configured to sequentially number the paragraphs in each text in the training data;

[0146] The paragraph editing module is configured to copy a preset number of bytes at the end of each currently numbered paragraph to the beginning of the next numbered paragraph.

[0147] In an optional embodiment, the parameter adjustment module 405 includes:

[0148] A vector conversion submodule is configured to convert the context information corresponding to the input context information into a vector to obtain a standard vector;

[0149] a loss rate calculation submodule, configured to calculate the loss rate based on the following information vector, the standard vector and a preset loss function;

[0150] A loss rate judgment submodule is configured to judge whether the loss rate is less than a preset loss rate threshold; if so, the contents of the training stop submodule are executed, and if not, the contents of the parameter adjustment submodule are executed;

[0151] A training stop submodule is configured to stop training and obtain the trained drug question-answering model;

[0152] The parameter adjustment submodule is configured to adjust the parameters of the drug question-answering model according to the loss rate, and returns to execute the content of the training data input module 404.

[0153] In an optional embodiment, the drug question-answering model training device further includes:

[0154] A test data acquisition module configured to acquire test data, wherein the test data includes multiple-choice questions on pharmaceutical questions and standard answers to the multiple-choice questions;

[0155] A selection answer output module, configured to input the multiple-choice question into the drug question-answering model to output a selection answer at an output layer of the drug question-answering model;

[0156] An overfitting judgment module is configured to judge whether the drug question-answering model is overfitting by comparing the selected answer with the standard answer; if so, the content of the retraining module is executed, and if not, the content of the training completion determination module is executed;

[0157] A retraining module, configured to retrain the drug question-answering model;

[0158] The training completion determination module is configured to determine that the training of the drug question-answering model is completed.

[0159] In an optional embodiment, the overfitting judgment module includes:

[0160] A semantic cosine value calculation submodule, configured to calculate the semantic cosine values ​​of the selected answer and the standard answer;

[0161] a semantic cosine value judgment submodule, configured to judge whether the semantic cosine value is less than a preset semantic cosine threshold;

[0162] a non-overfitting determination module submodule, configured to determine that the drug question-answering model is not overfitting if the semantic cosine value is less than a preset semantic cosine threshold;

[0163] The overfitting determination module submodule is configured to determine that the drug question-answering model is overfitting if the semantic cosine value is greater than or equal to a preset semantic cosine threshold.

[0164] The drug question-answering model training device provided in the embodiments of the present application can execute the drug question-answering model training method provided in any embodiment of the present application, and has functional modules corresponding to the execution method.

[0165] Example 5

[0166] Fig. 5 shows the block diagram of the electronic device 40 that can be used to implement the embodiment of the application.The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones and other similar computing devices.The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or required herein.

[0167] As shown in Figure 5, the electronic device 40 includes at least one processor 41 and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42 and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0168] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0169] The processor 41 can be a variety of general and / or specialized processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the drug question-answering model training method.

[0170] In some embodiments, the drug question and answer model training method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the drug question and answer model training method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the drug question and answer model training method in any other appropriate manner (for example, by means of firmware).

[0171] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0172] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0173] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM (Erasable Programmable Read-Only Memory) or a flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0175] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0176] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.

[0177] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0178] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.< / bos> < / sep> < / sep> < / bos> < / sep> < / bos>

Claims

1. A drug question-answering model training method, comprising: Acquire training data including a plurality of drug question-answering data sets, wherein the training data includes labeled data and unlabeled data, and the labeled data includes context information and context information; Setting the unlabeled data as the labeled data based on the context relationship; Setting a probability of extracting data from each data set based on the data volume of each data set, wherein the probability of extracting data from the data set is inversely proportional to the data volume; Inputting the context information extracted from the training data into the drug question-answering model to output a context information vector at the output layer of the drug question-answering model; The parameters of the drug question-answering model are adjusted according to the context information vector and the context information corresponding to the inputted previous information.

2. The drug question-answering model training method according to claim 1, wherein: The step of setting the probability of extracting data from the data set based on the data volume of each data set includes: Determine the probability and value of the extraction probability of all the data sets whose extraction probabilities are to be set, where the initial value of the probability and value is 1; Among the data sets for which the extraction probability is to be set, the data set with the largest amount of data is used as the first data, and the remaining data sets are used as the second data, Setting extraction probabilities of the first data and the second data based on the data amounts of the first data and the second data, wherein the extraction probabilities of the first data and the second data are inversely proportional to the corresponding data amounts; Determine whether the number of the data sets in the second data is greater than 2; If so, taking the extraction probability of the second data as the probability and value of the extraction probabilities of all the data sets whose extraction probabilities are to be set, and returning to the step of determining the probability and value of the extraction probabilities of all the data sets whose extraction probabilities are to be set; If not, complete the setting of the extraction probability.

3. The drug question-answering model training method according to claim 2, wherein: The step of setting the extraction probability of the first data and the second data based on the data amount of the first data and the second data comprises: Calculating the total amount of the first data and the second data; Calculate a first proportion of the amount of the first data in the total amount of the data; Calculate the difference between 1 and the first proportion to obtain a second proportion; The product of the second proportion and the probability sum value is used as the extraction probability of the first data, and the product of the first proportion and the probability sum value is used as the extraction probability of the second data.

4. The drug question-answering model training method according to claim 1, wherein: The training data includes a plurality of texts, each of which includes a plurality of paragraphs. Before inputting the context information in the training data into the drug question-answering model so as to output the context information vector at the output layer of the drug question-answering model, the method further includes: For each of the texts in the training data, number the paragraphs in the text in sequence; For each currently numbered paragraph, a preset number of bytes at the end of the currently numbered paragraph are copied to the beginning of the next numbered paragraph.

5. The drug question-answering model training method according to claim 1, wherein: The adjusting the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the inputted context information includes: Convert the context information corresponding to the input context information into a vector to obtain a standard vector; Calculate the loss rate based on the following information vector, the standard vector and a preset loss function; Determining whether the loss rate is less than a preset loss rate threshold; If yes, stop training and obtain the trained drug question-answering model; If not, adjust the parameters of the drug question and answer model according to the loss rate, and return to the step of inputting the context information in the training data into the drug question and answer model to output the context information vector in the output layer of the drug question and answer model.

6. The pharmaceutical question-answering model training method according to any one of claims 1 to 5, further comprising: after adjusting the parameters of the pharmaceutical question-answering model according to the context information vector and the context information corresponding to the input context information: Acquire test data, wherein the test data includes multiple-choice questions on drug questions and standard answers to the multiple-choice questions; Inputting the multiple-choice question into the drug question-answering model to output a multiple-choice answer at an output layer of the drug question-answering model; Determining whether the drug question-answering model is overfitting by comparing the selected answer with the standard answer; If yes, retrain the drug question answering model; If not, it is determined that the training of the drug question-answering model is completed.

7. The drug question-answering model training method according to claim 6, wherein: The step of comparing the selected answer with the standard answer to determine whether the drug question-answering model is overfitting includes: Calculating the semantic cosine value of the selected answer and the standard answer; Determining whether the semantic cosine value is less than a preset semantic cosine threshold; If so, determining that the drug question-answering model is not overfitting; If not, it is determined that the drug question-answering model is overfitting.

8. A drug question-answering model training device, comprising: A training data acquisition module, configured to acquire training data of a data set including multiple drug question-answering categories, wherein the training data includes labeled data and unlabeled data, and the labeled data includes context information and context information; A marking module, configured to set the unmarked data as the marked data based on a context relationship; A probability setting module, configured to set the probability of extracting data from the data set based on the data volume of each data set, wherein the probability of extracting data from the data set is inversely proportional to the data volume; A training data input module, configured to input the context information extracted from the training data into the drug question-answering model, so as to output a context information vector at an output layer of the drug question-answering model; The parameter adjustment module is configured to adjust the parameters of the drug question-answering model according to the context information vector and the context information corresponding to the input context information.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the drug question-and-answer model training method described in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the drug question-answering model training method according to any one of claims 1 to 7 when executed.

Images