Infectious disease question and answer method and system based on meta-learning
By constructing an infectious disease knowledge map and meta-learning question and answer model, the rapid adaptation and generalization of the infectious disease question and answer system are solved, and personalized question and answer and continuous optimization are achieved.
Patent Information
- Application Number
- CN202510362445.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-26
- Publication Date
- 2025-07-18
AI Technical Summary
The existing infectious disease question and answer system has strong data dependence, and it is difficult to quickly adapt to emerging infectious diseases and their mutations, and has limited generalization ability and lacks personalized response ability and self-learning ability.
Build an infectious disease knowledge graph, generate a training set and test set of Q&A data, train through meta-learning Q&A model, use gradient mean to adjust the number of layers of the network, and optimize the Q&A model based on user feedback.
The model's rapid adaptability and generalization ability has been improved, users' personalized Q&A is realized, and the quality of Q&A is optimized through continuous learning mechanisms.
Smart Images
Figure CN120336461A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of infectious diseases, and particularly to an infectious disease question and answer method and system based on meta-learning. Background Art
[0002] Timely and accurate acquisition of information related to infectious diseases is crucial for the prevention and control of epidemics. Traditional methods for obtaining infectious disease information mainly rely on expert lectures, news reports, and books and materials, which have problems such as slow information update, limited coverage, and poor user interactivity. In recent years, question and answer systems based on natural language processing (NLP) have gradually emerged. However, in the field of infectious diseases, existing systems often rely on a large amount of labeled data for training, making it difficult to quickly adapt to newly emerging infectious diseases and their mutations, and lacking the intelligent response ability to meet the personalized needs of different users. In summary, the existing technologies have the following defects and deficiencies: strong data dependence: existing infectious disease question and answer systems mostly rely on a large amount of labeled data for training, and the response speed to newly emerging infectious diseases is slow; poor generalization ability: the system is difficult to quickly adapt to the rapid update and mutation of infectious disease knowledge, and the generalization ability is limited; lack of personalization: it is impossible to provide accurate and differentiated answers according to the personalized needs of users; lack of self-learning ability: the system does not have the ability of autonomous learning and optimization, and it is difficult to continuously improve the quality of question and answer. Based on this, it is necessary to propose an infectious disease question and answer method and system based on meta-learning to solve the above technical problems. Summary of the Invention
[0003] The technical problem to be solved by the present invention is how to improve the rapid adaptation ability and generalization ability of the model through meta-learning technology and realize personalized question and answer for users. The purpose is to provide an infectious disease question and answer method and system based on meta-learning, which solves the above technical problems.
[0004] The present invention is achieved through the following technical solutions:
[0005] An infectious disease question and answer method based on meta-learning, comprising:
[0006] Generating an infectious disease knowledge graph; the entity types of the above infectious disease knowledge graph include infectious diseases, pathogens, symptoms, transmission routes, prevention measures, treatment drugs, and medical institutions; entity attribute identifiers are added to the above entity types; the relationship types of the above entity types take infectious diseases as the root node, and pathogens, symptoms, transmission routes, prevention measures, treatment drugs, and medical institutions as sub-nodes;
[0007] Collecting multiple groups of question and answer pair data of multiple infectious disease types and dividing them into a training set and a test set; both the above training set and the above test set contain all the above infectious disease types;
[0008] Construct a meta-learning Q&A model; train the meta-learning Q&A model according to each of the above infectious disease types using the above training set;
[0009] According to the average gradients trained for multiple above-mentioned infectious disease types, sequentially increase the number of network layers; change the average of the two largest gradient means to the final gradient of the network layer of the above meta-learning Q&A model, and the average of the corresponding two numbers of layers to the final number of layers, and use the above test set to train the above meta-learning Q&A model;
[0010] Use multiple query questions to respectively obtain the above-mentioned entity types and relationship types related in the above knowledge graph; input each of the above query questions, the above entity types and relationship types into the above meta-learning Q&A model, and return the output results to the corresponding user interface; select the results with positive user feedback and add them to the Q&A pair data for training each of the above meta-learning Q&A models.
[0011] The method for constructing the above meta-learning Q&A model includes:
[0012] The network layer of the meta-learning Q&A model includes an embedding layer, an LN layer, and a Feedforward layer sequentially arranged from bottom to top; the above embedding layer is used to convert the above Q&A pair data into a matrix; the above LN layer is used to perform layer normalization on the data output by the above embedding layer; the above Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the above LN layer.
[0013] The above embedding layer is used to convert the above Q&A pair data into a matrix, expressed as:
[0014] ;
[0015] where x represents the digital id representation of the above Q&A pair data; W represents the weight matrix of the embedding layer; T represents matrix transpose; b represents the bias weight matrix of the embedding layer.
[0016] The above LN layer is used to perform layer normalization on the data output by the above embedding layer, expressed as:
[0017] ;
[0018] where x represents the digital id representation of the above Q&A pair data; represents the mean of the current input data x; Var(x) represents the variance of the input data x; is a hyperparameter used to prevent the denominator from being zero, with a value of 0.0001; represents the weight matrix of the LN layer; β represents the bias weight matrix of the LN layer.
[0019] The above Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the above LN layer, expressed as:
[0020] ;
[0021] ;
[0022] ;
[0023] ;
[0024] Among them, the above Feedforward layer includes a query layer, a key layer, and a value layer; q is the data output by the query layer; k is the data output by the key layer; v is the data output by the value layer; is the data output after the calculation of the Feedforward layer; is the data output by the LN layer; , , are the weight matrices of the query layer, the key layer, and the value layer respectively; b1, b2, and b3 are the bias weight matrices of the query layer, the key layer, and the value layer respectively.
[0025] The above method for constructing a meta-learning question-answering model further includes:
[0026] The network layer of the above meta-learning question-answering model further includes a softmax layer; the above softmax layer normalizes the data output by the above Feedforward layer to obtain the final output, expressed as:
[0027] ;
[0028] Among them, represents the i-th data output by the Feedforward layer; represents the data output after passing through the softmax layer; represents the j-th data output by the Feedforward layer.
[0029] When the average of the means of the two largest changed gradients is the final gradient of the network layer of the above meta-learning question-answering model, update the weight matrices and bias weight matrices of the embedding layer, the LN layer, and the Feedforward layer according to the final gradient.
[0030] Updating the weight matrices and bias weight matrices of the embedding layer, LN layer, and Feedforward layer according to the final gradients is expressed as:
[0031] ;
[0032] where, is the weight matrix or bias weight matrix updated according to the final gradient of the network layer; is the weight matrix or bias weight matrix after random initialization; is the final gradient of the network layer; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
[0033] Training the above meta - learning Q&A model using the above test set includes:
[0034] The above test set is divided into multiple batches according to the above infectious disease types. When each batch is trained, the weights of the network layer of the above meta - learning Q&A model are updated according to the sum of the gradient values of all batches of the above infectious disease types, expressed as:
[0035] ;
[0036] where, is the weight of the network layer of the above updated meta - learning Q&A model; is the weight of the network layer of the above meta - learning Q&A model before update; is the total number of training batches; 、 、 are the gradient values after each batch is trained; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
[0037] A meta - learning - based infectious disease Q&A system includes:
[0038] A knowledge graph module for generating an infectious disease knowledge graph; the entity types of the above infectious disease knowledge graph include infectious diseases, pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions; entity attribute identifiers are added to the above entity types; the relationship types of the above entity types have infectious diseases as the root node and pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions as child nodes;
[0039] The data acquisition module collects multiple groups of Q&A pair data for multiple infectious disease types and divides them into a training set and a test set; both the above-mentioned training set and the above-mentioned test set include all the above-mentioned infectious disease types;
[0040] The model construction module constructs a meta-learning Q&A model; the above-mentioned training set trains the above-mentioned meta-learning Q&A model according to each of the above-mentioned infectious disease types;
[0041] The model optimization module sequentially increases the number of network layers according to the average gradient of training for multiple above-mentioned infectious disease types; changes the average of the two largest gradient means to the final gradient of the network layer of the above-mentioned meta-learning Q&A model, and the average of the corresponding two layer numbers is the final number of layers; updates the weights of the network layer of the above-mentioned meta-learning Q&A model according to the final gradient, and the above-mentioned test set trains the above-mentioned meta-learning Q&A model according to the updated weights and number of layers of the network layer;
[0042] The model feedback module respectively obtains the relevant above-mentioned entity types and above-mentioned relationship types in the above-mentioned knowledge graph by using multiple query questions; inputs each of the above-mentioned query questions, the above-mentioned entity types and the above-mentioned relationship types into the above-mentioned meta-learning Q&A model, and returns the output results to the corresponding user interface; selects the results of positive user feedback and adds them to the above-mentioned Q&A pair data for training each of the above-mentioned meta-learning Q&A models.
[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0044] The present invention provides an infectious disease Q&A method based on meta-learning, which takes infectious diseases, pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions as entities; takes infectious diseases as the root node of the entity relationship, and takes pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions as sub-nodes; generates an infectious disease knowledge graph from entities, entity attributes, and entity relationships; divides Q&A pair data of multiple infectious disease types into a training set and a test set; constructs meta-learning Q&A models for multiple infectious disease types; uses the training set to train the meta-learning Q&A models, calculates the average gradient of multiple infectious disease types, and calculates the final number of layers and gradient of the network layer according to the change of the gradient mean; thus uses the final number of layers and gradient as the initial weights for training the meta-learning Q&A model with the test set, and updates the weight matrix of the model using the trained gradient values; inputs the validation set into the meta-learning Q&A model, and selects the results of positive user evaluations and adds them to the Q&A pair data to train each of the meta-learning Q&A models. The present invention improves the fast adaptation ability and generalization ability of the model through meta-learning technology, realizes personalized Q&A for users in combination with user images, and at the same time introduces a continuous learning mechanism to ensure the continuous optimization of Q&A quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings. In the drawings:
[0046] Figure 1 It is a flowchart of the method for answering infectious disease questions based on meta-learning in the embodiments of the present application;
[0047] Figure 2 It is a schematic structural diagram of the meta-learning question-answering model in the embodiments of the present application. Detailed implementation manners
[0048] To make the purpose, technical solutions, and advantages of the present invention clearer and more understandable, the following will further elaborate on the present invention in combination with the embodiments and the drawings. The illustrative embodiments of the present invention and their descriptions are only used to explain the present invention and do not limit the present invention.
[0049] Embodiment
[0050] As Figure 1 shown, the embodiments of the present application provide a method for answering infectious disease questions based on meta-learning, including:
[0051] Generate an infectious disease knowledge graph; the entity types of the above infectious disease knowledge graph include infectious diseases, pathogens, symptoms, transmission routes, prevention measures, treatment drugs, and medical institutions; entity attribute identifiers are added to the above entity types; the relationship types of the above entity types take infectious diseases as the root node and pathogens, symptoms, transmission routes, prevention measures, treatment drugs, and medical institutions as the sub-nodes;
[0052] Collect multiple groups of question-and-answer pair data of multiple infectious disease types and divide them into a training set and a test set; both the above training set and the above test set contain all the above infectious disease types;
[0053] Build a meta-learning question-answering model; the above training set trains the above meta-learning question-answering model according to each of the above infectious disease types;
[0054] According to the gradient means trained for multiple above infectious disease types, sequentially increase the number of network layers; change the average of the two largest gradient means to the final gradient of the network layer of the above meta-learning question-answering model, and the average of the corresponding two numbers of layers is the final number of layers, and use the above test set to train the above meta-learning question-answering model;
[0055] Use multiple query questions to separately obtain the relevant entity types and relationship types in the above knowledge graph; input each of the above query questions, the above entity types, and the above relationship types into the above meta-learning question-answering model, and return the output results to the corresponding user interface; select the results with positive user feedback and add them to the above question-answer pair data for training each of the above meta-learning question-answering models.
[0056] The above method for constructing a meta-learning question-answering model includes:
[0057] As Figure 2 shown in the network layer of the meta-learning question-answering model, it includes an embedding layer, an LN layer, and a Feedforward layer sequentially arranged from bottom to top; the above embedding layer is used to convert the above question-answer pair data into a matrix; the above LN layer is used to perform layer normalization operations on the data output by the above embedding layer; the above Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the above LN layer.
[0058] The above embedding layer is used to convert the above question-answer pair data into a matrix, expressed as:
[0059] ;
[0060] where x represents the digital id representation of the above question-answer pair data; W represents the weight matrix of the embedding layer; T represents matrix transpose; b represents the bias weight matrix of the embedding layer.
[0061] The above LN layer is used to perform layer normalization operations on the data output by the above embedding layer, expressed as:
[0062] ;
[0063] where x represents the digital id representation of the above question-answer pair data; represents the mean of the current input data x; Var(x) represents the variance of the input data x; is a hyperparameter used to prevent the denominator from being zero, with a value of 0.0001; represents the weight matrix of the LN layer; β represents the bias weight matrix of the LN layer.
[0064] The above Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the above LN layer, expressed as:
[0065] ;
[0066] ;
[0067] ;
[0068] ;
[0069] Among them, the above Feedforward layer includes a query layer, a key layer, and a value layer; q is the data output by the query layer of the input layer; k is the data output by the key layer of the intermediate layer; v is the data output by the value layer of the output layer; is the data output after the calculation of the Feedforward layer; is the data output by the LN layer; , , are the weight matrices of the query layer, the key layer, and the value layer respectively; b1, b2, and b3 are the bias weight matrices of the query layer, the key layer, and the value layer respectively.
[0070] The above method for constructing a meta-learning question-answering model further includes:
[0071] The network layer of the above meta-learning question-answering model further includes a softmax layer; the above softmax layer normalizes the data output by the above Feedforward layer to obtain the final output, which is expressed as:
[0072] ;
[0073] Among them, represents the i-th data output by the Feedforward layer; represents the data output after passing through the softmax layer; represents the j-th data output by the Feedforward layer.
[0074] When the average of the means of the two gradients with the largest changes is the final gradient of the network layer of the above meta-learning question-answering model, update the weight matrices and bias weight matrices of the embedding layer, the LN layer, and the Feedforward layer according to the final gradient.
[0075] The above updating the weight matrices and bias weight matrices of the embedding layer, the LN layer, and the Feedforward layer according to the final gradient is expressed as:
[0076] ;
[0077] Among them, is the weight matrix or bias weight matrix updated according to the final gradient of the network layer; is the weight matrix or bias weight matrix after random initialization; is the final gradient of the network layer; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
[0078] Training the above-mentioned meta-learning Q&A model using the above-mentioned test set includes:
[0079] The above-mentioned test set is divided into multiple batches according to the above-mentioned infectious disease types. When each batch is trained, the weights of the network layer of the above-mentioned meta-learning Q&A model are updated according to the sum of the gradient values of all batches of the above-mentioned infectious disease types, expressed as:
[0080] ;
[0081] wherein, are the weights of the network layer of the above-mentioned updated meta-learning Q&A model; are the weights of the network layer of the above-mentioned meta-learning Q&A model before update; is the total number of training batches; , , are the gradient values after each batch is trained; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
[0082] The embodiments of this application construct data in two parts: on the one hand, collecting Q&A pair data of multiple infectious disease types as model training data meta_data, and on the other hand, using the infectious disease knowledge graph generated by entity types, entity attributes, and entity relationships.
[0083] Among them, the basic examples of Q&A pair data are as follows:
[0084] Example 1:
[0085] Question: What is the ranking of the number of tuberculosis patients in China in the world currently?
[0086] Answer: The number of tuberculosis patients in China currently ranks second in the world and is one of the 22 high-burden tuberculosis countries in the world.
[0087] Example 2:
[0088] Question: What diseases are included in Class A infectious diseases?
[0089] Answer: Class A infectious diseases include plague and cholera.
[0090] The following are some examples of infectious diseases:
[0091] Viral infectious diseases (P1): such as influenza, Japanese encephalitis, measles, chickenpox, viral hepatitis, etc.
[0092] Bacterial infectious diseases (P2): such as typhoid fever, paratyphoid fever, bacillary dysentery, cholera, epidemic cerebrospinal meningitis, etc.
[0093] Infectious diseases caused by other pathogens (P3): such as rickettsiosis (epidemic typhus, endemic typhus, etc.), spirochetosis (leptospirosis, syphilis, etc.), protozoal infectious diseases (malaria, amoebiasis, etc.) and helminthic infectious diseases (ascariasis, pinworm disease, schistosomiasis, hookworm disease, filariasis, etc.).
[0094] Respiratory infectious diseases (P4): The pathogen is transmitted through the air or droplets, such as influenza, measles, chickenpox, etc.
[0095] Digestive tract infectious diseases (P5): The pathogen is transmitted through food or water, such as bacillary dysentery, typhoid fever, etc.
[0096] Blood infectious diseases (P6): The pathogen is transmitted through blood or body fluids, such as hepatitis B, AIDS, etc.
[0097] Skin infectious diseases (P7): The pathogen is transmitted through contact, such as rabies, tetanus, etc.
[0098] Suppose there are n types of infectious diseases P1, P2, P3...Pn. For each type of infectious disease, 20 pairs of question-and-answer data are collected according to the data format of the question-and-answer pair data example. For example, for influenza, Japanese encephalitis, measles, chickenpox, and viral hepatitis under viral infectious diseases (P1), 20 relevant question-and-answer data pairs are collected for each infectious disease type. After the data collection is completed, the data is split into two parts, meta_training and test set meta_test, according to the ratio of 8:2. For the meta_training part of the data, we continue to split it into support set data and query set data. Among them, for the 20 pairs of question-and-answer data constructed for each infectious disease type in meta_training, 15 pieces of data can be split out, and these data are combined as the training set and named support set data, and the remaining 5 pieces of data are extracted into multiple query questions and named query set. After removing the 15 pieces of data used to construct the support set from the 20 pairs of question-and-answer data constructed for each infectious disease type in meta_training, the data for the test set meta_test can be named fintune set data.
[0099] Construct an infectious disease knowledge graph in a manual way. The infectious disease knowledge graph is defined as a structured knowledge base, aiming to systematically organize and represent various entities, attributes related to infectious diseases, and the relationships between them. When constructing the infectious disease knowledge graph, it is necessary to clarify the entity types and relationship types in order to effectively store and query relevant information.
[0100] The attributes of various entity types are as follows:
[0101] Infectious disease: refers to a class of diseases caused by various pathogens that can be transmitted between humans, between animals, or between humans and animals. For example, plague, cholera, AIDS, tuberculosis, rabies, etc.
[0102] Pathogen: the microorganism or parasite that causes an infectious disease. For example, Yersinia pestis, Vibrio cholerae, Mycobacterium tuberculosis, HIV virus, rabies virus, etc.
[0103] Symptom: the abnormal phenomena shown by an infectious disease in a patient. For example, fever, cough, rash, vomiting, diarrhea, etc.
[0104] Transmission route: the route through which a pathogen is discharged from an infectious source and invades another susceptible organism. For example, airborne droplet transmission, contact transmission, fecal-oral transmission, blood transmission, etc.
[0105] Preventive measure: various measures taken to prevent the occurrence or spread of an infectious disease. For example, vaccination, frequent handwashing, wearing masks, maintaining social distance, etc.
[0106] Therapeutic drug: a drug used to treat an infectious disease. For example, anti-tuberculosis drugs, antiviral drugs, antibacterial drugs, etc.
[0107] Patient: a person or animal suffering from an infectious disease.
[0108] Medical institution: an institution that provides services for the diagnosis, treatment, and prevention of infectious diseases. For example, hospitals, clinics, disease control and prevention centers, etc.
[0109] The relationship attributes of various relationship types are as follows:
[0110] Pathogen - Infectious disease: indicates that a certain pathogen is the root cause of a certain infectious disease. For example, "Yersinia pestis - Plague".
[0111] Infectious disease - Symptom: indicates which symptoms will appear due to a certain infectious disease. For example, "Cholera - Severe diarrhea".
[0112] Infectious disease - Transmission route: indicates through which routes a certain infectious disease is transmitted. For example, "Influenza - Airborne droplet transmission", "Tuberculosis - Respiratory transmission".
[0113] Infectious diseases - Preventive measures: It represents the measures that can be taken to prevent a certain infectious disease. For example, "Influenza - Wash hands frequently".
[0114] Infectious diseases - Therapeutic drugs: It represents the drugs used to treat a certain infectious disease. For example, "Tuberculosis - Antituberculosis drugs".
[0115] Patients - Infectious diseases: It represents the types of infectious diseases that a patient has. For example, "Zhang San - Influenza".
[0116] Such as Figure 2 The structure of the defined meta - learning Q&A model is shown as follows. The calculation of the embedding layer is:
[0117] ;
[0118] Among them, x represents the digital id representation of the Q&A pair data input to the embedding layer, W represents the weight matrix of the embedding layer, T represents matrix transpose, and b represents the bias weight matrix of the embedding layer.
[0119] The conversion process of the digital id of the above - mentioned Q&A pair data is as follows: First, construct a conversion table, assign a unique digital id to each group of strings in advance, and then for the input data, look up the corresponding character id in this table. The weight matrix is a numerical matrix with a custom dimension, and the values inside are obtained by randomly assigning values. The bias weight matrix is obtained in the same way.
[0120] The LN layer represents the layer normalization operation performed after the embedding transformation. The calculation of the LN layer is:
[0121] ;
[0122] Among them, x represents the current input data of the LN layer, represents the mean of the current input data, Var(x) represents the variance of the input x, is a hyperparameter to prevent the denominator from being zero, and the value is set to 0.0001, represents the weight matrix of the LN layer, and β represents the bias weight matrix of the LN layer.
[0123] The weight matrix of the embedding layer is used to calculate the embedding vector representation of the training data; the weight matrix of the LN layer is used to calculate the result after normalizing the input data of the LN layer; the bias weight matrix of the LN layer is used to participate in the calculation of the LN layer.
[0124] The Feedforward layer encapsulates multi - dimensional matrix calculations. In the Feedforward layer, the calculation process is:
[0125] ;
[0126] ;
[0127] ;
[0128] ;
[0129] Among them, is the output result after passing through the LN layer, , , are the weight matrices of the q layer, k layer, and v layer respectively, and b1, b2, and b3 are the bias weight matrices of the q layer, k layer, and v layer respectively, is the final output calculated by the Feedforward layer. Among them, the number of layers of FeedForward is not a fixed value and will be dynamically adjusted during the training process. The specific adjustment process can be seen in the following meta-learning training part.
[0130] After that, it is calculated through the softmax layer. Softmax represents the normalization layer, and the calculation is:
[0131] ;
[0132] represents the final output of the softmax layer, represents the exponential function with the natural constant e as the base, where i and j represent the i-th and j-th inputs respectively, represents the cumulative calculation.
[0133] After defining the model structure, the weight matrices in the model are randomly initialized, including the embedding layer, LN layer, and Feedforward layer, and an initial weight value is assigned, denoted as w_o. Here, w_o represents the weight matrix and bias weight matrix in the meta-learning question-answering model. The model is trained using the support set data. The number of categories of infectious diseases in the support set data is denoted as m, and the number of categories in the support set is the same as that in meta-training. For the data training under each category of infectious diseases among the m infectious diseases, the gradient value used to update the model weights is obtained, and this gradient value is denoted as g. For the m categories of infectious diseases, after sequential training, m gradient values: g1, g2, g3...gm are obtained, and the following summation and averaging calculation are performed:
[0134] ;
[0135] Among them, g_mean is the calculation result after summation and averaging, regarded as the average value of the gradients obtained after m infectious disease trainings.
[0136] Step 2: Assume that the number of layers of each network layer of the model, such as the embedding layer, LN layer, and FeedForward layer, is N layers. Construct a total of N models from layer 1 to layer N, and perform the calculation of the first step respectively to obtain N average gradient values: g_mean1, g_mean2, g_mean3......g_meanN. Calculate the layer number corresponding to the gradient with the largest change amplitude among them:
[0137] The gradient reflects the magnitude of the variable that the model needs to do in the next step, indicating that the network layer of the model has changed significantly compared to the previous training effect. Find the two network layers corresponding to g_max, assume them to be p and q; and the two average gradients corresponding to g_max, assume them to be g_meanp and g_meanq. Finally, the network layer corresponding to FeedForward is:
[0138] .
[0139] Step 3: After determining the model structure and the number of layers of each network layer, such as the number of layers num_f of the Feedward layer, continue to calculate the mean value using the two average gradients g_meanp and g_meanq corresponding to g_max obtained in the second step to obtain :
[0140] ;
[0141] Perform weight update:
[0142] ;
[0143] Among them, is the defined learning rate, which is an adjustable parameter, and the default setting is 0.00001.
[0144] Load the weight matrix w_t updated for each network layer into the defined model structure in turn, and perform the fine-tuning training of the model on the next test set. The model training splits the data according to the infectious disease type, and the split unit for each input calculation of the model is batch. Assume that the total number of batches for training is k, and use the fintune setme data to train the model. After each batch training is completed, a gradient value will be obtained, denoted as , so after k times of training are completed, the final model weights can be expressed as:
[0145] ;
[0146] Among them, is the learning rate defined by us, which is an adjustable parameter with a default setting of 0.00001. w_f is the finally obtained model weight. Loading w_f into the defined model structure completes the training of the model.
[0147] For the query problem query set data to be tested collected, or the query problem directly input by the user, direct matching is performed according to the text content and the knowledge graph to retrieve the relevant entity types and relationship types in the knowledge graph. The relevant results retrieved by the query problem entities are spliced and input into the trained meta-learning question-answering model together, so that the model utilizes the original question query while utilizing the relevant information in the infectious disease knowledge graph for comprehensive question answering, and returns the answer result to the user interface. At the same time, using the evaluations collected from users on the answers, according to these evaluation information, the response data with positive evaluation results are selected, which can be further added to meta_data after manual review for result reflux, expanding the training data, forming a closed loop, and iteratively optimizing the infectious disease question-answering system.
[0148] The specific implementation manners described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only the specific implementation manners of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for answering questions about infectious diseases based on meta - learning, characterized in that, Including: Generate an infectious disease knowledge graph; the entity types of the infectious disease knowledge graph include infectious diseases, pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions; entity attribute identifiers are added to the entity types; the relationship types of the entity types take infectious diseases as the root node, and pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions as the sub-nodes; Collect multiple groups of Q&A pair data of multiple infectious disease types and divide them into a training set and a test set; both the training set and the test set contain all the infectious disease types; Construct a meta-learning Q&A model; the training set trains the meta-learning Q&A model according to each infectious disease type respectively; Increment the number of network layers in sequence according to the gradient means trained for multiple infectious disease types; Change the average of the two largest gradient means to the final gradient of the network layer of the meta-learning Q&A model, and the average of the corresponding two layer numbers is the final number of layers, and use the test set to train the meta-learning Q&A model; Use multiple query questions to respectively obtain the relevant entity types and relationship types in the knowledge graph; Input each of the query questions, the entity types, and the relationship types into the meta-learning Q&A model, and return the output results to the corresponding user interface; Select the results with positive user feedback and add them to the Q&A pair data for training each meta-learning Q&A model.
2. The method for answering questions about infectious diseases based on meta-learning according to claim 1, wherein The method for constructing the meta-learning Q&A model includes: The network layer of the meta-learning Q&A model includes an embedding layer, an LN layer, and a Feedforward layer arranged from bottom to top in sequence; the embedding layer is used to convert the Q&A pair data into a matrix; the LN layer is used to perform layer normalization on the data output by the embedding layer; the Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the LN layer.
3. The method for answering questions about infectious diseases based on meta-learning according to claim 2, wherein The embedding layer is used to convert the Q&A pair data into a matrix, expressed as: ； where x represents the digital id representation of the Q&A pair data; W represents the weight matrix of the embedding layer; T represents matrix transpose; b represents the bias weight matrix of the embedding layer.
4. The method for answering infectious disease questions based on meta - learning according to claim 2, characterized in that, The LN layer is used to perform layer normalization on the data output by the embedding layer, expressed as: ； Among them, x represents the digital id representation of the question-answer pair data; represents the mean value of the current input data x; Var(x) represents the variance of the input data x; is a hyperparameter used to prevent the denominator from being 0, and the value is 0.0001; represents the weight matrix of the LN layer; β represents the bias weight matrix of the LN layer.
5. The method for answering questions about infectious diseases based on meta-learning according to claim 2, wherein, The Feedforward layer is used to perform multi-dimensional matrix calculations on the data output by the LN layer, expressed as: ； ； ； ； Among them, the Feedforward layer includes a query layer, a key layer, and a value layer; q is the data output by the query layer; k is the data output by the key layer; v is the data output by the value layer; is the data output after the Feedforward layer calculation; is the data output by the LN layer; , , are the weight matrices of the query layer, the key layer, and the value layer respectively; b1, b2, and b3 are the bias weight matrices of the query layer, the key layer, and the value layer respectively.
6. The method for answering questions about infectious diseases based on meta - learning according to claim 2, wherein, The method for constructing the meta-learning Q&A model further includes: The network layer of the meta-learning Q&A model further includes a softmax layer; the softmax layer normalizes the data output by the Feedforward layer to obtain the final output, expressed as: ； Among them, represents the i-th data output by the Feedforward layer; represents the data output after passing through the softmax layer; represents the j-th data output by the Feedforward layer.
7. The method for answering questions about infectious diseases based on meta - learning according to claim 2, wherein, When changing the average of the two largest gradient means to the final gradient of the network layer of the meta-learning Q&A model, update the weight matrix and bias weight matrix of the embedding layer, the LN layer, and the Feedforward layer according to the final gradient.
8. A method for answering questions about infectious diseases based on meta-learning according to claim 7, characterized in that, Update the weight matrices and bias weight matrices of the embedding layer, LN layer, and Feedforward layer according to the final gradient, expressed as: ； Among them, is the weight matrix or bias weight matrix updated according to the final gradient of the network layer; is the weight matrix or bias weight matrix after random initialization; is the final gradient of the network layer; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
9. A method for answering questions about infectious diseases based on meta-learning according to claim 1, characterized in that, Training the meta-learning Q&A model using the test set includes: The test set is divided into multiple batches according to the infectious disease types. When each batch is trained, update the weights of the network layers of the meta-learning Q&A model according to the sum of the gradient values of all batches of infectious disease types, expressed as: ； Among them, is the weight of the network layer of the updated meta-learning Q&A model; is the weight of the network layer of the meta-learning Q&A model before update; is the total number of batches for training; 、 、 are the gradient values after each batch of training is completed; is the defined learning rate, which is an adjustable parameter and is default set to 0.00001.
10. An infectious disease Q&A system based on meta-learning, characterized in that, Including: A knowledge graph module for generating an infectious disease knowledge graph; the entity types of the infectious disease knowledge graph include infectious diseases, pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions; entity attribute identifiers are added to the entity types; the relationship types of the entity types have infectious diseases as the root node and pathogens, symptoms, transmission routes, preventive measures, treatment drugs, and medical institutions as child nodes; A data collection module that collects multiple sets of Q&A pair data of multiple infectious disease types and divides them into a training set and a test set; both the training set and the test set contain all the infectious disease types; A model training module that constructs a meta-learning Q&A model; the training set trains the meta-learning Q&A model according to each infectious disease type, and according to the average gradient of training for multiple infectious disease types, sequentially increase the number of network layers; change the average of the two largest gradient means to the final gradient of the network layer of the meta-learning Q&A model, and the average of the corresponding two layer numbers is the final number of layers; A model construction module that constructs a meta-learning Q&A model; the training set trains the meta-learning Q&A model according to each infectious disease type; A model optimization module that sequentially increases the number of network layers according to the average gradient of training for multiple infectious disease types; Change the average of the two largest gradient means to the final gradient of the network layer of the meta-learning Q&A model, and the average of the corresponding two layer numbers is the final number of layers, and use the test set to train the meta-learning Q&A model; A model feedback module that uses multiple query questions to respectively obtain the relevant entity types and relationship types in the knowledge graph; Input each of the query questions, the entity types, and the relationship types into the meta-learning Q&A model, and return the output results to the corresponding user interface; Select the results with positive user feedback and add them to the Q&A pair data for training each meta-learning Q&A model.