A medical question and answer system based on improved named entity recognition and a construction method thereof

Abstract

A medical question and answer system based on improved named entity recognition and a construction method thereof, the system and method extract the features of the text by using the BERT pre-training language model, have strong semantic expression ability, add a disturbance factor to the obtained word vector, and enhance the generalization ability and robustness of the model; introduce the adversarial training to solve the problem that the number of labels may not be enough or missed in the data set, reduce the influence of the data set noise on the implementation result; set the learning rate in layers to achieve the effect that the BERT layer does not decrease, the lower connection layer trains faster, and the training is synchronized; the features output by the BERT layer and the BiLSTM layer are spliced, the connection between layers is closer, deeper features are obtained, and the original features of the BERT layer are not lost; through the two-round intention recognition function, the casual conversation sentence is increased, so that the system can answer the casual conversation topic of the user, the slot inheritance makes the system have the multi-round question and answer function, and the accuracy of the reply of the question and answer system is greatly improved.

Description

TECHNICAL FIELD

[0001] The application relates to a medical question and answer system based on improved named entity recognition and a construction method thereof, and belongs to the technical field of knowledge graphs and natural language processing. BACKGROUND

[0002] With the continuous development of natural language processing technology, knowledge graphs are gradually applied to various fields, and question and answer systems based on knowledge graphs have also emerged as the times require. At present, there are few question and answer assistant platforms in the precise medical vertical field. Traditional question and answer systems mainly search according to keywords to obtain related content, but too many pages are returned, and the user needs to perform screening and judgment by himself / herself, so it is difficult to guarantee the accuracy of the results. Moreover, the data of the medical industry is huge and complex, and only the traditional method cannot meet the diversified needs of users. SUMMARY

[0003] In view of the problems existing in the prior art, the application provides a medical question and answer system based on improved named entity recognition and a construction method thereof. The question and answer system constructed by the method can improve the named entity recognition algorithm in question analysis, and improve the accuracy of the question and answer system reply.

[0004] In order to achieve the above purpose, the technical scheme adopted by the application is: a medical question and answer system based on improved named entity recognition, comprising a data acquisition module, a knowledge storage module, a natural language understanding module, a knowledge calculation module and a dialogue management and interaction module, the output end of the data acquisition module is connected with the input end of the knowledge storage module, the output end of the knowledge storage module is connected with the input end of the natural language understanding module, the output end of the natural language understanding module is connected with the input end of the knowledge calculation module, and the output end of the knowledge calculation module is connected with the input end of the dialogue management and interaction module.

[0005] The data acquisition module is used for crawling medical data on a website, and cleaning and preprocessing the data to construct a knowledge graph dataset.

[0006] The knowledge storage module is used for storing the knowledge triplets extracted from the dataset by using a Neo4j graph database, and displaying a visual knowledge graph.

[0007] The natural language understanding module is used for performing a named entity recognition task and an intent recognition task on a question input by a user, and understanding the specific meaning of the question input by the user.

[0008] The knowledge calculation module is used for converting the question input by the user through the natural language understanding module into a structured query statement, and querying the knowledge graph by using a Cypher statement to obtain an answer.

[0009] The dialogue management and interaction module is used for page development of a knowledge graph-based question and answer system, supports user input of a question sentence, returns a corresponding answer, and supports multi-round question and answer.

[0010] A medical question and answer system construction method based on improved named entity recognition, comprising the following steps:

[0011] Step one, crawling relevant medical information, cleaning and preprocessing the data, constructing a data set, and defining the required data mode, using a Neo4j graph database to store the collected data according to the defined data mode, and completing the construction of the knowledge graph;

[0012] Step two, construct a named entity recognition model structure based on the BERT-FGM-BiLSTM-CRF-lr splicing improved network, perform the medical named entity recognition task, and extract the medical entities in the user input question sentence;

[0013] Step three, construct an intent recognition model structure based on the BERT-TextCNN network, perform the question intent recognition task, and identify the intent implied in the user input question;

[0014] Step four, use the technical route of "intent recognition + semantic slot design + template" to construct the medical question and answer system, perform two rounds of intent recognition, judge the intent, fill the slot, and then query in the knowledge graph according to the structured semantic slot;

[0015] Step five, use slot inheritance to realize the multi-round question and answer function of the system;

[0016] Step six, design the medical question and answer system and its user page, use PyQt5 to develop the page, support human-computer interaction, and realize the function of online auxiliary diagnosis.

[0017] Further, the step of constructing the data set and the knowledge graph in step one is as follows:

[0018] (1.1) Extract the semi-structured data in the webpage, Xunyi asks medicine network provides very comprehensive disease knowledge and treatment methods for users, and the data has a certain authority and clear structure, which is suitable for crawling, and Xunyi asks medicine network is selected as the data source for constructing the knowledge graph in this paper;

[0019] (1.2) Analyze the nature of the webpage to obtain the corresponding URL address of the data;

[0020] (1.3) Use the urllib.request data request module to send the network address and obtain the HTML format data of the webpage; the network address is the URL address obtained in step (1.2);

[0021] (1.4) Using XPath to parse HTML tags, extract the required data and its associated relationship;

[0022] (1.5) Define eight types of entities: disease, symptom, examination, drug, pharmaceutical company, department, food, and recipe. Design eleven types of relationships: disease-symptom, disease-complicated disease, disease-examination, disease-recommended drug, disease-common drug, disease-department, disease-recipe, disease-appropriate food, disease-avoid food, department-department, and pharmaceutical company-drug;

[0023] (1.6) Save the data in step (1.5) to the local computer to obtain the initial corpus;

[0024] (1.7) For missing values in the data, directly delete or fill in;

[0025] (1.8) For the noise in the data, use regular expressions to standardize, remove stop words, garbled characters, special characters, redundant information, and punctuation and letter formats;

[0026] (1.9) Use a word segmentation tool to perform word segmentation processing. Format the data and store it in the form of key-value pairs. Export the regular JSON data as a dataset for building a medical knowledge graph;

[0027] (1.10) Analyze the entities, attributes, and relationships between entities in the constructed dataset. Define the required data pattern in combination with the application of the question and answer system;

[0028] (1.11) Extract knowledge triples from the data according to the Schema;

[0029] (1.12) Use the Py2neo module in Python to connect to Neo4j. Use Cypher statements to establish entity nodes and entity relationship edges of the knowledge graph according to the Schema. Write the attributes of the disease entity. Complete the construction of the knowledge graph.

[0030] Further, the construction steps of the named entity recognition model based on BERT-FGM-BiLSTM-CRF-lr splicing improved network in step two are as follows:

[0031] (2.1) Merge CCKS2019 dataset and cMedQANER dataset: CCKS2019 dataset contains disease and diagnosis, anatomical site, surgery, examination, drug, and inspection; cMedQANER dataset includes disease, symptom, detection, physiological function, treatment method, body part, population, department, drug, local, and time; Both datasets are annotated using the BIO annotation method;

[0032] (2.2) Take CCKS2019 dataset as the basic dataset, extract part of the available category entities from the cMedQANER dataset and merge them into the CCKS2019 dataset, including "detection-examination, treatment method-surgery, drug-drug", and make a small range expansion on the CCKS2019 dataset;

[0033] (2.3) Introduce BERT model for word embedding, which is obtained through pre-training and Fine-tune link. All word vectors are obtained through BERT model, text features are extracted, and the ability of the model to obtain character semantic features is enhanced, denoted as e1;

[0034] (2.4) Introduce adversarial training, add the disturbance factor r of adversarial training to all word vectors e1 obtained through the BERT model, denoted as e2; wherein the adversarial training formula is as follows:

[0035]

[0036] In the formula, D is the training set, x is the input, y is the label, θ is the model parameter, L(x+Δx,y;θ) is the loss value of a single sample, Ω is the disturbance space, and Δx is the adversarial disturbance;

[0037] The disturbance factor r is obtained by standardizing the word vector loss value and the current gradient value output by BERT, and the sum of the word vector and the disturbance amount is the adversarial sample; The calculation formula of the adversarial disturbance is as follows:

[0038]

[0039] In the formula, g is the gradient value, that is, the partial derivative of the loss function with respect to x, and ε is the scaling factor;

[0040] (2.5) Send the word vector e2 with disturbance to the BiILSTM network to obtain the context feature information, denoted as e3;

[0041] (2.6) The output e3 of the BiLSTM layer is spliced with the word vector e2 with disturbance, and the output features of the two layers are retained, denoted as e4;

[0042] (2.7) Send the vector e4 to the full connection layer for dimension reduction processing, denoted as e5;

[0043] (2.8) Input e5 into the CRF layer for decoding processing to obtain the label sequence corresponding to each character, denoted as outputs; In the decoding process, the CRF layer judges the label according to the transition probability matrix, which is specifically randomly initialized during training, and then optimized to make it more consistent with the actual transition probability between the labels of the training data, as follows:

[0044]

[0045] where A is a transition matrix, represents the transition probability from label yi to y i+1 i; P is the output of the BiLSTM network, represents the score of the y i th label of the i th character by the BiLSTM layer.

[0046] (2.9) Train the BERT layer and the model structure connected thereto with different learning rates, and adjust them according to different stages to keep the training synchronized, wherein the learning rate of the BERT layer is set as lr1, the learning rate of the BiLSTM layer is set as lr2, and the learning rate of the CRF layer is set as lr3.

[0047] Further, the method for constructing the intent recognition model structure based on the BERT-TextCNN network in step three is as follows:

[0048] (3.1) Select the public CMID dataset;

[0049] (3.2) According to the constructed knowledge graph and the defined entities and relationships, extract 13 types of intents that can be answered from the CMID dataset, and write them into the label file;

[0050] (3.3) Use the template-based supplementary dataset generation method to set the generation rule template for the part of the intent with small data amount, and manually write the keywords of each rule;

[0051] (3.4) Combine the keywords in a certain order, and finally randomly generate them to obtain the supplementary corpus and balance the data in the original dataset;

[0052] (3.5) Use the BERT model as the Embedding layer to convert the text into a vector and extract the text features, denoted as b1, b2, …, b n ;

[0053] (3.6) Concatenate the vectors b1, b2, …, b n to obtain an embedding matrix, denoted as B 1:n ,

[0054] B 1:n = [b1, b2, …, b n ]

[0055] where b1, b2, …, b n represent word vectors;

[0056] (3.7) Concatenate the embedding matrix B 1:nThe semantic feature a is sent to a convolutional layer for feature extraction, and a convolution operation is performed using a convolution kernel with a size of (3, 4, 5) to extract the semantic feature of the sentence, denoted as a i e1;

[0057] The semantic feature a i is calculated according to the following formula:

[0058] a i =f(W·M i:i+h-1 +b)

[0059] In the formula, M is a word vector matrix, b is a bias, W is a neural network weight, h is a convolution kernel size, f is a nonlinear function used to calculate a feature value, M i:i+h-1 is a word vector at a different position in the text;

[0060] (3.8) The semantic feature a i extracted in step (3.7) is sent to a pooling layer, and the pooling layer performs a max_pooling pooling operation on the semantic feature a i to downsample it while keeping the same vector dimension, and the output obtained is denoted as f1;

[0061] (3.9) After the pooling layer, sentences of different lengths are all converted into fixed-length expressions;

[0062] (3.10) Dropout is added to prevent overfitting;

[0063] (3.11) The f1 after pooling is sent to a fully connected layer to obtain the probability of each label;

[0064] (3.12) The classification result of the text is output by using softmax, denoted as outputs.

[0065] Further, the step of constructing a medical question and answer system in step four is as follows:

[0066] (4.1) The input question of the user is defined as Q;

[0067] (4.2) The named entity recognition model in step two is used to extract the medical-related entities in Q;

[0068] (4.3) The first round of intent recognition is performed on Q using a logistic regression algorithm to determine whether the input of the user is a casual intent or a diagnosis intent;

[0069] (4.4) If it is determined to be a casual intent, a set casual template is used to answer;

[0070] (4.5) If it is determined to be a diagnosis intent, the second round of intent recognition is entered, i.e., the intent recognition model in step three is used to determine the specific diagnosis intent of the user;

[0071] (4.6) After obtaining the probability of each category, the system sorts the probabilities in descending order and selects the intent with the highest confidence;

[0072] (4.7) Three threshold intervals of confidence are set in advance, which are greater than 0.8, between 0.4 and 0.8, and less than 0.4. The intent confidence is compared with the threshold;

[0073] (4.8) The reply strategy is determined according to the obtained intent confidence, the slot filling is performed, and the structured semantic slot is used to query the knowledge graph using the Cypher statement;

[0074] (4.9) When the highest intent confidence returned is greater than 0.8, the "accept" strategy is adopted, and the answer is given according to the combination of the intent and the slot value and the reply template;

[0075] (4.10) If the slot value is empty, it means that no relevant result is found in the knowledge graph, and the deny_response template is directly replied;

[0076] (4.11) When the highest intent confidence returned is between 0.4 and 0.8, the "clarification" strategy is adopted, and the system will ask questions according to the template;

[0077] (4.12) When the highest intent confidence returned is less than 0.4, the "refuse" strategy is adopted, and the answer is refused.

[0078] Further, the steps of the multi-round question and answer of the system in step five are as follows:

[0079] (5.1) The semantic slot template is defined in advance for the entity;

[0080] (5.2) Identify which slots are included in the user input question;

[0081] (5.3) Extract the slot values and fill them into the predefined semantic slot, i.e. slot filling;

[0082] (5.4) When analyzing the user input question, if the slot filling is completed and there are still slots with empty slot values, the slot values of the previous round of questions are inherited, i.e. slot inheritance;

[0083] (5.5) Use the Cypher statement to query the knowledge graph to realize multi-round question and answer.

[0084] Further, the construction steps of the medical question and answer system and its user page in step six are as follows:

[0085] (6.1) The medical question and answer system is designed by adopting a hierarchical structure, including a data layer, a construction layer and a user layer; wherein the data layer is responsible for providing data support; the construction layer contains two contents, one is to construct a knowledge graph according to a self-built medical field data set, and the other is to construct a medical question and answer system; the user layer is user-oriented, and mainly carries out dialogue management and interaction, and the question input in the interface is transmitted to the construction layer for specific analysis operation;

[0086] (6.2) The page is designed by using PyQt5, and the question input by the user is sequentially subjected to entity extraction, intent judgment, slot filling, structured sentence query and template reply work, the above trained model is directly called, and the input of the user is transmitted to the construction layer for specific analysis operation;

[0087] (6.3) The generated answer is fed back to the user in the user operation page.

[0088] The application extracts the features of the text by using the BERT pre-training language model, can learn rich prior information through unsupervised training on a large amount of corpus, has strong semantic expression ability, and adds a disturbance factor to the obtained word vector, thereby enhancing the generalization ability and robustness of the model; the introduction of the adversarial training solves the problem that the number of labels in the data set may not be enough or missing, reduces the influence of the noise of the data set on the implementation result; the hierarchical learning rate is set, the learning rate of the pre-training layer is reduced, and a larger learning rate is set for the lower interface layer, so that the BERT layer effect does not decrease, the lower interface layer training is faster, and the training is synchronized; the features output by the BERT layer and the BiLSTM layer are spliced, so that the connection between the layers is closer, deeper features are obtained, and the original features of the BERT layer are not lost; through the two rounds of intent recognition function, the casual sentence is increased, so that the system can answer the casual topics of the user, and the slot inheritance makes the system have the multi-round question and answer function, thereby greatly improving the accuracy of the reply of the question and answer system. BRIEF DESCRIPTION OF DRAWINGS

[0089] Figure 1 is a structural schematic diagram of the medical question and answer system of the application;

[0090] Figure 2 is a work flow chart of the construction method of the medical question and answer system of the application;

[0091] Figure 3 is a framework diagram of the construction of the medical question and answer system of the application;

[0092] Figure 4 is a model structure diagram of the named entity recognition module of the application;

[0093] Figure 5 is a work flow chart of the intent recognition module of the application;

[0094] Figure 6 is a system question and answer example of the present application;

[0095] Figure 7 is a system multi-round question and answer example of the present application. DETAILED DESCRIPTION

[0096] The present application will be further described below with reference to the accompanying drawings.

[0097] As shown in Figure 1 , a medical question and answer system based on improved named entity recognition includes a data acquisition module, a knowledge storage module, a natural language understanding module, a knowledge computing module and a dialogue management and interaction module, the output end of the data acquisition module is connected with the input end of the knowledge storage module, the output end of the knowledge storage module is connected with the input end of the natural language understanding module, the output end of the natural language understanding module is connected with the input end of the knowledge computing module, and the output end of the knowledge computing module is connected with the input end of the dialogue management and interaction module.

[0098] The data acquisition module is used for crawling medical data on a website, and cleaning and preprocessing the data to construct a knowledge graph dataset.

[0099] The knowledge storage module is used for storing the knowledge triples extracted from the dataset in a Neo4j graph database, and displaying a visual knowledge graph.

[0100] The natural language understanding module is used for performing named entity recognition and intent recognition tasks on the user input question, and understanding the specific meaning of the user question.

[0101] The knowledge computing module is used for converting the question passed through the natural language understanding module into a structured query statement, and querying the knowledge graph using a Cypher statement to obtain an answer.

[0102] The dialogue management and interaction module is used for developing a knowledge graph-based question and answer system page, supporting user input questions, returning corresponding answers, and supporting multi-round question and answer.

[0103] As shown in Figure 2 and Figure 3 , a medical question and answer system construction method based on improved named entity recognition includes the following steps:

[0104] Step one, crawling related medical information, cleaning and preprocessing the data, constructing a dataset, defining the required data mode, storing the collected data according to the defined data mode using a Neo4j graph database, and completing the construction of the knowledge graph.

[0105] Step two, build a named entity recognition model structure based on BERT-FGM-BiLSTM-CRF-lr splicing improved network, perform medical named entity recognition task, and extract medical entities in user input question;

[0106] Step three, build an intent recognition model structure based on BERT-TextCNN network, perform question intent recognition task, and identify the intent implied in the user input question;

[0107] Step four, adopt the technical route of "intent recognition + semantic slot design + template" to build a medical question and answer system, perform two rounds of intent recognition, fill in the slot after determining the intent, and query in the knowledge graph according to the structured semantic slot;

[0108] Step five, use slot inheritance to realize the multi-round question and answer function of the system;

[0109] Step six, design the medical question and answer system and its user page, use PyQt5 to develop the page, support human-computer interaction, and realize the function of online auxiliary diagnosis.

[0110] The steps of building the dataset and knowledge graph are as follows:

[0111] (1.1) Extract the semi-structured data in the webpage, and provide very comprehensive disease knowledge and treatment methods for users. The data has a certain authority and the structure is relatively clear, which is suitable for crawling. Selecting the medical and drug information network as the data source for building the knowledge graph in this paper;

[0112] (1.2) Analyze the nature of the webpage to get the corresponding URL address of the data;

[0113] (1.3) Use the urllib.request data request module to send the network address and get the HTML format data of the webpage; the network address is the URL address obtained in step (1.2);

[0114] (1.4) Use XPath to parse the HTML tags and extract the required data and its associated relationships;

[0115] (1.5) Define eight types of entities: diseases, symptoms, examinations, drugs, pharmaceutical companies, departments, foods, and recipes, and design eleven types of relationships: disease-symptom, disease-complicated disease, disease-examination, disease-recommended drug, disease-common drug, disease-department, disease-recipe, disease-appropriate food, disease-avoidance food, department-department, and pharmaceutical company-drug;

[0116] (1.6) Save the data in step (1.5) to the local computer to get the initial corpus;

[0117] (1.7) For missing values of data, directly delete or fill in;

[0118] (1.8) For the noise existing in the data, use regular expression normalization to remove stop words, garbled codes, special characters, redundant information, and punctuation and letter formats in the data;

[0119] (1.9) Use word segmentation tools for word segmentation processing, format the data, convert the data into key-value pairs for storage, export regular JSON data as a dataset for building a medical knowledge graph;

[0120] (1.10) For the constructed dataset, analyze the entities, attributes and relationships between entities, and define the required data mode in combination with the application of the question and answer system;

[0121] (1.11) Extract knowledge triples from data according to Schema;

[0122] (1.12) Use the Py2neo module in Python to realize the connection with Neo4j, use Cypher statements to build entity nodes and entity relationship edges of the knowledge graph according to Schema, and write the attributes of the disease entity, complete the construction of the knowledge graph.

[0123] As shown in Figure 4 , the construction steps of the named entity recognition model based on the BERT-FGM-BiLSTM-CRF-lr splicing improved network are as follows:

[0124] (2.1) Merge CCKS2019 dataset and cMedQANER dataset: CCKS2019 dataset contains diseases and diagnoses, anatomical sites, surgeries, examinations, drugs, and tests; cMedQANER dataset includes diseases, symptoms, tests, physiological functions, treatment methods, body parts, people, departments, drugs, localities, and time; Both datasets are annotated using the BIO annotation method;

[0125] (2.2) Take CCKS2019 dataset as the basic dataset, and merge some available category entities from cMedQANER dataset into CCKS2019 dataset, including "test-examination, treatment method-surgery, and drug-drug", and do small-scale expansion on CCKS2019 dataset;

[0126] (2.3) Introduce BERT model for word embedding, which is obtained through pre-training and Fine-tune links, obtain all word vectors through BERT model, extract text features, and enhance the model's ability to obtain character semantic features, denoted as e1;

[0127] (2.4) Introducing adversarial training, all word vectors e1 obtained by BERT model are added to the perturbation factor r of adversarial training, denoted as e2; wherein the formula of adversarial training is as follows:

[0128]

[0129] In the formula, D is the training set, x is the input, y is the label, θ is the model parameter, L(x+Δx, y; θ) is the loss value of a single sample, Ω is the perturbation space, and Δx is the adversarial perturbation;

[0130] The perturbation factor r is obtained by standardizing the word vector loss value output by BERT and the current gradient value, and the sum of the word vector and the perturbation amount is the adversarial sample; the calculation formula of the adversarial perturbation is as follows:

[0131]

[0132] In the formula, g is the gradient value, i.e. the partial derivative of the loss function with respect to x, and ε is the scaling factor;

[0133] (2.5) The word vector e2 added with the perturbation is sent to the BiILSTM network to obtain the context feature information, denoted as e3;

[0134] (2.6) The output e3 of the BiLSTM layer is spliced with the word vector e2 added with the perturbation, and the output features of the two layers are reserved, denoted as e4;

[0135] (2.7) The vector e4 is sent to the full connection layer for dimension reduction processing, denoted as e5;

[0136] (2.8) e5 is input into the CRF layer for decoding processing to obtain the label sequence corresponding to each character, denoted as outputs; in the decoding process of the CRF layer, the label is judged according to the transition probability matrix, specifically, the transition matrix is randomly initialized during training, and then it is optimized to make it more consistent with the actual transition probability between the labels of the training data, and the formula is as follows:

[0137]

[0138] In the formula, A is the transition matrix, represents the transition probability from label yi to y i+1 , P is the output of the BiLSTM network, represents the score of the BiLSTM layer for the y i th label of the i th character.

[0139] (2.9) The BERT layer and the model structure connected thereto are trained with different learning rates, and are adjusted according to different stages to keep the training synchronized, wherein the learning rate of the BERT layer is set as lr1, the learning rate of the BiLSTM layer is set as lr2, and the learning rate of the CRF layer is set as lr3.

[0140] As shown in Figure 5 , the construction method of the intent recognition model structure based on the BERT-TextCNN network is as follows:

[0141] (3.1) Select the CMID dataset disclosed;

[0142] (3.2) According to the constructed knowledge graph and the defined entities and relations, 13 types of system answerable intents are extracted from the CMID dataset and written into the label file;

[0143] (3.3) In the form of generating a supplementary dataset based on a template, for the part of the intent with less data, a generation rule template is set, and the keywords of each rule are manually written;

[0144] (3.4) The keywords are combined in a certain order, and finally randomly generated to obtain the supplementary corpus and balance the data in the original dataset;

[0145] (3.5) The BERT model is used as the Embedding layer to convert the text into a vector and extract the text features, denoted as b1, b2, …, b m ;

[0146] (3.6) The vectors b1, b2, …, b , are spliced to obtain an embedding matrix, denoted as B 1:n ,

[0147] B 1:n =[b1,b2,…,b n ]

[0148] Wherein b1, b2, …, b n represent word vectors;

[0149] (3.7) The embedding matrix B 1:n is sent to the convolution layer for feature extraction, and the convolution operation is performed with a convolution kernel of size (3, 4, 5) to extract the semantic features of the sentence, denoted as a i e1;

[0150] The calculation formula of the semantic feature a i is as follows:

[0151] a i =f(W·M i:i+h-1 +b)

[0152] where M is the word vector matrix, b is the bias, W is the neural network weight, h is the size of the convolution kernel, f is a nonlinear function used to calculate the feature value, M i:i+h-1 is the word vector at different positions in the text;

[0153] (3.8) The semantic features a i extracted in step (3.7) are sent to the pooling layer, which uses the max_pooling pooling operation to downsample the semantic features a i while keeping the same vector dimension, and the output is denoted as f1;

[0154] (3.9) After the pooling layer, sentences of different lengths are all converted into fixed-length expressions;

[0155] (3.10) Dropout is added to prevent overfitting;

[0156] (3.11) The f1 after pooling is sent to the fully connected layer to obtain the probability of each label;

[0157] (3.12) The classification result of the text is output using softmax, denoted as outputs.

[0158] As Figure 6 shown, the steps for constructing a medical question and answer system are as follows:

[0159] (4.1) Define the user input question as Q;

[0160] (4.2) Use the named entity recognition model in step two to extract the medical-related entities in Q;

[0161] (4.3) Use the logistic regression algorithm to perform the first round of intent recognition on Q to determine whether the user's input is a casual intent or a diagnosis intent;

[0162] (4.4) If it is determined to be a casual intent, use the set casual template to answer;

[0163] (4.5) If it is determined to be a diagnosis intent, proceed to the second round of intent recognition, i.e., use the intent recognition model in step three to determine the specific diagnosis intent of the user;

[0164] (4.6) After obtaining the probability of each category, the system sorts the probabilities in descending order and takes the intent with the highest confidence;

[0165] (4.7) Pre-set three confidence threshold intervals, which are greater than 0.8, between 0.4 and 0.8, and less than 0.4, and compare the intent confidence with the threshold;

[0166] (4.8) According to the obtained intention confidence, determine the reply strategy, fill the slot, and then query in the knowledge graph according to the structured semantic slot using the Cypher statement;

[0167] (4.9) When the returned highest intention confidence is greater than 0.8, the "accept" strategy is adopted, and the answer is given according to the combination of the intention and the slot value and the reply template;

[0168] (4.10) If the slot value is empty, it proves that no relevant result is queried in the knowledge graph, and at this time, the deny_response template is directly replied;

[0169] (4.11) When the returned highest intention confidence is between 0.4 and 0.8, the "clarification" strategy is adopted, and the system will ask according to the template;

[0170] (4.12) When the returned highest intention confidence is less than 0.4, the "refuse" strategy is adopted, and the answer is refused.

[0171] As shown in Figure 7 , the steps of the multi-round question and answer of the system are as follows:

[0172] (5.1) The semantic slot template is defined in advance for the entity;

[0173] (5.2) Identify which slots are contained in the user input question;

[0174] (5.3) Extract the slot value and fill it into the predefined semantic slot, that is, slot filling;

[0175] (5.4) When analyzing the user input question, if the slot filling is completed and there are still slots with empty slot values, the slot values of the last round of questions are inherited, that is, slot inheritance;

[0176] (5.5) Query in the knowledge graph using the Cypher statement to realize multi-round question and answer.

[0177] Further, the construction steps of the medical question and answer system and the user page thereof in the sixth step are as follows:

[0178] (6.1) The medical question and answer system is designed in a hierarchical structure, including a data layer, a construction layer and a user layer; wherein the data layer is responsible for providing data support; the construction layer includes two major contents, one is to construct a knowledge graph according to the self-built medical field data set, and the other is to construct a medical question and answer system; the user layer faces users, mainly carries out dialogue management and interaction, and the question input in this interface will be transmitted to the construction layer for specific analysis operation;

[0179] (6.2) using PyQt5 to design a page, for the user input question in turn to extract entity, judge intention, slot filling, structured query, template reply work, directly call the above trained model, the user's input into the construction layer for specific analysis operation;

[0180] (6.3) the generated answer is fed back to the user in the user operation page.

[0181] Experiments prove that the named entity recognition model based on the BERT-FGM-BiLSTM-CRF-lr splicing improved network of the application has f1 value of 87.70% on the self-built data set, the intention recognition model has f1 value of 76.64% on the self-built data set, which greatly improves the accuracy of the reply of the question and answer system.

Claims

1. A method for constructing a medical question-answering system based on improved named entity recognition, characterized in that, Includes the following steps: Step 1: Crawl relevant medical information, clean and preprocess the data, build a dataset, define the required data pattern, and use Neo4j graph database to store the collected data according to the defined data pattern to complete the construction of the knowledge graph. Step 2: Construct a named entity recognition model structure based on the BERT-FGM-BiLSTM-CRF-lr concatenation improved network, and perform a medical named entity recognition task to extract medical entities from the user's input question. The construction steps of the named entity recognition model based on the BERT-FGM-BiLSTM-CRF-lr concatenation improved network are as follows: (2.1) Integration of CCKS2019 and cMedQANER datasets: The CCKS2019 dataset includes diseases and diagnoses, anatomical sites, surgeries, examinations, drugs, and tests; the cMedQANER dataset includes diseases, symptoms, tests, physiological functions, treatments, body parts, populations, departments, drugs, local areas, and time. Both datasets are labeled using the BIO annotation method. (2.2) Using the CCKS2019 dataset as the base dataset, some entities of available categories were extracted from the cMedQANER dataset and merged into the CCKS2019 dataset, including "detection-examination, treatment method-surgery, medicine-drug", to make a small-scale expansion of the CCKS2019 dataset; (2.3) Introduce the BERT model for word embedding. The word embedding is obtained through pre-training and fine-tune. All word vectors are obtained through the BERT model, and text features are extracted to enhance the model's ability to acquire character semantic features. This is denoted as e1. (2.4) Introduce adversarial training by adding all word vectors e1 obtained through the BERT model to the perturbation factor r for adversarial training, denoted as e2; where the adversarial training formula is as follows: ; In the formula, D is the training set. x For input, y For tags, θ For model parameters, L(x+Δx,y;θ) Let Ω be the loss value for a single sample, and Ω be the perturbation space. Δx To counteract disturbances; The perturbation factor r is obtained by standardizing the word vector loss value output by BERT with the current gradient value. The sum of the word vector and the perturbation amount is the adversarial example. The formula for calculating the adversarial perturbation is as follows: ; In the formula, g is the gradient value, i.e., the loss function with respect to gradient. x The partial derivative of ε is the scaling factor; (2.5) The perturbated word vector e2 is fed into the BiLSTM network to obtain the contextual feature information, denoted as e3; (2.6) The output e3 of the BiLSTM layer is concatenated with the perturbated word vector e2, while retaining the output features of the two layers, denoted as e4; (2.7) The vector e4 is fed into the fully connected layer for dimensionality reduction and denoted as e5; Step 3: Construct an intent recognition model structure based on the BERT-TextCNN network, perform the question intent recognition task, and identify the intent contained in the user's input question; Step 4: The medical question-answering system is built using the technical approach of "intent recognition + semantic slot design + template". Two rounds of intent recognition are performed. After the intent is determined, slots are filled and then the knowledge graph is queried based on the structured semantic slots. Step 5: Implement the system's multi-turn question-and-answer function using slot inheritance; Step 6: Design the medical question-and-answer system and its user pages. Use PyQt5 for page development to support human-computer interaction and realize the function of online auxiliary diagnosis.

2. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The steps for constructing the dataset and knowledge graph in step one are as follows: (1.1) Extracting semi-structured data from web pages, Xunyiwenyao.com provides users with very comprehensive disease knowledge and treatment methods, and the data is authoritative, clearly structured, and suitable for crawling. Xunyiwenyao.com is selected as the data source for constructing the knowledge graph in this paper. (1.2) Analyze the properties of the webpage to obtain the URL address corresponding to the data; (1.3) Use the urllib.request data request module to send a network address and obtain the HTML format data of the web page; the network address is the URL address obtained in step (1.2); (1.4) Use XPath to parse HTML tags and extract the required data and their relationships; (1.5) Define eight types of entities: disease, symptom, examination, drug, pharmaceutical company, department, food, and recipe; and design eleven types of relationships: disease-symptom, disease-complication, disease-examination, disease-recommended drug, disease-general drug, disease-department, disease-recipe, disease-suitable food, disease-avoidable food, department-department, and pharmaceutical company-drug. (1.6) Save the data from step (1.5) locally to obtain the initial corpus; (1.7) For missing values ​​in the data, delete them directly or fill them in; (1.8) To address the noise in the data, use regular expressions to normalize it, removing stop words, garbled characters, special characters, redundant information, and formatting of punctuation marks and letters; (1.9) Use a word segmentation tool to perform word segmentation, format the data, convert the data into key-value pairs for storage, and export regular JSON data as a dataset for building a medical knowledge graph; (1.10) For the constructed dataset, analyze the entities, attributes and relationships between entities, and define the required data pattern in conjunction with the application of the question-answering system; (1.11) Extract knowledge triples from the data according to the schema; (1.12) Use the Py2neo module in Python to connect with Neo4j, and use Cypher statements to build entity nodes and entity relationship edges of the knowledge graph according to the schema. Write the attributes of the disease entity to complete the construction of the knowledge graph.

3. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The construction steps of the named entity recognition model based on the BERT-FGM-BiLSTM-CRF-lr concatenation improved network in step two also include: (2.8) Input e5 into the CRF layer for decoding to obtain the label sequence corresponding to each character, denoted as outputs; during the decoding process, the CRF layer determines the label based on the transition probability matrix. Specifically, during training, the transition matrix is ​​first randomly initialized and then optimized to better match the actual transition probabilities between labels in the training data, as shown in the following formula: ; In the formula, A It is a transition matrix. Representative tags arrive The transition probability, P It is the output of the BiLSTM network. Indicates the BiLSTM layer for the th The first character Scoring of each tag; (2.9) The BERT layer and its subordinate model structure are trained with different learning rates and adjusted appropriately according to different stages to keep the training synchronized. The learning rate of the BERT layer is set to lr1, the learning rate of the BiLSTM layer is set to lr2, and the learning rate of the CRF layer is set to lr3.

4. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The method for constructing the intent recognition model structure based on the BERT-TextCNN network in step three is as follows: (3.1) Select a publicly available CMID dataset; (3.2) Based on the constructed knowledge graph and the defined entities and relationships, extract 13 types of intents that the system can answer from the CMID dataset and write them into the label file; (3.3) A template-based approach to generating supplementary datasets is adopted. For the intentions with small amounts of data, generation rule templates are set, and keywords for each rule are manually written. (3.4) Combine the keywords in a certain order and then generate them randomly to obtain supplementary corpus and balance the data in the original dataset; (3.5) Using the BERT model as the embedding layer, the text is converted into vectors, and text features are extracted, denoted as... ; (3.6) Transform the vector The concatenation is performed to obtain the embedding matrix, denoted as . , ; in, Represents word vectors; (3.7) Embedding matrix The data is fed into a convolutional layer for feature extraction. A convolutional operation is performed using kernels of size (3, 4, 5) to extract the semantic features of the sentence, denoted as... ; semantic features The calculation formula is as follows: ; In the formula, For word vector matrix, For bias, For neural network weights, The kernel size is the convolution kernel size. This is a nonlinear function used to calculate eigenvalues. These are word vectors at different positions in the text. (3.8) The semantic features extracted in step (3.7) The entire data is fed into the pooling layer, which uses the max_pooling operation to process the semantic features. We perform downsampling to maintain the same vector dimension, and the resulting output is denoted as f1. (3.9) After passing through the pooling layer, statements of different lengths are converted into fixed-length expressions; (3.10) Add Dropout to prevent overfitting; (3.11) Feed the pooled f1 into the fully connected layer to obtain the probability of each label; (3.12) Use softmax to output the classification results of the text, denoted as outputs.

5. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The steps for constructing the medical question-and-answer system in step four are as follows: (4.1) Define the question entered by the user as Q; (4.2) Use the named entity recognition model in step two to extract medical-related entities from Q; (4.3) Use logistic regression algorithm to perform the first round of intent recognition on Q to determine whether the user's input is a casual conversation intent or a diagnostic intent; (4.4) If the intention is determined to be casual conversation, use the set casual conversation template to answer; (4.5) If the intention is determined to be diagnostic, proceed to the second round of intention recognition, that is, use the intention recognition model in step three to determine the user's specific diagnostic intention; (4.6) After obtaining the probability of each category, the system sorts them in descending order according to the probability and selects the intent with the highest confidence. (4.7) Pre-set three confidence threshold ranges: greater than 0.8, between 0.4 and 0.8, and less than 0.

4. Compare the intention confidence with the thresholds. (4.8) Determine the response strategy based on the obtained intent confidence, fill the slots, and then use Cypher statements to query the knowledge graph based on the structured semantic slots; (4.9) When the highest confidence level of the returned intent is greater than 0.8, the "accept" strategy is adopted, and the response is made according to the intent and slot value combined with the response template; (4.10) If the slot value is empty, it means that no relevant results were found in the knowledge graph. In this case, reply directly with the deny_response template. (4.11) When the highest confidence level of the returned intent is between 0.4 and 0.8, the "clarification" strategy is adopted, and the system will ask follow-up questions according to the template; (4.12) When the highest confidence level of the returned intent is less than 0.4, the "rejection" strategy is adopted and the answer is rejected.

6. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The multi-round question-and-answer process in step five is as follows: (5.1) Predefine semantic slot templates for entities; (5.2) Identify which slots are contained in the user's input question; (5.3) Extract the slot values ​​and fill them into the predefined semantic slots, i.e., slot filling; (5.4) When parsing the user's input question, if there are still empty slots after the slot filling is completed, the slot value of the previous question is inherited, that is, slot inheritance; (5.5) Use Cypher statements to query in the knowledge graph to achieve multi-round question answering.

7. The method for constructing a medical question-answering system based on improved named entity recognition according to claim 1, characterized in that, The steps for constructing the medical Q&A system and its user pages in step six are as follows: (6.1) The medical question-answering system is designed with a hierarchical structure, including a data layer, a construction layer and a user layer. The data layer is responsible for providing data support. The construction layer includes two main parts: first, constructing a knowledge graph based on a self-built medical domain dataset; and second, constructing the medical question-answering system. The user layer is user-facing, managing dialogue and interaction. Questions entered on this interface will be passed to the construction layer for specific parsing operations. (6.2) Use PyQt5 for page design. For user-input questions, extract entities, determine intent, fill slots, query structured statements, and reply with templates in sequence. Directly call the trained model and pass the user input to the construction layer for specific parsing operations. (6.3) The generated answer is fed back to the user on the user operation page.

8. A medical question-answering system for use in the construction method of a medical question-answering system based on improved named entity recognition as described in any one of claims 1 to 7, characterized in that, It includes a data acquisition module, a knowledge storage module, a natural language understanding module, a knowledge computing module, and a dialogue management and interaction module. The output of the data acquisition module is connected to the input of the knowledge storage module, the output of the knowledge storage module is connected to the input of the natural language understanding module, the output of the natural language understanding module is connected to the input of the knowledge computing module, and the output of the knowledge computing module is connected to the input of the dialogue management and interaction module. The data acquisition module is used to crawl medical data from the website, clean and preprocess the data, and construct a knowledge graph dataset. The knowledge storage module is used to store the knowledge triples extracted from the dataset using the Neo4j graph database and display a visualized knowledge graph. The natural language understanding module is used to perform named entity recognition and intent recognition tasks on the user's input questions to understand the specific meaning of the user's questions; The knowledge computing module is used to convert the questions processed by the natural language understanding module into structured query statements, and to use Cypher statements to query and obtain answers in the knowledge graph; The dialogue management and interaction module is used to develop pages for building a knowledge graph-based question-and-answer system, which supports user input of questions and returns corresponding answers, and supports multi-round question-and-answer sessions.

Images