Data processing method and device for medical question-answering system
The proposed data processing method for medical question-answering systems uses a BERT-BiGRU-BiLSTM architecture with CRF layers to enhance natural language understanding, addressing the challenge of complex medical sentences and improving task-specific accuracy.
Patent Information
- Application Number
- CN202411054736.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-12-03
AI Technical Summary
Traditional medical question-answering systems face challenges in accurately processing long and complex medical sentences due to difficulties in capturing long-range dependencies and complex semantic relationships.
A data processing method and device utilizing a BERT layer, followed by BiGRU and BiLSTM layers, and TimeDistributed Dropout layer, combined with fully connected layers and CRF layers for entity recognition, intent recognition, and slot filling, to enhance natural language understanding.
The method achieves more accurate natural language understanding by processing user input text through a multi-task capable model, improving the accuracy of entity recognition, intent recognition, and slot filling tasks simultaneously.
Smart Images

Figure CN119066165A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a data processing method and device for a medical question-answering system. Background Art
[0002] The research background of intelligent medical question-answering system stems from the rapid development of medical information technology and artificial intelligence technology. The traditional medical service model has problems such as uneven resource distribution and long diagnosis and treatment time. The intelligent medical question-answering system can answer patients' questions efficiently and accurately through natural language processing technology, thereby improving the quality and efficiency of medical services.
[0003] In the field of natural language processing (NLP), medical question-answering systems have attracted much attention because they can provide users with timely and accurate medical information. However, due to the professionalism and complexity of medical language, building an efficient medical question-answering system faces huge challenges. Traditional machine learning methods often perform poorly when dealing with long-range dependencies and complex semantic relationships.
[0004] With the continuous advancement of artificial intelligence technology, especially the development of deep learning and big data technology, the application potential of intelligent question-answering systems in the medical field has gradually emerged. Traditional medical question-answering systems often rely on preset rules and limited knowledge bases, which are difficult to meet the complex and diverse needs of users. Modern intelligent medical question-answering systems have achieved accurate processing of medical entity recognition, intent recognition and slot filling by introducing advanced language models such as BERT (Bidirectional Encoder Representation Transform), BiGRU (Bidirectional Gated Recurrent Unit), BiLSTM (Bidirectional Long Short-Term Memory Network) and CRF (Conditional Random Field). These models can learn from a large amount of medical literature and data, improve the understanding ability and response accuracy of the question-answering system, and thus provide users with more intelligent and humane medical services. The application of this technology not only improves the efficiency of medical consultation, but also alleviates the problem of tight medical resources to a certain extent, and has broad social value and development prospects.
[0005] Tohti T et al. proposed a multi-task learning model for Chinese online medical question intent classification and named entity recognition based on ALBERT-BILSTM in "Medical QA-Oriented Multi-Task Learning Model for Question Intent Classification and Named Entity Recognition". The model is used to optimize the performance of medical query intent classification and named entity recognition tasks. Encoder parameter sharing is adopted so that the underlying network of the model can simultaneously consider named entity recognition and intent classification features. The model learns the shared information between the two tasks while maintaining its unique characteristics in the decoding stage. The method uses the Albert pre-trained language model to obtain word vectors containing semantic information and uses a bidirectional LSTM network for training. The model consists of an Albert word embedding layer, a BILSTM network layer, and a task-specific decoding layer. The model adopts a parameter hard sharing model, and the named entity recognition task and the intent classification task share the Albert word embedding layer and the BILSTM layer. During training, the model alternates between intent classification and named entity recognition tasks. In the entity recognition task, the model's precision on the Chinese Question Answering Dataset (CMedQA2) was improved to 81.02%, the recall rate was improved to 80.36%, and the F1 value was improved to 80.69%. In the intent recognition task, the precision on the Chinese Question Intent Classification Dataset (CMID) was improved to 88.42%, the recall rate was improved to 86.54%, and the F1 value was improved to 87.47%.
[0006] Huang Yong et al. proposed a BERT-BiGRU-based model in "Intelligent Medical Question Answering System Based on BERT-BiGRU Model" to solve the limitations of contextual semantic understanding faced by medical question answering systems when dealing with complex problems. A medical question answering system is established by pre-training the language model BERT and the bidirectional gated recurrent unit BiGRU, in which BERT extracts text semantic features and BiGRU learns the sequential dependency information of the text, thereby fully representing the text semantic structure information. Compared with the baseline method on the CBLUE medical question answering dataset, the precision of the model in the intent recognition task is improved to 79.22%, the recall rate is improved to 81.23%, and the F1 value (the harmonic mean of precision and recall) is improved to 79.82%. The model includes an input layer, a BERT layer, a bidirectional GRU layer, and a Softmax output layer. The pre-trained BERT layer is used to load the word Embedding, learn the semantic and grammatical knowledge of the language, and map the input text sequence to a semantic vector sequence. By receiving the output of the BERT layer, the semantic and sequential features of the input sequence are learned, and important information is provided for relationship classification judgment. The resulting hidden state of the GRU layer is received, and the final relationship type is output through the Softmax function. The input representation of the classification task is optimized through the Attention mechanism, and the optimized GRU output is passed to the Softmax classification layer to output the final multi-classification judgment result.
[0007] Hu Wen divided the question semantic parsing module into two submodules, entity recognition and intent recognition, in "Research and Implementation of Intelligent Medical Question Answering System Based on RoBERTa". In the entity recognition submodule, this paper proposed the RoBERTa-BiGRU-Attention-CRF model, used the RoBERTa pre-trained model to generate the word vector representation of the input question, and used the BiGRU model to extract the contextual semantic information. Next, the multi-head self-attention layer was used to further extract the text features to obtain the weight relationship of different words, thereby making up for the problem that the RoBERTa model was not fine-tuned. Finally, the CRF model was used to constrain the entity label. The F1 values of this model in the CMeEE dataset and the MedER dataset were 69.51% and 89.16% respectively. In the intent recognition submodule, this paper used the RoBERTa-DPCNN model, which embedded the text data by introducing the RoBERTa pre-trained model, and used the DPCNN model to perform convolution and pooling operations on the text features and classify the user intent. The F1 value of this model in the MedIR dataset was 86.42%.
[0008] Due to the complexity and variability of natural language, most traditional medical question-answering systems have difficulty processing long and complex sentences, and have difficulty achieving more accurate natural language understanding. Summary of the invention
[0009] In order to solve the technical problem that the medical question-answering system in the prior art is difficult to achieve relatively accurate natural language understanding, the embodiment of the present invention provides a data processing method and device for a medical question-answering system. The technical solution is as follows:
[0010] In one aspect, a data processing method for a medical question answering system is provided, the method comprising:
[0011] Receiving user input text, preprocessing the user input text, and converting the preprocessed user input text into first format data, where the first format data can be processed by the BERT layer;
[0012] Inputting the first format data into a data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer;
[0013] Obtain output data of the data processing model, and input the output data into each fully connected layer corresponding to each task;
[0014] The data output by each fully connected layer is processed for task output and the processed data is output.
[0015] Optionally, the inputting the output data into each fully connected layer corresponding to each task respectively includes:
[0016] Inputting the output data into the first fully connected layer corresponding to the entity recognition task;
[0017] Inputting the output data into the second fully connected layer corresponding to the intent recognition task;
[0018] The output data is input into the third fully connected layer corresponding to the slot filling task.
[0019] Optionally, the output dimension of the first fully connected layer is batch size , seq length , num_entity_labels;
[0020] The output dimension of the second fully connected layer is batch size , num_intent_labels;
[0021] The output dimension of the third fully connected layer is batch size ,seq_length,num_slot_labels.
[0022] Optionally, performing task output processing on the data output by each fully connected layer and outputting the processed data includes:
[0023] The data output by the first fully connected layer is input into the first CRF layer for processing and the processed data is output;
[0024] The data output from the second fully connected layer is input into the softmax layer for classification, processed by combining DPCNN and multi-head attention mechanism, and the processed data is output;
[0025] The data output by the third fully connected layer is input into the second CRF layer for processing and the processed data is output.
[0026] Optionally, the formula of the first CRF layer is ner =CRF ner (H seq1 ), where H seq1 is the input data of the first CRF layer, y ner is the output data of the first CRF layer;
[0027] The formula for the second CRF layer is y slot =CRF slot (H seq2 ), where H seq2 is the input data of the second CRF layer, y slot is the output data of the second CRF layer.
[0028] Optionally, the preprocessing includes:
[0029] Segment words and remove stop words.
[0030] Optionally, the first format data includes:
[0031] token IDs, segment IDs and attention mask.
[0032] On the other hand, a data processing device for a medical question-answering system is provided, and the data processing device for a medical question-answering system is used to implement the data processing method for a medical question-answering system provided in an embodiment of the present invention, and the device includes:
[0033] A receiving module, configured to receive user input text, preprocess the user input text, and convert the preprocessed user input text into first format data, where the first format data can be processed by the BERT layer;
[0034] A data processing module, used for inputting the first format data into a data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer;
[0035] A fully connected module, used to obtain output data of the data processing model, and input the output data into each fully connected layer corresponding to each task;
[0036] The output module is used to perform task output processing on the data output by each fully connected layer and output the processed data.
[0037] On the other hand, a data processing device for a medical question answering system is provided, the data processing device for a medical question answering system comprising:
[0038] processor;
[0039] A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the method provided by the embodiment of the present invention.
[0040] On the other hand, a computer-readable storage medium is provided, in which a program code is stored. The program code can be called by a processor to execute the method provided by an embodiment of the present invention.
[0041] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0042] The embodiment of the present invention processes the preprocessed user input text through a data processing model, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer, and then performs multiple fully connected layer processing and task output processing corresponding to multiple tasks on the data output by the data processing model, thereby achieving more accurate natural language understanding while processing multiple tasks at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0044] Figure 1 is a flow chart of a data processing method for a medical question-answering system provided by an embodiment of the present invention;
[0045] Figure 2 It is a principle design diagram of a data processing method for a medical question-answering system provided by an embodiment of the present invention;
[0046] Figure 3 is a schematic diagram of the system architecture of a data processing method for a medical question-and-answer system provided by an embodiment of the present invention;
[0047] Figure 4 is a block diagram of a data processing device for a medical question-and-answer system provided by an embodiment of the present invention;
[0048] Figure 5 It is a structural schematic diagram of a data processing device for a medical question-and-answer system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0050] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0051] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.
[0052] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.
[0053] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0054] In order to solve the technical problem that the medical question-answering system in the prior art is difficult to achieve relatively accurate natural language understanding, the embodiment of the present invention provides a data processing method and device for a medical question-answering system. The technical solution is as follows:
[0055] like Figure 1As shown, the present invention provides a data processing method for a medical question answering system, the method comprising:
[0056] S1. Receive user input text, preprocess the user input text, and convert the preprocessed user input text into first format data, where the first format data can be processed by the BERT layer.
[0057] Optionally, the preprocessing includes:
[0058] Segment words and remove stop words.
[0059] Optionally, the first format data includes:
[0060] token IDs, segment IDs and attention mask.
[0061] S2. Input the first format data into a data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a Time Distributed Dropout layer.
[0062] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer architecture, which places special emphasis on the ability to capture bidirectional (or non-autoregressive) context.
[0063] The basis of BERT is the encoder part of Transformer. The Transformer model mainly consists of an attention mechanism and a feed-forward neural network. It allows the model to focus on all other words in the input sequence when processing each word, thereby capturing global dependencies, and through the multi-head attention mechanism, the model can learn different feature representations in different subspaces. Because Transformer has no built-in sequential information, it is necessary to inject the position information of the sequence through position encoding. Unlike previous language models, BERT is bidirectional, which means that it takes into account contextual information on both the left and right sides. This allows BERT to better understand the meaning of words in sentences.
[0064] BERT pre-training consists of two main tasks:
[0065] Masked Language Model (MLM): 15% of the words in the input sequence are randomly selected and replaced with a special (MASK) token, and the model is then asked to predict what these masked words are.
[0066] Next Sentence Prediction (NSP): Sentences are provided in pairs in the input sequence, and the model is asked to predict whether the second sentence is the successor of the first sentence, which can help the model understand sentence-level relationships and logic.
[0067] Through bidirectional training, BERT can capture rich contextual information and is more effective than traditional unidirectional language models. The pre-trained BERT layer can be easily adapted to various downstream tasks by fine-tuning the pre-trained model. By pre-training on large-scale text data, BERT has powerful language understanding capabilities and can achieve excellent performance in a variety of natural language processing tasks.
[0068] Bidirectional long short-term memory network (BiLSTM) is a variant of recurrent neural network (RNN) that combines the advantages of standard LSTM and bidirectional propagation, and is particularly suitable for sequence data modeling, such as text processing tasks in natural language processing. BiLSTM improves the traditional unidirectional LSTM by considering both past and future contextual information at each time step. Its structure is as follows:
[0069] Forward LSTM:
[0070] in, is the forward hidden state at time step t, is the forward hidden state at the previous time step, x t is the input sequence at time step t.
[0071] Backward LSTM:
[0072] in, is the forward hidden state at time step t, is the forward hidden state at the previous time step, x t is the input sequence at time step t.
[0073] The output of the BiLSTM is the concatenation of the forward and backward hidden states:
[0074]
[0075] Among them, [;] represents a connection operation.
[0076] Bidirectional Gated Recurrent Unit (BiGRU) is a neural network model that combines GRU (Gated Recurrent Unit) and bidirectional propagation. It can more comprehensively capture the contextual relationship in the sequence by considering both the forward and backward information of the sequence data at each time step. BiGRU consists of two GRU layers: a forward GRU and a backward GRU. The output of each time step is the concatenation of the outputs of the two GRU layers. Its structure is similar to BiLSTM.
[0077] S3. Obtain output data of the data processing model, and input the output data into each fully connected layer corresponding to each task.
[0078] Optionally, the inputting the output data into each fully connected layer corresponding to each task respectively includes:
[0079] The output data is input into the first fully connected layer corresponding to the entity recognition task.
[0080] The output data is input into the second fully connected layer corresponding to the intent recognition task.
[0081] The output data is input into the third fully connected layer corresponding to the slot filling task.
[0082] Optionally, the output dimension of the first fully connected layer is batch size , seq length , num_entity_labels.
[0083] The output dimension of the second fully connected layer is batch size , num_intent_labels.
[0084] The output dimension of the third fully connected layer is batch size ,seq_length,num_slot_labels.
[0085] S4. Perform task output processing on the data output by each fully connected layer and output the processed data.
[0086] Optionally, performing task output processing on the data output by each fully connected layer and outputting the processed data includes:
[0087] The data output by the first fully connected layer is input into the first CRF layer for processing and the processed data is output.
[0088] The data output by the second fully connected layer is input into the softmax layer for classification, processed by combining DPCNN and multi-head attention mechanism, and the processed data is output.
[0089] The data output by the third fully connected layer is input into the second CRF layer for processing and the processed data is output.
[0090] Conditional Random Field (CRF) is a probabilistic graph model for sequence labeling tasks, especially suitable for sequence data in natural language processing, such as named entity recognition (NER), part-of-speech tagging (POSTagging) and other tasks. CRF can model the output sequence under the condition of a given input sequence, taking into account contextual information, thereby improving the accuracy of labeling.
[0091] Optionally, the formula of the first CRF layer is
[0092] y ner =CRF ner (H seq1 ), where H seq1 is the input data of the first CRF layer, y ner is the output data of the first CRF layer.
[0093] The formula for the second CRF layer is y slot =CRF slot (H seq2 ), where H seq2 is the input data of the second CRF layer, y slot is the output data of the second CRF layer.
[0094] In some embodiments, Figure 2 As shown, the present invention is implemented by building a sharing layer and a task layer.
[0095] Shared layer: This part of the model parameters is shared among all tasks. It includes the input layer, BERT encoding layer, BiGRU layer, BiLSTM layer, and Time Distributed Dropout layer. The output of the shared layer provides a common contextual representation for each task.
[0096] Task layer: Each task has its own dedicated layer, including a fully connected layer and a CRF layer, for prediction of a specific task.
[0097] Input layer: The input layer is responsible for receiving text input by the user. The text sequence is first preprocessed, including word segmentation, stop word removal, etc. Then, the processed text sequence is converted into an input format that the BERT layer can understand, namely the first format data, which includes token IDs, segment IDs, and attention mask (indicating the location of valid words in the input text to avoid processing the padding part).
[0098] BERT layer: Use the pre-trained BERT layer to convert the input word ID sequence into a high-dimensional word embedding representation. The BERT layer can capture the semantic information of words in context through the bidirectional Transformer structure, thereby generating rich word embedding representations. BERT consists of multiple layers of Transformers, each of which contains a multi-head self-attention mechanism and a feedforward neural network. BERT generates a contextual representation of each token through bidirectional encoding (from left to right and from right to left). After the input text sequence passes through the BERT encoding layer, an output tensor with dimensions of batch_size, seq_length, hidden_sizebatch is generated, where hidden_size is 768.
[0099] BiGRU layer: The BiGRU layer receives the output of BERT, extracts sequence features, processes and captures global context information. The BiGRU consists of two GRU layers, one processes the sequence from front to back, and the other processes the sequence from back to front. The output of the BiGRU layer is the concatenation of the outputs of the two GRU layers, with dimensions batch_size, seq_length, 2\*hidden_size.
[0100] BiLSTM layer: The BiLSTM layer is placed after the BiGRU layer to further process features and capture finer-grained context representations. The BiLSTM layer consists of two LSTM layers, one that processes the sequence from front to back and the other that processes the sequence from back to front. The output of the BiLSTM layer is the concatenation of the outputs of the two LSTM layers, with dimensions of batch_size, seq_length, 2\*hidden_size.
[0101] Time Distributed Dropout layer: To prevent overfitting, a TimeDistributed Dropout layer is added after the BiLSTM layer.
[0102] CRF layer: The CRF (Conditional Random Field) layer is used for sequence labeling tasks to improve the accuracy of labeling by considering the dependencies between labels. The CRF layer calculates the score of each label sequence and selects the label sequence with the highest score as the output.
[0103] Assume that the input sequence is X=(x 1 ,x 2 ,...,x t ), where t is the sentence length.
[0104] In practical applications, BERT is first used to extract the context representation of the input sentence:
[0105] H = BERT(x) = [h 1 ,h 2 ,...,h t ]
[0106] Here, H is a matrix of dimension t×d, and d is the hidden layer dimension of BERT output.
[0107] BiGRU-BiLSTM is used for feature extraction. The BiGRU layer and BiLSTM layer are superimposed on the output of the BERT layer to further extract semantic features and contextual information in the input sequence. The BiGRU layer and BiLSTM layer have strong sequence modeling capabilities and can effectively capture long-distance dependencies and sentence structure information.
[0108] The data processed by the BERT layer is used as the input of the BiGRU layer. The output of the BiGRU layer is calculated as follows:
[0109] Forward GRU output:
[0110] Backward GRU output:
[0111] Among them, BERT(x t ) represents the BERT layer for the input sequence x t The encoding result is is the forward hidden state at the previous time step, x t is the input sequence at time step t, is the backward hidden state at the previous time step.
[0112] Combine the hidden states in both directions: Among them, [;] represents a connection operation.
[0113] BiLSTM layer: Based on the output of the BiGRU layer, the BiLSTM layer is further used to strengthen the capture of sequence features, and the output of the BiGRU layer is used as the input of the BiLSTM layer:
[0114] Forward LSTM output:
[0115] Backward LSTM output:
[0116] Combine the hidden states in both directions: Among them, [;] represents a connection operation.
[0117] For the task layer, each task has its own dedicated fully connected layer, where the fully connected layer is used to convert the output of the shared layer into predictions for a specific task. The output dimensions of the fully connected layer for different tasks are different:
[0118] Entity recognition: The output dimensions are batch_size, seq_length, num_entity_labels.
[0119] Intent recognition: The output dimension is batch_size, num_intent_labels.
[0120] Slot filling: The output dimensions are batch_size, seq_length, num_slot_labels.
[0121] The CRF layer (only applicable to tasks that require sequence labeling) is used to receive the output of the fully connected layer and perform sequence labeling prediction. The output is a label sequence for each word. After the data is processed by the BiGRU layer and the BiLSTM layer, the sequence can be labeled by the conditional random field (CRF) layer for entity recognition and slot filling tasks. The CRF layer can capture the dependencies between labels in the sequence and further improve the accuracy of the labeling task.
[0122] For entity recognition and slot filling tasks, two CRF layers are designed respectively, where the first CRF layer is used for the entity recognition task and the second CRF layer is used for the slot filling task.
[0123] Optionally, the formula of the first CRF layer is ner =CRF ner (H seq1 ), where H seq1 is the input data of the first CRF layer, y ner is the output data of the first CRF layer.
[0124] The formula for the second CRF layer is yslot =CRF slot (H seq2 ), where H seq2 is the input data of the second CRF layer, y slot is the output data of the second CRF layer.
[0125] For the intent recognition task, a fully connected layer and a softmax layer are used for classification, and DPCNN and a multi-head attention mechanism are used to help capture key phrases and vocabulary patterns in intent recognition, thereby improving the accuracy of intent recognition.
[0126] y intent =softmax(Wh DPCNN +b)
[0127] Among them, y intent is the output probability distribution of intent recognition, W is the weight matrix of the fully connected layer, and h DPCNN is the feature representation obtained by DPCNN and multi-head attention mechanism, and b is the bias term of the fully connected layer.
[0128] The loss function of multiple tasks is composed of the weighted sum of the loss functions of each task. The details are as follows:
[0129] Entity recognition loss: The CRF loss function is used to calculate the loss of the entity recognition task.
[0130] Intent recognition loss: The cross entropy loss function is used to calculate the loss of the intent recognition task.
[0131] Slot filling loss: The CRF loss function is used to calculate the loss of the slot filling task.
[0132] The total loss function is defined as:
[0133] L total =αL ner +βL intent +γL slot
[0134] Among them, α, β, and γ are weight hyperparameters used to balance the losses of different tasks. ner is the entity recognition loss, L intent To identify the loss of intention, L slot Fill the loss for the slot.
[0135] In some embodiments, Figure 3 As shown, the Django framework can be used to integrate the question-answering algorithm model for system development and testing. The text input by the user can be processed through entity recognition, intent recognition, etc. to understand the question, generate Cypher language, search in the knowledge graph database, and return the answer to the question.
[0136] The overall system is divided into three layers: data layer, model layer, and application layer.
[0137] Data layer: mainly obtains medical data through web crawlers, pre-processes text data, and extracts medical-related information from structured and semi-structured data. For the extracted medical information, a medical knowledge graph is constructed and stored in the Neo4j database. When the business service calls data operations, Cypher statements are used to search and maintain the database, realizing data access and knowledge addition, deletion, modification and query.
[0138] Model layer: In this layer, the input text data is semantically analyzed, and the entity recognition and intent recognition models are used to extract key entity and intent information. Subsequently, the Cypher query statement is constructed using slot filling technology, and information is retrieved in the Neo4j database by calling the data layer service to obtain the corresponding answer.
[0139] Application layer: mainly realizes the visualization and interactive functions of the system, including the visualization of knowledge graphs and the realization of question and answer.
[0140] like Figure 4 As shown, the present invention further provides a data processing device for a medical question-answering system, the data processing device for a medical question-answering system is used to implement the data processing method for a medical question-answering system provided in an embodiment of the present invention, and the device includes:
[0141] The receiving module 401 is used to receive user input text, preprocess the user input text, and convert the preprocessed user input text into a first format data, which can be processed by the BERT layer.
[0142] The data processing module 402 is used to input the first format data into the data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer.
[0143] The fully connected module 403 is used to obtain the output data of the data processing model and input the output data into each fully connected layer corresponding to each task.
[0144] The output module 404 is used to perform task output processing on the data output by each fully connected layer and output the processed data.
[0145] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0146] The embodiment of the present invention processes the preprocessed user input text through a data processing model, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer, and then performs multiple fully connected layer processing and task output processing corresponding to multiple tasks on the data output by the data processing model, thereby achieving more accurate natural language understanding while processing multiple tasks at the same time.
[0147] Figure 5 FIG. 1 is a schematic diagram of a data processing device for a medical question-answering system provided by an embodiment of the present invention. Figure 5 As shown, optionally, the data processing device 510 for the medical question answering system may include a first processor 2001 .
[0148] Optionally, the data processing device 510 for the medical question answering system may further include a memory 2002 and a transceiver 2003 .
[0149] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.
[0150] Combine the following Figure 5 The components of the data processing device 510 used in the medical question-answering system are specifically introduced as follows:
[0151] The first processor 2001 is the control center of the data processing device 510 for the medical question-answering system, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).
[0152] Optionally, the first processor 2001 can perform various functions of the data processing device 510 for the medical question and answer system by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .
[0153] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.
[0154] In a specific implementation, as an embodiment, the data processing device 510 for the medical question answering system may also include multiple processors, such as Figure 5 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0155] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.
[0156] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the computer through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.
[0157] The transceiver 2003 is used to communicate with a network device or a terminal device.
[0158] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.
[0159] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.
[0160] It should be noted that Figure 5 The structure of the data processing device 510 for the medical question-answering system shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0161] In addition, the technical effects of the data processing device 510 for the medical question and answer system can refer to the technical effects of the data processing method for the medical question and answer system described in the above method embodiment, and will not be repeated here.
[0162] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0163] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0164] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by infrared, wireless, microwave or the like. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0165] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.
[0166] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0167] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0168] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0170] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0172] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0173] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0174] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A data processing method for a medical question answering system, It is characterized in that The method comprises: Receiving user input text, preprocessing the user input text, and converting the preprocessed user input text into first format data, where the first format data can be processed by the BERT layer; Inputting the first format data into a data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer; Obtain output data of the data processing model, and input the output data into each fully connected layer corresponding to each task; The data output by each fully connected layer is processed for task output and the processed data is output.
2. The method according to claim 1, It is characterized in that The step of inputting the output data into each fully connected layer corresponding to each task includes: Inputting the output data into the first fully connected layer corresponding to the entity recognition task; Inputting the output data into the second fully connected layer corresponding to the intent recognition task; The output data is input into the third fully connected layer corresponding to the slot filling task.
3. The method according to claim 2, It is characterized in that The output dimension of the first fully connected layer is batch size , seq length , num_entity_labels; The output dimension of the second fully connected layer is batch size , num_intent_labels; The output dimension of the third fully connected layer is batch size ,seq_length,num_slot_labels.
4. The method according to claim 2, It is characterized in that The step of performing task output processing on the data output by each fully connected layer and outputting the processed data comprises: The data output by the first fully connected layer is input into the first CRF layer for processing and the processed data is output; The data output from the second fully connected layer is input into the softmax layer for classification, processed by combining DPCNN and multi-head attention mechanism, and the processed data is output; The data output by the third fully connected layer is input into the second CRF layer for processing and the processed data is output.
5. The method according to claim 4, It is characterized in that The formula of the first CRF layer is y ner =CRF ner (H seq1 ), where H seq1 is the input data of the first CRF layer, y ner is the output data of the first CRF layer; The formula for the second CRF layer is y slot =CRF slot (H seq2 ), where H seq2 is the input data of the second CRF layer, y slot is the output data of the second CRF layer.
6. The method according to claim 1, It is characterized in that The preprocessing comprises: Segment words and remove stop words.
7. The method according to claim 1, It is characterized in that The first format data includes: token IDs, segment IDs and attention mask.
8. A data processing device for a medical question-answering system, the data processing device for a medical question-answering system being used to implement the data processing method for a medical question-answering system as claimed in any one of claims 1 to 7, It is characterized in that The device comprises: A receiving module, configured to receive user input text, preprocess the user input text, and convert the preprocessed user input text into first format data, where the first format data can be processed by the BERT layer; A data processing module, used for inputting the first format data into a data processing model for data processing, wherein the first layer of the data processing model is a BERT layer, the second layer is a BiGRU layer, the third layer is a BiLSTM layer, and the fourth layer is a TimeDistributed Dropout layer; A fully connected module, used to obtain output data of the data processing model, and input the output data into each fully connected layer corresponding to each task; The output module is used to perform task output processing on the data output by each fully connected layer and output the processed data.
9. A data processing device for a medical question-answering system, It is characterized in that The data processing device for the medical question answering system comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.
Citation Information
Cited By
Large language model retrieval enhancement method and device, electronic equipment and storage medium
CN119829744A
Large language model retrieval augmentation method and apparatus, electronic device, and storage medium
WO2026149071A1
