Intelligent medical question-answering system based on pre-training language model

Abstract

The invention relates to the technical field of deep learning, in particular to an intelligent medical question answering system based on a pre-training language model. Comprising a natural language understanding module used for carrying out natural language processing on user input and extracting key information; the knowledge base management module is used for querying a knowledge graph based on the key information; the pre-training language model module is used for generating a responded natural language based on a pre-training language model and a query result of the knowledge base management module; the multi-modal data processing module is used for fusing and analyzing the query result of the knowledge base management module based on a multi-modal model; and the self-supervised learning module is connected with the multi-modal data processing module and the pre-training language model module and is used for optimizing the pre-training language model and the multi-modal model. Natural language questions of users can be effectively processed, accurate and professional answers can be provided, answering strategies can be continuously optimized and adjusted, the satisfaction degree of the users is improved, and the method has the capability of processing multi-modal data.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an intelligent medical question-answering system based on a pre-trained language model. Background Art

[0002] Traditional medical question-and-answer systems rely mainly on two approaches: rules and knowledge bases. Rule-based systems generate answers using predefined rules and logic, using keywords to match user input. These systems have a rule engine inside that triggers corresponding rules based on the input keywords to find relevant answers. After the user enters the question, the system returns the preset answer by matching keywords. Knowledge base-based systems use a pre-built medical knowledge base, which contains a large number of question-answer pairs and related information. The system queries the knowledge base to find the answer that best matches the user's question. In this process, the user enters the question, the system searches the knowledge base, finds the most relevant question-answer pair and returns the answer.

[0003] With the development of deep learning and natural language processing technology, modern medical question-answering systems have gradually adopted pre-trained language models and other advanced technologies. Systems based on pre-trained language models, such as GPT-4 and BERT, have been fine-tuned in the medical field and use complex neural network structures to process user input and generate high-quality answers. After the user enters a question, the system understands the question through the pre-trained model and generates the corresponding answer.

[0004] In addition, the multimodal learning system combines text and image data, uses models such as ViLBERT and UNITER, integrates text processing modules and image processing modules, and generates answers through multimodal data fusion. After the user enters the question and related images, the system can provide a comprehensive answer.

[0005] Self-supervised learning systems use large-scale unlabeled medical data for pre-training to improve the performance of the model on medical texts. These models are first pre-trained on a large amount of unlabeled data, then fine-tuned on labeled data, and finally applied to actual question answering.

[0006] The reinforcement learning system continuously improves the quality of answers based on user feedback. The system includes a reinforcement learning module that uses user feedback to adjust model parameters to optimize answer performance. After the user enters a question, the system generates a preliminary answer and uses reinforcement learning to further optimize model performance based on user feedback.

[0007] However, when processing user questions, existing medical question-and-answer systems often lack accuracy and professionalism in their answers due to the limitations of natural language understanding; existing systems have weak learning capabilities and lack a mechanism to continuously optimize and adjust answer strategies based on user feedback, which results in the system being unable to flexibly respond to the needs of different users and affects user satisfaction; they only support text data and lack the ability to process multimodal data (such as images and text), which limits the comprehensiveness and accuracy of information. When users upload medical images, the system cannot effectively combine image information to provide detailed answers. Summary of the invention

[0008] The purpose of the present invention is to provide an intelligent medical question-answering system based on a pre-trained language model to solve the above technical problems;

[0009] An intelligent medical question-answering system based on a pre-trained language model, comprising:

[0010] The natural language understanding module is used to process user input into natural language and extract key information;

[0011] A knowledge base management module, connected to the natural language understanding module, for querying the knowledge graph based on the key information;

[0012] A pre-trained language model module, connected to the knowledge base management module, generates a responsive natural language based on the pre-trained language model and the query result of the knowledge base management module;

[0013] A multimodal data processing module, connected to the knowledge base management module, for analyzing the query results of the knowledge base management module based on a multimodal model fusion;

[0014] A self-supervised learning module is connected to the multimodal data processing module and the pre-trained language model module, and is used to optimize the pre-trained language model and the multimodal model.

[0015] In a preferred embodiment, the natural language understanding module includes:

[0016] A preprocessing unit, for performing word segmentation and / or tagging and / or syntactic analysis on user input and identifying entities in the user input;

[0017] An intention recognition unit, connected to the preprocessing unit, for recognizing and classifying the intention of the user input;

[0018] A slot filling unit, connected to the preprocessing unit, is used to extract key entities in the user input and map the entities to the slots.

[0019] In a preferred embodiment, the knowledge base management module includes:

[0020] A knowledge representation unit, constructing the knowledge graph and defining entity relationships in the knowledge graph;

[0021] A knowledge retrieval unit, connected to the knowledge representation unit, for retrieving and querying the knowledge graph;

[0022] A knowledge updating unit, connected to the knowledge representation unit, is used to update the knowledge graph.

[0023] In a preferred embodiment, the pre-trained language model module includes:

[0024] A pre-training unit, used to construct a language model and perform pre-training to obtain the pre-trained language model;

[0025] A model adjustment unit is connected to the pre-training unit and is used to adjust the pre-trained language model according to a specific task.

[0026] In a preferred embodiment, the multimodal data processing module includes:

[0027] A data processing unit, used for processing user input to obtain multimodal data;

[0028] The multimodal fusion unit is connected to the data processing unit, extracts the features of the multimodal data through the multimodal model, and fuses the features of different modalities.

[0029] In a preferred embodiment, the self-supervised learning module includes:

[0030] Self-supervised unit, which learns feature representation from unlabeled data through self-supervised tasks;

[0031] Unsupervised training unit, which obtains universal feature representation by learning from unlabeled data.

[0032] In a preferred embodiment, the self-supervision task includes masked language training and contrastive learning training.

[0033] In a preferred embodiment, a reinforcement learning module is further included, connecting the multimodal data processing module and the pre-trained language model module, and the reinforcement learning module learns the optimal question-answering strategy through interaction between the intelligent agent and the environment.

[0034] In a preferred embodiment, the reinforcement learning module further includes a reward unit for guiding the agent to learn through a reward mechanism.

[0035] In a preferred embodiment, it also includes a real-time response module, which is connected to the multimodal data processing module and the pre-trained language model module, and the real-time response module processes user requests in parallel based on a multi-threaded and asynchronous processing framework.

[0036] The beneficial effect of the present invention lies in that, through the above-mentioned technical scheme, the present invention can effectively process users' natural language questions, provide accurate and professional answers through deep learning and natural language processing technology, can continuously optimize and adjust the answer strategy, improve user satisfaction, have the ability to process multimodal data, and can understand and analyze text and image information at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of an intelligent medical question-answering system based on a pre-trained language model in an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of a natural language understanding module in an embodiment of the present invention;

[0039] Figure 3 is a schematic diagram of a knowledge base management module in an embodiment of the present invention;

[0040] Figure 4 Schematic diagram of a pre-trained language model module in an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of a multimodal data processing module in an embodiment of the present invention;

[0042] Figure 6 Schematic diagram of a self-supervised learning module in an embodiment of the present invention.

[0043] In the accompanying drawings: 1. Natural language understanding module; 11. Preprocessing unit; 12. Intent recognition unit; 13. Slot filling unit; 2. Knowledge base management module; 21. Knowledge representation unit; 22. Knowledge retrieval unit; 23. Knowledge updating unit; 3. Pre-trained language model module; 31. Pre-training unit; 32. Model adjustment unit; 4. Multimodal data processing module; 41. Data processing unit; 42. Multimodal fusion unit; 5. Self-supervised learning module; 51. Self-supervised unit; 52. Unsupervised training unit. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0047] An intelligent medical question-answering system based on a pre-trained language model, comprising:

[0048] Natural language understanding module 1, used to perform natural language processing on user input and extract key information;

[0049] The knowledge base management module 2 is connected to the natural language understanding module 1 and is used to query the knowledge graph based on key information;

[0050] The pre-trained language model module 3 is connected to the knowledge base management module 2 and generates a responsive natural language based on the pre-trained language model and the query result of the knowledge base management module 2;

[0051] The multimodal data processing module 4 is connected to the knowledge base management module 2 and is used to analyze the query results of the knowledge base management module 2 based on the multimodal model fusion;

[0052] The self-supervised learning module 5 is connected to the multimodal data processing module 4 and the pre-trained language model module 3, and is used to optimize the pre-trained language model and the multimodal model.

[0053] Specifically, the present invention provides an intelligent medical question-and-answer system based on a pre-trained language model. The natural language understanding module 1 analyzes the questions input by the user, extracts key information, and retrieves related medical information through the knowledge base management module 2. The pre-trained language model module 3 provides accurate and professional answers based on the pre-trained language model and the query results of the knowledge base management module 2. The multimodal data processing module 4 enables the intelligent medical question-and-answer system to have the ability to process multimodal data, and can simultaneously understand and analyze text and image information. At the same time, the self-supervised learning module 5 can continuously optimize and adjust the answer strategy to improve user satisfaction.

[0054] In a preferred embodiment, Figure 2 As shown, the natural language understanding module 1 includes:

[0055] A preprocessing unit 11, configured to perform word segmentation and / or annotation and / or syntactic analysis on user input and identify entities in the user input;

[0056] The intention recognition unit 12 is connected to the preprocessing unit 11 and is used to recognize and classify the intention of the user input;

[0057] The slot filling unit 13 is connected to the preprocessing unit 11 and is used to extract key entities in the user input and map the entities to the slots.

[0058] Specifically, the preprocessing unit 11 uses word segmentation tools, such as NLTK and Spacy, to divide the text into words, mark the part of speech of each word, such as noun, verb, etc., identify entities in the text, such as names of people, places, names of medicines, etc., parse the syntactic structure of the text, and understand the relationship between sentences.

[0059] The intent recognition unit 12 uses a pre-trained model, such as BERT and RoBERTa, to perform fine-tuning and train an intent classification model. The conversation data in the medical field is used for training to ensure that the model can recognize common medical intents.

[0060] The slot filling unit 13 uses the NER model to extract key entities in the user input and maps the extracted entities to predefined slots, such as symptoms, time, drug name, etc.

[0061] In a preferred embodiment, the knowledge base management module 2 includes:

[0062] The knowledge representation unit 21 constructs a knowledge graph and defines entity relationships in the knowledge graph;

[0063] The knowledge retrieval unit 22 is connected to the knowledge representation unit 21 and is used to retrieve and query the knowledge graph;

[0064] The knowledge updating unit 23 is connected to the knowledge representation unit 21 and is used to update the knowledge graph.

[0065] Specifically, the knowledge representation unit 21 extracts knowledge from data sources such as medical literature and electronic medical records to construct a knowledge graph and define the relationship between entities, such as the relationship between symptoms and diseases, the relationship between drugs and diseases, etc.

[0066] The knowledge retrieval unit 22 uses a graph database such as Neo4j to store and manage the knowledge graph, optimizes query statements, and improves the efficiency of knowledge retrieval.

[0067] The knowledge updating unit 23 is used to regularly monitor and crawl data sources such as medical literature and databases. The knowledge graph is automatically updated using data processing and knowledge extraction technology to ensure the accuracy and timeliness of knowledge.

[0068] In a preferred embodiment, the pre-trained language model module 3 includes:

[0069] A pre-training unit 31 is used to construct a language model and perform pre-training to obtain a pre-trained language model;

[0070] The model adjustment unit 32 is connected to the pre-training unit 31 and is used to adjust the pre-trained language model according to a specific task.

[0071] Specifically, the pre-training unit 31 selects a pre-training model such as BERT, GPT, etc. Large-scale general corpus, such as Wikipedia and BooksCorpus, is used for pre-training.

[0072] The model adjustment unit 32 uses text data in the medical field, such as electronic medical records and medical literature, to perform fine-tuning, and defines specific downstream tasks, such as question answering, text classification, entity recognition, etc., for fine-tuning training.

[0073] In a preferred embodiment, the multimodal data processing module 4 includes:

[0074] A data processing unit 41 is used to process user input to obtain multimodal data;

[0075] The multimodal fusion unit 42 is connected to the data processing unit 41, extracts the features of the multimodal data through a multimodal model, and fuses the features of different modalities.

[0076] Specifically, the data processing unit 41 performs preprocessing such as word segmentation, part-of-speech tagging, and NER on the text data, and performs image preprocessing such as normalization and data enhancement on the image data.

[0077] The multimodal fusion unit 42 uses a multimodal Transformer or a model that integrates an attention mechanism to extract features, such as text features and image features, from data of different modalities, and integrates the features of different modalities through an attention mechanism or a fusion layer.

[0078] In a preferred embodiment, the self-supervised learning module 5 includes:

[0079] Self-supervised unit 51, which learns feature representation from unlabeled data through self-supervised tasks;

[0080] The unsupervised training unit 52 obtains universal feature representation by learning unlabeled data.

[0081] Specifically, the masked language training of the self-supervisory unit 51 specifically involves randomly masking some words, and training the model to predict the masked words. The contrastive learning training involves generating positive and negative sample pairs, and training the model to distinguish between positive and negative samples.

[0082] The unsupervised training unit 52 selects large-scale unlabeled medical data, such as electronic medical records and medical documents, designs self-supervised tasks suitable for the medical field, and performs unsupervised pre-training.

[0083] In a preferred embodiment, the self-supervised tasks include masked language training and contrastive learning training.

[0084] In a preferred embodiment, a reinforcement learning module is also included, which is connected to the multimodal data processing module 4 and the pre-trained language model module 3. The reinforcement learning module learns the optimal question-answering strategy through the interaction between the intelligent agent and the environment.

[0085] In a preferred embodiment, the reinforcement learning module also includes a reward unit for guiding the agent to learn through a reward mechanism.

[0086] Specifically, the reinforcement learning module learns the optimal question-answering strategy through the interaction between the agent and the environment, and guides the agent to learn through the reward mechanism; the reward mechanism includes:

[0087] When the agent outputs the correct answer, a positive reward is added to the agent;

[0088] Add a negative reward to the agent when it outputs an incorrect answer.

[0089] Specifically, the existing knowledge base is updated at a low frequency and cannot reflect the latest medical research and treatment methods in a timely manner, resulting in the information provided being outdated or inaccurate, affecting the practicality and reliability of the system. The reinforcement learning module of the present invention first defines the state of the system, such as user input, system response, etc., and defines the actions that the agent can take, such as answering questions, requesting more information, etc. Using reinforcement learning algorithms such as DQN and PPO, the strategy is continuously updated through the interaction between the agent and the environment to learn the optimal behavior.

[0090] In a preferred embodiment, it also includes a real-time response module connected to the multimodal data processing module 4 and the pre-trained language model module 3, and the real-time response module processes user requests in parallel based on a multi-threaded and asynchronous processing framework.

[0091] Specifically, when processing requests, traditional systems often have long response times due to insufficient computing resources and lack of efficient cache management mechanisms, which affects user experience. Users have slow query speeds for common problems and cannot achieve instant feedback. The real-time response module of the present invention uses a microservice architecture to decompose system functions and improve response speed. Use load balancing technology to distribute requests to different servers to improve system response speed. Use multi-threading technology to process multiple requests at the same time. Use asynchronous processing frameworks, such as Node.js and Asyncio, to improve the concurrent processing capabilities of the system.

[0092] In a specific embodiment, the user inputs a question through text or voice, and the natural language understanding module 1 performs preprocessing, intent recognition, and slot filling. The key information extracted by the natural language understanding module 1 is passed to the knowledge base management module to perform a knowledge graph query and obtain relevant medical knowledge. The pre-trained language model module 3 generates a natural language response based on the query results. If multimodal data is involved, such as images and structured data, the multimodal data processing module 4 will perform a fusion analysis and provide a comprehensive result. The self-supervised learning module 5 continuously learns from unlabeled data to improve the performance of the language model and the multimodal model. The reinforcement learning module 6 continuously optimizes the question-and-answer strategy through user feedback to improve the intelligence of the system. The real-time response module ensures that the system processes user requests quickly and provides a smooth interactive experience.

[0093] In a specific embodiment, the implementation process of the natural language understanding module 1 includes:

[0094] The user inputs a question, such as "What medicine should I take for headache?" The system recognizes the user's intention to "ask for treatment methods" through the intent recognition submodule. The entity extraction submodule extracts the key entity "headache" from the question.

[0095] The implementation process of the knowledge base management module 2 includes:

[0096] According to the extracted key entity "headache", relevant treatment methods are retrieved in the structured knowledge base. If no relevant information is found in the structured knowledge base, a full-text search is performed in the unstructured knowledge base. The dynamic update submodule regularly extracts knowledge from the latest medical literature and updates the knowledge base.

[0097] The implementation process of the pre-trained language model module 3 includes:

[0098] The domain fine-tuning submodule fine-tunes the pre-trained language model, such as GPT-3, on medical datasets. The model reasoning submodule generates answers based on user questions and retrieved knowledge. The GPT-3 model is used and fine-tuned on medical datasets, such as MIMIC-III. Combined with the information in the knowledge base, accurate answers are generated through model reasoning.

[0099] The implementation process of the multimodal data processing module 4 includes:

[0100] If the user's question contains image data, such as an uploaded X-ray, the image processing submodule analyzes the image. The text processing submodule analyzes the text information in the user's question. The data fusion submodule combines image and text features to generate a comprehensive answer.

[0101] The implementation process of self-supervised learning module 5 includes:

[0102] The contrastive learning submodule learns features from a large amount of unlabeled medical data, and the data enhancement submodule generates more training data through data enhancement techniques such as rotation and cropping.

[0103] The implementation process of reinforcement learning module 6 includes:

[0104] The user provides feedback on the system's answers, the user feedback processing submodule collects the feedback information, and the policy gradient submodule optimizes the answer strategy based on the feedback information.

[0105] The implementation process of the real-time response module includes:

[0106] After the user asks a question, the system processes it in parallel through the distributed computing submodule. The cache management submodule caches the answers to common questions to reduce repeated calculations.

[0107] Taking the user input "What medicine should I take for headache?" as an example, the specific implementation process of the system is as follows:

[0108] Step 1, the natural language understanding module 1 recognizes the user's intention as "asking for treatment methods" and extracts the key entity "headache".

[0109] Step 2, the knowledge base management module 2 retrieves the treatment method of "headache" in the structured knowledge base and returns "ibuprofen or acetaminophen can be used".

[0110] Step 3: The pre-trained language model module 3 combines the retrieved knowledge and generates an answer through the fine-tuned GPT-3 model: "For headaches, you can consider using ibuprofen or acetaminophen. Please use it as prescribed by your doctor."

[0111] Step 4, Multimodal Data Processing Module 4 If the user uploads relevant images, such as head CT scans, the system will provide a more detailed answer based on the image information.

[0112] Step 5, the self-supervised learning module 5 learns more features from unlabeled data through comparative learning to improve the system's ability to answer similar questions.

[0113] Step 6, the reinforcement learning module 6 continuously optimizes the answer strategy based on user feedback, such as user satisfaction with the answer.

[0114] Step 7: The real-time response module ensures that the system can quickly respond to user questions through distributed computing and cache management.

[0115] The accuracy of the natural language understanding module 1 on the test dataset increased by 15%, from 75% to 90%. Using the fine-tuned GPT-3 model, the accuracy of the answer increased by 20%.

[0116] The real-time response module greatly improves the system's processing power and response speed through distributed computing and cache management. The cache management submodule reduces repeated calculations and can quickly return results for common problems. The average system response time is reduced from 3 seconds to 1 second, an increase of about 66% in response speed. The response time for common problems is shortened to 0.5 seconds, improving the user experience.

[0117] The self-supervised learning module 5 uses contrastive learning and data enhancement techniques to learn useful features from unlabeled data and continuously improve the generalization ability of the model. The reinforcement learning module 6 continuously optimizes the answer strategy based on user feedback so that the system can adapt to the needs of different users. The system's performance on unlabeled data sets has improved by 10%, from the original 85% to 95%. The user satisfaction survey shows that the system's answer satisfaction has increased from the original 70% to 85%.

[0118] The multimodal data processing module 4 can process and fuse image and text data to provide more comprehensive and accurate medical consultation services. Through the image processing submodule, the system can analyze the medical images uploaded by users, such as X-rays and CT scans, and provide more detailed answers in combination with text information. The accuracy on the multimodal data set increased by 12%, from the original 78% to 90%. The user's satisfaction with the answer after uploading the image increased by 15%.

[0119] The knowledge base management module 2 supports the dynamic update of the knowledge base to ensure the timeliness and accuracy of the knowledge. The dynamic update submodule can regularly extract knowledge from the latest medical literature and update the knowledge base. The update frequency of the knowledge base has been increased from once a month to once a week, ensuring the timeliness of the knowledge. The coverage of the knowledge base has increased by 20%, from 80% to 100%.

[0120] In summary, the present application provides an intelligent medical question-and-answer system based on a pre-trained language model. The natural language understanding module 1 accurately understands the user's question intention and key entities through intent recognition and entity extraction, thereby improving the accuracy of question parsing. The knowledge base management module 2 retrieves the knowledge base and provides relevant medical knowledge based on the key information extracted by NLU. The pre-trained language model module 3 provides powerful language understanding and generation capabilities, supports NLU intent recognition and slot filling, and generates natural dialogue responses. The multimodal data processing module 4 combines multiple data such as text and images to provide comprehensive analysis results and support complex medical questions and answers. The self-supervised learning module 5 learns feature representations from a large amount of unlabeled data to improve the performance of pre-trained language models and multimodal models. The reinforcement learning module 6 optimizes the question-and-answer strategy through interaction with users, thereby improving the intelligence and response quality of the system. The real-time response module ensures that the system can respond quickly to user requests and provide a smooth user experience.

[0121] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent medical question-answering system based on a pre-trained language model, characterized in that: include, A natural language understanding module (1), used to perform natural language processing on user input and extract key information; A knowledge base management module (2), connected to the natural language understanding module (1), for querying the knowledge graph based on the key information; A pre-trained language model module (3) is connected to the knowledge base management module (2) and generates a responsive natural language based on the pre-trained language model and the query result of the knowledge base management module (2); A multimodal data processing module (4), connected to the knowledge base management module (2), for analyzing the query results of the knowledge base management module (2) based on a multimodal model fusion; A self-supervised learning module (5) is connected to the multimodal data processing module (4) and the pre-trained language model module (3) and is used to optimize the pre-trained language model and the multimodal model.

2. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: The natural language understanding module (1) comprises: A preprocessing unit (11), used for performing word segmentation and / or tagging and / or syntactic analysis on user input and identifying entities in the user input; An intention recognition unit (12), connected to the preprocessing unit (11), used to recognize and classify the intention of the user input; A slot filling unit (13), connected to the preprocessing unit (11), is used to extract key entities in the user input and map the entities to the slots.

3. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: The knowledge base management module (2) comprises: A knowledge representation unit (21) constructs the knowledge graph and defines entity relationships in the knowledge graph; A knowledge retrieval unit (22), connected to the knowledge representation unit (21), for retrieving and querying the knowledge graph; A knowledge updating unit (23), connected to the knowledge representation unit (21), is used to update the knowledge graph.

4. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: The pre-trained language model module (3) comprises: A pre-training unit (31), used for constructing a language model and performing pre-training to obtain the pre-trained language model; A model adjustment unit (32) is connected to the pre-training unit (31) and is used to adjust the pre-trained language model according to a specific task.

5. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: The multimodal data processing module (4) comprises: A data processing unit (41), used for processing user input to obtain multimodal data; A multimodal fusion unit (42) is connected to the data processing unit (41), extracts features of the multimodal data through the multimodal model, and fuses features of different modalities.

6. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: The self-supervised learning module (5) comprises: Self-supervised unit (51), which learns feature representations from unlabeled data through self-supervised tasks; The unsupervised training unit (52) obtains universal feature representations by learning from unlabeled data.

7. The intelligent medical question-answering system based on a pre-trained language model according to claim 6, characterized in that: The self-supervised tasks include masked language training and contrastive learning training.

8. The intelligent medical question-answering system based on a pre-trained language model according to claim 1, characterized in that: It also includes a reinforcement learning module, which is connected to the multimodal data processing module (4) and the pre-trained language model module (3), and the reinforcement learning module learns the optimal question-answering strategy through the interaction between the intelligent agent and the environment.

9. The intelligent medical question-answering system based on a pre-trained language model according to claim 8, characterized in that: The reinforcement learning module also includes a reward unit for guiding the agent to learn through a reward mechanism.

10. The intelligent medical question-answering system based on a pre-trained language model according to claim 8, characterized in that: It also includes a real-time response module, which is connected to the multimodal data processing module (4) and the pre-trained language model module (3), and the real-time response module processes user requests in parallel based on a multi-threaded and asynchronous processing framework.

Images