Intelligent medical diagnosis engine based on AI large model

Abstract

The invention relates to the technical field related to artificial intelligence, in particular to an intelligent medical diagnosis engine based on an AI large model, which comprises an auxiliary diagnosis module for collecting data such as an electronic medical record and a medical image of a patient and predicting an auxiliary result by using an integrated learning method; the drug recommendation module is used for analyzing drug recommendation data by using an AI large model and predicting potential curative effects and side effects of new drugs; and the operation simulation module is used for constructing an operation environment and a human body model in combination with a virtual reality technology. A large amount of medical data can be quickly processed and analyzed through auxiliary diagnosis, the diagnosis time is shortened, gene, physiological and pathological multi-omics data of a patient can be integrated by reducing unnecessary invasive examination and misdiagnosis and drug recommendation, a personalized drug treatment scheme is provided for the patient, and the patient experience is improved. Comprehensive and accurate diagnosis basis and treatment suggestions are provided for doctors; by accurately simulating a real operation scene including mechanical and optical characteristics, a more real training experience is provided for doctors and medical students.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly to an intelligent medical diagnosis engine based on an AI large model. Background Art

[0002] With the rapid development of information technology, the application of artificial intelligence (AI) technology in the medical field has become increasingly widespread. The application of AI in the field of medical health is becoming more and more extensive, showing great potential in many aspects from auxiliary diagnosis to personalized treatment, greatly promoting the innovation and upgrading of the medical service model. As a new type of medical service model, intelligent medicine has become the development trend of the medical industry by using AI technology to improve the efficiency and quality of medical services.

[0003] However, despite the remarkable achievements, the current AI-based intelligent medical diagnosis models still face some challenges and deficiencies, which are specifically manifested in the following aspects: 1. Although some AI-based auxiliary diagnosis systems have emerged in recent years, these systems can often only handle simple cases, and it is difficult to comprehensively analyze cases by combining multiple data, which is prone to misdiagnosis; 2. Drug treatment is one of the main means of modern medicine, but the responses of different patients to the same drug may vary greatly. Although there are currently some AI-based drug recommendation systems, these systems often ignore the individual differences of patients and cannot provide doctors with comprehensive and accurate diagnostic bases and treatment suggestions; 3. Although virtual reality (VR) and augmented reality (AR) technologies have been used in surgical training and planning, there is still much room for improvement in the realism, force feedback and seamless docking of the actual surgical environment in existing surgical simulation systems.

[0004] Therefore, it is necessary to propose an intelligent medical diagnosis engine based on an AI large model to solve the above problems. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides an intelligent medical diagnosis engine based on an AI large model, which solves the problems that the existing intelligent medical diagnosis systems are prone to misdiagnosis, cannot provide doctors with accurate diagnostic bases and treatment suggestions, and the surgical simulation systems have poor realism.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: An intelligent medical diagnosis engine based on an AI large model, the medical diagnosis engine includes: An auxiliary diagnosis module: collecting patient electronic medical record data (EMR), medical image data, real-time monitoring data and genomic data, preprocessing and feature extraction of the data, training the extracted data set using a learning model, and predicting the results using an ensemble learning method; Medical Literature Assistant: Doctors can query relevant medical literature and research results through voice or text input, and recommend treatment plans or suggest diagnostic examinations. Drug Recommendation Module: Utilize an AI large model to analyze drug recommendation data, including compound structure, biological activity, and clinical trial results. Through deep learning algorithms, predict the potential efficacy and side effects of new drugs, providing valuable reference information for drug recommendation. At the same time, combine the patient's genetic data to recommend drug treatment plans for the patient. Surgical Simulation Module: Combine medical imaging data and surgical operation knowledge, and use virtual reality technology to construct a surgical environment and human body model. Doctors can practice surgeries in the virtual environment to improve their surgical skills and the ability to handle complex surgical situations. At the same time, the system can provide real-time feedback and suggestions based on the doctor's operation performance to help doctors continuously improve their surgical techniques.

[0007] Optionally, the electronic medical record data (EMR) in the auxiliary diagnosis module includes the patient's medical history, diagnosis records, treatment processes, and laboratory test results. The medical imaging data includes X-ray films, CT scans, and magnetic resonance imaging (MRI). The real-time monitoring data includes physiological indicators such as heart rate, blood pressure, and blood sugar. The genomic data includes the patient's genetic information. The preprocessing includes removing noise data, filling in missing values, and standardizing the data format for the data. Feature extraction includes extracting some features from the original data.

[0008] Optionally, the specific unit structure of the Medical Literature Assistant includes: Database: Includes medical literature, disease databases, and patient medical record information data. The data is collected and processed through the data interface layer, providing a basis for subsequent analysis and matching. Auxiliary Model Unit: Use natural language processing technology to perform text analysis on voice descriptions, extract text keywords and sentences, and match them with existing medical knowledge bases. At the same time, combine machine learning algorithms to analyze a large-scale case database, find similarities and patterns among patients, and display similar medical literature and research results. Intelligent Recommendation Unit: The system will recommend treatment plans or suggest further diagnostic examinations for the patient based on medical guidelines and practical experience.

[0009] Optionally, the specific unit structure of the Drug Recommendation Module includes: Data acquisition unit: Responsible for collecting and organizing multi-source heterogeneous data and gene expression data related to drug recommendation, including compounds, drug targets, biological activities, clinical trial data. The compound database contains information on the chemical structures, biological activities, and pharmacological properties of known compounds. Gene expression data includes the gene expression profiles of patients, which helps in personalized medicine. Clinical trial data includes the results of historical clinical trials and patient response data; Data processing unit: Cleans, de-duplicates, and standardizes the data, and converts it into a format suitable for input to a machine learning model. Chemical informatics methods are used to convert compound molecules into feature vectors recognizable by a computer, including molecular fingerprints of compounds, gene mutation sites, SMILES strings, and fingerprint descriptors; Drug analysis unit: Applies a deep neural network (DNN) for unsupervised pre-training to learn the potential efficacy and side effects of compounds. Combining with a network pharmacology model, it predicts the interactions between drugs and targets.

[0010] Optionally, the specific operation steps of the surgical simulation module are as follows: S1: Data acquisition: Includes medical image data, surgical tools, human anatomy data, and surgical operation data. The data is collected and processed through data acquisition technologies, providing a basis for subsequent modeling and simulation; S2: Model construction: Uses deep learning algorithms to analyze and process medical images to extract feature information. At the same time, combines virtual reality technology to construct a realistic surgical environment and human model, providing an immersive surgical experience for learners. Utilizes computer graphics models and physical simulation technologies to perform 3D modeling on elements such as the operating room, surgical instruments, and patient model, and simulates various mechanical reactions and interaction behaviors during the surgical process; S3: Application operation: Applies the constructed virtual surgical scene to actual surgical training, and doctors operate through virtual reality helmets and controller devices; S4: Evaluation and feedback: The system evaluates the operator's operation ability in real time and provides feedback.

[0011] Optionally, a generative pre-training model (such as GPT) is introduced into the auxiliary diagnosis module and the drug recommendation module to capture long-range dependencies in the input sequence. By calculating the similarity between queries (Query), keys (Key), and values (Value), a weighted sum representation is generated to capture context information. Each word has its corresponding query, key, and value vectors, which are generated through linear transformation. By calculating the dot product of the query and the key and scaling and normalizing the result, the correlation between each word and other words is obtained to achieve complex natural language processing and deep text analysis capabilities. This will improve the accuracy and efficiency of the system in medical literature retrieval and personalized recommendation generation.

[0012] Optionally, a generative pre-trained model is introduced into the auxiliary diagnosis module and the drug recommendation module to capture long-range dependencies in the input sequence. By calculating the similarity between queries (Query), keys (Key), and values (Value), a weighted sum representation is generated to capture context information. Each word has its corresponding query, key, and value vectors, which are generated through linear transformation. By calculating the dot product of the query and the key and scaling and normalizing the result, the correlation between each word and other words is obtained to achieve complex natural language processing and deep text analysis capabilities.

[0013] Optionally, self-supervised learning and transfer learning methods are applied to the processing of medical images and multimodal data in the intelligent medical diagnosis engine. The specific framework of self-supervised learning: Data preprocessing: Operations such as cleaning and normalizing the original data are performed to make it suitable for model processing; Feature learning: Select a self-supervised learning strategy and train a self-supervised learning model; The specific framework of transfer learning: Parameter transfer: The parameters of the already trained model are directly applied to the new task for fine-tuning; Knowledge transfer: The knowledge of the already trained model is applied to the new task; Structure transfer: The structure of the already trained model is applied to the new task; Model transfer: The learned features are applied to the model of the new task and fine-tuned.

[0014] Optionally, a knowledge graph (KG) is introduced into the auxiliary diagnosis module and the drug recommendation module. The specific framework and mathematical formula of the knowledge graph: Data collection and preprocessing: Data is collected from multiple data sources, and preprocessing operations such as cleaning, deduplication, and standardization are performed on the data; Entity recognition and relationship extraction: Natural language processing (NLP) techniques are used to identify entities in the text and extract the relationships between entities to form triples; Knowledge graph construction: The extracted entities and relationships are stored in a graph database to construct a knowledge graph. The knowledge graph is usually represented as a directed graph, where nodes represent entities and edges represent the relationships between entities; Inference and application: Use graph neural network (GNN) algorithms to perform inference on the knowledge graph. Inference can be used to discover new entity relationships, predict drug interactions, and assist in diagnosis.

[0015] Optionally, in the surgical simulation module and the auxiliary diagnosis module, the fusion and cross-modal learning of multimodal data are introduced. The specific framework is as follows: Data preprocessing layer: Image data processing: Standardize medical images; Text data processing: Perform natural language processing operations such as cleaning, word segmentation, and stop word removal on electronic medical records and diagnostic report text data; Gene data processing: Conduct quality control, variant detection, and functional annotation analysis on gene sequence data to extract genetic information related to diseases; Feature extraction and representation layer: Use deep learning models to extract features from data of different modalities respectively. These features are the visual features of images, the semantic features of text, and the biological features of gene data. Further process the features through feature transformation or dimensionality reduction techniques to reduce dimensions and retain key information; Cross-modal fusion layer: Adopt early fusion, late fusion, or hybrid fusion strategies to fuse features of different modalities; Decision and reasoning layer: Based on the fused feature vectors, use machine learning or deep learning models for decision-making and reasoning, and output diagnostic results, surgical planning suggestions, or other relevant information.

[0016] Optionally, an interpretability model (SHAP or LIME tool) is added to the medical diagnosis engine to explain the prediction results of the model through the SHAP or LIME tool. The specific architecture is as follows: Data preparation layer: Collect and organize the dataset to be explained, including input features and target variables, and preprocess the data; Model training layer: Use the selected machine learning algorithm to train the model on the training set, evaluate the performance of the model, and ensure its good generalization ability on the test set; Interpreter construction layer: According to the selected interpretability tool (SHAP or LIME), construct the corresponding interpreter object. For LIME, it is necessary to specify a subset of the training data as the local neighborhood and train a simplified model within this neighborhood. For SHAP, it is necessary to select a suitable interpreter according to the model type; Explanation generation layer: Use the interpreter to explain specific instances or the entire dataset, generate the contribution degree or importance score of each feature, and visualize the explanation results; Result analysis layer: Analyze the explanation results to understand the decision-making process of the model and the importance of features.

[0017] The present invention provides an intelligent medical diagnosis engine based on an AI large model, which has the following beneficial effects: 1. Through intelligent assisted diagnosis, the present invention can quickly process and analyze a large amount of medical data, shorten the diagnosis time. During the diagnosis process, doctors can rely on the real-time reasoning and analysis results of the system, improve work efficiency, reduce patient waiting time, optimize the allocation of medical resources. By reducing unnecessary invasive examinations and misdiagnoses, the accurate diagnosis suggestions provided by the system can reduce unnecessary medical interventions, thus avoiding waste of medical resources and improving the overall quality of medical services.

[0018] 2. The drug recommendation of the present invention can integrate the patient's gene, physiology, and pathology multi-omics data, and provide personalized drug treatment plans for patients. For example, by analyzing the patient's gene characteristics and clinical data, the drug recommendation can recommend the most suitable drugs and dosages for the patient, improve the treatment effect. The drug recommendation technology can provide comprehensive and accurate diagnostic basis and treatment suggestions for doctors.

[0019] 3. The present invention accurately simulates real surgical scenarios, including mechanical and optical properties, providing doctors and medical students with a more realistic training experience. The simulation technology can significantly improve the mastery of surgical skills. The highly realistic virtual surgical environment helps doctors familiarize themselves with the use of various surgical tools and master the surgical process. Doctors can better prepare to handle complex surgical situations, thereby reducing the risks in actual surgeries. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the operation process of the intelligent medical diagnosis engine of the present invention; Figure 2 It is a schematic diagram of the operation process of the surgical simulation module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. Embodiment 1

[0022] Please refer to FIG. 1. The intelligent medical diagnosis engine based on the AI large model, and the medical diagnosis engine includes: Auxiliary Diagnosis Module: Collects patients' electronic medical record data (EMR), medical imaging data, real-time monitoring data, and genomic data, preprocesses and extracts features from the data, trains the extracted dataset using a learning model, and predicts results using an ensemble learning method. The electronic medical record data (EMR) includes the patient's medical history, diagnosis records, treatment process, and laboratory test results. The medical imaging data includes X-rays, CT scans, and magnetic resonance imaging (MRI). The real-time monitoring data includes physiological indicators such as heart rate, blood pressure, and blood sugar. The genomic data includes the patient's genetic information. The preprocessing includes removing noisy data, filling in missing values, and standardizing the data format. The feature extraction includes extracting some features from the original data, such as extracting the features of the lesion area from medical images; The learning model is a logistic regression model. Logistic regression is a classification algorithm used for binary classification problems, and its mathematical formula is: where Y is the target variable (0 or 1), X is the feature variable, and β is the model parameter; The ensemble learning method is random forest, which makes the final prediction by constructing multiple decision trees and taking their average or majority vote. Its mathematical formula is:

[0023] where is the final prediction result, N is the number of decision trees, and T i is the prediction result of the i-th decision tree; Medical Literature Assistant: Doctors can query relevant medical literature and research results through voice or text input, and recommend treatment plans or suggest diagnostic examinations; Drug Recommendation Module: Uses an AI large model to analyze drug recommendation data, including compound structure, biological activity, and clinical trial results. Through deep learning algorithms, it predicts the potential efficacy and side effects of new drugs, provides valuable reference information for drug recommendation. At the same time, combined with the patient's genetic data, it recommends drug treatment plans for patients; Surgical Simulation Module: Combines medical imaging data and surgical operation knowledge, and constructs a surgical environment and human model using virtual reality technology. Doctors can practice surgeries in the virtual environment to improve their surgical skills and the ability to handle complex surgical situations. At the same time, the system can provide real-time feedback and suggestions based on the doctor's operation performance to help doctors continuously improve their surgical techniques.

[0024] Suppose there is an intelligent assisted diagnosis system based on convolutional neural network (CNN) for identifying tumors in medical images. First, we need to collect a large amount of labeled medical image data as the training set. Then, use this data to train a CNN model. In actual applications, when doctors upload new medical images, the system will use the trained CNN model for inference, identify the tumor location and size in the images, and display the results to doctors in the form of a heat map. Doctors can make more accurate diagnostic decisions based on this information.

[0025] In this embodiment, through intelligent assisted diagnosis, using deep learning and big data analysis technologies, it is possible to identify difficult-to-detect disease patterns and characteristics. By learning a large amount of medical data, the system can provide more accurate diagnostic suggestions, significantly improving the accuracy of disease diagnosis; It can quickly process and analyze a large amount of medical data, shortening the diagnostic time. Doctors can rely on the real-time inference and analysis results of the system during the diagnosis process, improving work efficiency, reducing patient waiting time, and optimizing the allocation of medical resources; By reducing unnecessary invasive examinations and misdiagnoses, intelligent assisted diagnosis helps reduce medical costs. The accurate diagnostic suggestions provided by the system can reduce unnecessary medical interventions, thus avoiding waste of medical resources and improving the overall quality of medical services. Embodiment 2

[0026] This embodiment is a further optimization based on Embodiment 1. Specifically, the specific unit architecture of the medical literature assistant includes: Database: It includes medical literature, disease databases, and patient medical record information data. The data is collected and processed through the data interface layer, providing a basis for subsequent analysis and matching; Auxiliary model unit: Using natural language processing technology to analyze the text of voice descriptions, extract text keywords and sentences, and match them with the existing medical knowledge base. At the same time, combined with machine learning algorithms, analyze the large-scale case database to find similarities and patterns among patients, and display similar medical literature and research results; Intelligent recommendation unit: The system will recommend treatment plans or suggest further diagnosis and examinations for patients according to medical guidelines and practical experience; The natural language processing technology is the word embedding (WordEmbedding) technology, which is used to convert words into vector representations, specifically Word2Vec. The mathematical representation of the optimization objective during its training process is:

[0027] Among them, T is the total number of words in the corpus, c is the size of the context window, w t is the target word, wt+j is a context word, P （ w t+j∣ w t） is the probability of the context word appearing given the target word; The machine learning algorithm is the support vector machine (SVM) classification algorithm. For linearly separable data, the goal of SVM is to find an optimal hyperplane that maximizes the distance from samples of different classes to this hyperplane. Its optimization problem is mathematically represented as: , , where w is the weight vector, b is the bias term, and y i is the label of the i-th sample, and x i is the feature vector of the i-th sample (i = 1, 2,..., n).

[0028] In this embodiment, by quickly querying relevant medical literature and information, doctors can quickly obtain the latest research results and clinical guidelines, thereby improving the accuracy and efficiency of diagnosis. The system can recommend the best treatment plan according to medical guidelines and practical experience or suggest that doctors conduct further diagnosis and examinations, which helps to formulate a more personalized and precise treatment plan. By analyzing a large-scale case database, the system can find the similarities and patterns among patients and assist doctors in making diagnosis and treatment decisions, thereby reducing medical risks; The intelligent medical literature assistant can be personalized according to the specific situation of the patient, enabling doctors to formulate surgical plans for specific patients in a simulated environment, which helps to achieve more precise and personalized medical services. Embodiment 3

[0029] This embodiment is a further optimization based on Embodiment 1. Specifically, the specific unit architecture of the drug recommendation module includes: Data acquisition unit: responsible for collecting and organizing multi-source heterogeneous data and gene expression data related to drug recommendation, including compounds, drug targets, biological activities, clinical trial data. The compound database contains information on the chemical structure, biological activity, and pharmacological properties of known compounds. The gene expression data includes the gene expression profiles of patients, which helps in personalized medicine. The clinical trial data includes the results of historical clinical trials and patient response data; Data processing unit: performs cleaning, deduplication, and standardization on the data, converts it into the format required for input to the machine learning model, and uses cheminformatics methods to convert compound molecules into feature vectors recognizable by a computer, including molecular fingerprints of compounds, gene mutation sites, SMILES strings, and fingerprint descriptors; Drug analysis unit: Apply deep neural network (DNN) for unsupervised pre-training to learn the potential efficacy and side effects of compounds, and combine with network pharmacology models to predict the interactions between drugs and targets; The network pharmacology model is the PageRank algorithm. The network pharmacology model combines graph theory and network analysis techniques to predict the interactions between drugs and targets. Its basic principle is to construct a drug-target interaction network and analyze the importance of nodes and edges in the network. Its mathematical formula is: Among them, PR(p i ) is the PageRank value of node p i , M(p i ) is the set of nodes pointing to p i , L(p j ) is the out-degree of node p j , d is the damping coefficient, and N is the total number of nodes; Suppose there is an intelligent drug recommendation assistant based on DNN for predicting the potential efficacy and side effects of new drugs. First, a large amount of compound and clinical trial data need to be collected as the training set. Then, these data are used to train a DNN model. In actual applications, when researchers input a new compound structure, the system will use the trained DNN model for inference, predict the potential efficacy and possible side effects of the compound, and display the results to researchers in the form of a report. Researchers can make more scientific and reasonable R & D decisions based on this information.

[0030] In this embodiment, drug recommendation can integrate the patient's gene, physiological, and pathological omics data to provide personalized drug treatment plans for patients. For example, by analyzing the patient's gene characteristics and clinical data, drug recommendation can recommend the most suitable drugs and dosages for patients, improving the treatment effect; Drug recommendation technology can provide comprehensive and accurate diagnostic basis and treatment suggestions for doctors, assisting doctors to make more scientific decisions. For example, by analyzing the patient's medical records and medical imaging data, drug recommendation can assist doctors in formulating personalized treatment plans. By accurately predicting the efficacy and side effects of drugs, drug recommendation can help doctors avoid prescribing unnecessary drugs and reduce the burden on patients; Drug recommendation technology can help researchers discover new drug targets, providing new directions for drug R & D. Drug recommendation can simulate the interactions between drugs and organisms, discover potential drug combinations, and provide a basis for combination drug use. For the R & D of drugs for rare diseases, due to the small sample size of patients, it is difficult to conduct large-scale research using traditional methods. Drug recommendation technology can accelerate the R & D process of drugs for rare diseases through simulation and prediction. Example 4

[0031] Referring to FIG. 2, this embodiment is a further optimization based on Embodiment 1. Specifically, the specific operation steps of the surgical simulation module are as follows: S1: Data acquisition: including medical image data, surgical tools, human anatomy data, and surgical operation data. The data is collected and processed through data acquisition technology to provide a basis for subsequent modeling and simulation; S2: Model construction: Use deep learning algorithms to analyze and process medical images to extract feature information. At the same time, combine virtual reality technology to construct a realistic surgical environment and human model to provide learners with an immersive surgical experience. Use computer graphics models and physical simulation technology to perform three-dimensional modeling on elements such as the operating room, surgical instruments, and patient model, and simulate various mechanical reactions and interaction behaviors during the surgical process; S3: Application operation: Apply the constructed virtual surgical scene to actual surgical training, and doctors operate through virtual reality helmets and controller devices; S4: Evaluation and feedback: The system evaluates the operator's operation ability in real time and provides feedback; In virtual surgical simulation, the deep learning algorithm is the core part. Taking the three-dimensional convolutional neural network as an example, its forward propagation formula can be expressed as: where, a （l） is the activation value of the l-th layer of neurons, a （l−1） is the activation value of the (l - 1)-th layer of neurons, W （l） is the weight matrix of the l-th layer, b （l） is the bias vector of the l-th layer, and g is the activation function; The computer graphics model is the lighting model, specifically the Phong Illumination Model, and its formula is: where, I is the final lighting intensity, Ia is the ambient light intensity, Ka is the ambient light reflection coefficient, Id is the diffuse reflection light intensity, Kd is the diffuse reflection coefficient, N is the normal vector, L is the light source direction vector, Is is the specular reflection light intensity, Ks is the specular reflection coefficient, R is the reflection vector, V is the viewing direction vector, and n is the specular coefficient; The physical simulation technology uses a physical engine for physical simulation. The mechanical equation in the physical simulation is expressed as: F = ma, where F is the force, m is the mass, and a is the acceleration, to calculate the motion state of the object under force, thereby simulating various physical phenomena during the surgical process.

[0032] In this embodiment, by precisely simulating the real surgical scenario, including mechanical and optical properties, a more realistic training experience is provided for doctors and medical students. The simulation technology can significantly improve the mastery of surgical skills. In a highly realistic virtual surgical environment, doctors can perform a large number of surgical operation exercises in the simulation, thereby improving their surgical skills, helping doctors become familiar with the use of various surgical tools, master the surgical process, and accumulate valuable experience before actually operating on patients; virtual surgical simulation allows doctors to try new surgical techniques and methods without risk. By repeatedly practicing in the simulation environment, doctors can better prepare to handle complex surgical situations, thereby reducing the risks in actual surgeries; Through virtual surgical simulation, medical institutions can provide more surgical training opportunities for doctors without increasing the actual surgical burden, which helps optimize the allocation of medical resources and improve the efficiency of medical services. Example 5

[0033] This embodiment is a further optimization based on Embodiment 1. Specifically, a generative pre-trained model (such as GPT) is introduced into the auxiliary diagnosis module and the drug recommendation module to capture long-range dependencies in the input sequence. The architecture of GPT is as follows: Transformer architecture: GPT is based on the Transformer architecture, which is a deep learning model structure with powerful natural language processing capabilities. The core of Transformer is the self-attention mechanism, which can capture long-range dependencies in the input sequence; Self-attention mechanism: The self-attention mechanism generates a weighted sum representation by calculating the similarity between queries (Query), keys (Key), and values (Value) to capture context information. Specifically, each word has its corresponding query, key, and value vectors, which are generated through linear transformation. By calculating the dot product of the query and the key and scaling and normalizing the result, the correlation (i.e., attention weight) between each word and other words is obtained; Pre-training and fine-tuning: The training of the GPT model is divided into two stages: pre-training and fine-tuning. In the pre-training stage, language model training is carried out on large-scale text data, and the goal is to maximize the conditional probability P(wt∣w1, w2, ..., wt−1); in the fine-tuning stage, supervised learning is performed on the pre-trained model using task-specific data to improve the performance of the model on this task.

[0034] In this embodiment, by introducing a generative pre-trained model, more complex natural language processing and in-depth text analysis capabilities are achieved, which will improve the accuracy and efficiency of the system in medical literature retrieval and personalized recommendation generation. The generative pre-trained model can quickly generate high-quality diagnostic reports or recommendations, thus greatly shortening the time for doctors to write medical records or diagnostic reports, which is of great significance for improving the overall work efficiency of the hospital and patient satisfaction. By learning a large amount of medical literature and medical record data, the generative pre-trained model can capture complex medical knowledge and patterns, thereby providing doctors with more accurate diagnostic suggestions, which helps reduce the risks of misdiagnosis and missed diagnosis and improve the quality of medical services. Embodiment 6

[0035] This embodiment is a further optimization based on Embodiment 1. Specifically, self-supervised learning and transfer learning methods are applied to the processing of medical images and multi-modal data in the intelligent medical diagnosis engine, reducing the dependence on labeled data, while enhancing the adaptability of the model in new fields and improving the data utilization efficiency. Self-supervised learning means that during the training process, the model self-learns features and patterns from unlabeled data for training. It utilizes the structure and relationships of the data itself and converts unlabeled data into labeled data through predefined functions to achieve the training of the model. Specific framework of self-supervised learning: Data preprocessing: Perform operations such as cleaning and normalizing the original data to make it suitable for model processing. Feature learning: Select self-supervised learning strategies (such as Generative Adversarial Networks GANs, Variational Autoencoders VAEs, self-supervised pre-training), and train the self-supervised learning model. Transfer learning means applying the knowledge on a trained model to different but related fields or problems to improve the performance of new tasks. Specific framework of transfer learning: Parameter transfer: Directly apply the parameters of the trained model to the new task for fine-tuning. Knowledge transfer: Apply the knowledge (such as features, rules) of the trained model to the new task. Structure transfer: Apply the structure (such as the number of layers, connection methods) of the trained model to the new task. Model transfer: Apply the learned features to the model of the new task and perform fine-tuning.

[0036] In this embodiment, the annotation cost of medical images and multi-modal data is high and time-consuming because professional medical knowledge is required to ensure the accuracy of annotation. Self-supervised learning performs pre-training by constructing a series of self-supervised tasks, enabling the model to learn features and patterns from unlabeled data on its own, thereby reducing the dependence on labeled data. Self-supervised learning allows the model to be pre-trained on a large amount of unlabeled data, extracting more useful features that can be transferred and applied to different medical tasks, improving the generalization ability of the model on new tasks; Self-supervised learning and transfer learning methods can help the model capture the latest medical research results and clinical practice guidelines, providing timely and effective knowledge support for doctors, which helps to promote the inheritance and update of medical knowledge and improve the overall level of the medical industry. By automatically processing a large amount of medical images and multi-modal data, self-supervised learning and transfer learning methods can reduce the workload of doctors and lower the labor cost. Embodiment 7

[0037] This embodiment is a further optimization based on Embodiment 1. Specifically, in the medical assistant diagnosis module and the drug recommendation module, a knowledge graph (KG) is introduced. By enhancing the reasoning ability, the knowledge graph can significantly improve the intelligence level of the system. The knowledge graph can help the system perform associative reasoning when analyzing patient data and relevant medical knowledge, providing more in-depth decision support. The specific architecture and mathematical formula of the knowledge graph are as follows: Data collection and preprocessing: Collect data from multiple data sources (such as medical literature, medical records, drug databases), and perform preprocessing operations such as cleaning, deduplication, and standardization on the data; Entity recognition and relationship extraction: Use natural language processing (NLP) techniques to identify entities (such as diseases, symptoms, drugs, genes) in the text, extract the relationships between entities, and form triples (entity-relationship-entity); Knowledge graph construction: Store the extracted entities and relationships in a graph database to construct a knowledge graph. The knowledge graph is usually represented as a directed graph, where nodes represent entities and edges represent the relationships between entities; Reasoning and application: Use graph neural network (GNN) algorithms to perform reasoning on the knowledge graph. Reasoning can be used to discover new entity relationships, predict drug interactions, and assist in diagnosis; In the knowledge graph, the reasoning process often involves graph embedding learning, that is, converting the nodes and edges in the graph into low-dimensional vectors for machine learning algorithms to process. Graph embedding learning methods and their mathematical formulas: TransE Model: The TransE model is one of the classical methods for knowledge graph embedding. It assumes that the difference between the head entity vector and the tail entity vector is equal to the relation vector, i.e., h + r ≈ t, where h, r, and t represent the vector representations of the head entity, relation, and tail entity respectively; ComplEx Model: The ComplEx model is a complex version of the TransE model. It considers more complex relation patterns, h1 + r - h2 ≈ t, where h1 and h2 represent two different head entity vectors.

[0038] In this embodiment, the knowledge graph integrates a large amount of medical knowledge and data to provide doctors with fast and accurate information retrieval and decision support, thereby shortening the diagnosis time. In image diagnosis, the knowledge graph can assist doctors in identifying abnormal patterns in images, improving the accuracy and efficiency of diagnosis. The knowledge graph contains rich medical entities and relationships, which can help doctors capture complex medical knowledge and patterns, reducing the risk of misdiagnosis and missed diagnosis; By reasoning and analyzing the information in the knowledge graph, doctors can obtain a more comprehensive assessment of the patient's condition, thereby making more accurate diagnostic decisions. The knowledge graph can generate personalized treatment plans or suggestions based on information such as the patient's medical history and examination results, helping doctors to understand the patient's condition more comprehensively and thus making more reasonable treatment decisions. In drug recommendation, the knowledge graph can combine the patient's specific condition and drug characteristics to recommend the most suitable drug treatment plan, improving the treatment effect. Example 8

[0039] This embodiment is a further optimization based on Example 1. Specifically, in the surgical simulation module and the auxiliary diagnosis module, the fusion and cross-modal learning of multi-modal data (such as medical images, text, gene data) are introduced. The specific architecture and mathematical formulas are as follows: Data Preprocessing Layer: Image Data Processing: Standardize medical images, including normalizing pixel values, adjusting contrast and brightness. At the same time, image segmentation and feature extraction steps are involved; Text Data Processing: Perform natural language processing operations such as cleaning, word segmentation, and stop word removal on electronic medical records and diagnostic report text data, and extract key features; Gene Data Processing: Perform quality control, variant detection, and functional annotation analysis on gene sequence data to extract genetic information related to diseases; Feature Extraction and Representation Layer: Extract features from data of different modalities using deep learning models (such as convolutional neural network CNN for images, recurrent neural network RNN or Transformer for texts). These features are the visual features of images, the semantic features of texts, and the biological features of gene data. Further process the features through feature transformation or dimensionality reduction techniques (such as principal component analysis PCA, autoencoder AE) to reduce the dimensions and retain key information; Cross-modal fusion layer: Adopt early fusion, late fusion or hybrid fusion strategies to fuse features of different modalities. Early fusion: Immediately fuse after feature extraction to form a unified feature vector. For example, fuse at the feature layer level. First extract features from each modality and then fuse them. Late fusion: After separately extracting and modeling features for each modality, fuse their respective results. For example, process different modalities separately and then fuse their respective results. Hybrid fusion: Combine the advantages of early fusion and late fusion and perform fusion at different levels. For example, first perform feature extraction based on a single modality and then fuse at a higher level. Use specific fusion algorithms (such as attention mechanism, bidirectional multimodal attention mechanism) to discover and encode the correlations between different modalities; Decision and inference layer: Based on the fused feature vector, use machine learning or deep learning models (such as support vector machine SVM, random forest RF, neural network NN) for decision-making and inference, and output diagnostic results, surgical planning suggestions or other relevant information; The following are some application examples of common mathematical concepts and formulas in cross-modal learning: Feature extraction: For image data, use convolutional neural network (CNN) for feature extraction, and its mathematical expressions usually involve convolution operations and pooling operations; for text data, word embedding (such as Word2Vec, BERT) can be used to convert text into vector representation; Feature fusion: In early fusion, directly concatenate or perform weighted summation on feature vectors of different modalities. In late fusion, use ensemble learning methods (such as weighted average, voting) to combine the outputs of different models.

[0040] In this embodiment, multimodal data fusion can comprehensively consider various information of patients, such as medical images, electronic medical records, gene data, etc., so as to provide more comprehensive and accurate diagnostic results. For example, by integrating CT image and electronic medical record data, pulmonary embolism can be more accurately identified and the common causes of acute respiratory failure can be distinguished; in surgical simulation and assisted diagnosis, doctors need to make decisions based on a large amount of complex information, and multimodal data fusion can provide richer context information to help doctors make more reasonable and personalized treatment decisions. Example 9

[0041] This example is a further optimization based on Example 1. Specifically, a reinforcement learning algorithm is applied in the drug recommendation module and the surgical simulation module. The framework and mathematical formula of the reinforcement learning algorithm are as follows: Environment definition: In the drug recommendation module, the environment includes the patient's electronic health record (EHR), current symptoms, and drug inventory. In the surgical simulation module, the environment involves the patient's specific physiological state, the state of surgical equipment, and the environmental conditions of the operating room; Agent: The agent is the core of the reinforcement learning algorithm. It makes decisions based on the observed environmental state and executes corresponding actions. In the drug recommendation module, the agent is a drug recommendation system that recommends appropriate drugs and dosages based on the patient's condition and historical medication data. In the surgical simulation module, the agent is a virtual surgeon or surgical planning system that formulates surgical plans based on the surgical environment and patient status; Action space: The action space is the set of all possible actions that the agent can execute. In the drug recommendation module, actions include selecting specific drugs, adjusting drug dosages, adding or deleting drugs. In the surgical simulation module, actions include performing specific surgical steps, adjusting the parameters of surgical equipment, and changing surgical strategies; Reward function: The reward function is used to evaluate the quality of the actions taken by the agent and give the agent corresponding rewards or punishments according to the evaluation results. In the drug recommendation module, the reward function is designed based on factors such as the efficacy, side effects, and cost of drugs. In the surgical simulation module, the reward function is designed based on factors such as surgical success rate, postoperative recovery, and surgical time; Learning process: The agent learns how to make better decisions by interacting with the environment. It updates its internal policy according to the observed environmental state and the reward function so as to make better decisions when encountering similar situations in the future.

[0042] In this example, the reinforcement learning algorithm can formulate personalized treatment strategies according to the patient's individual characteristics and condition. By learning historical case and clinical trial data, the algorithm can predict the effects of different treatment plans and guide doctors to select the best treatment plan; The reinforcement learning algorithm can optimize surgical steps and improve the surgical success rate. By simulating the surgical process, the algorithm can learn and improve surgical strategies, reduce the surgical time and the occurrence of complications; Through accurate drug recommendation, unnecessary drug use can be reduced, and the cost of drug treatment can be lowered. At the same time, optimizing the treatment plan can reduce the hospital stay and medical expenses. Optimizing the surgical process and reducing the surgical time can reduce the surgical cost and improve the utilization rate of the operating room; Personalized drug recommendations can improve treatment efficacy and reduce side effects. By learning from a large amount of medical record data, algorithms can provide more accurate drug combination and dosage suggestions. Through simulating and optimizing surgical procedures, reinforcement learning algorithms can help doctors select the best surgical plan, improving the success rate of surgery and the postoperative recovery effect. Example 10

[0043] This example is a further optimization based on Example 1. Specifically, an interpretability model (SHAP or LIME tool) is added to the medical diagnosis engine to explain the prediction results of the model. The specific architecture and mathematical formulas are as follows: Data preparation layer: Collect and organize the dataset to be explained, including input features and target variables, and preprocess the data, such as cleaning, standardizing, and normalizing. Model training layer: Use the selected machine learning algorithms (such as random forest, neural network) to train the model on the training set, evaluate the performance of the model, and ensure its good generalization ability on the test set. Interpreter construction layer: According to the selected interpretability tool (SHAP or LIME), construct the corresponding interpreter object. For LIME, a subset of the training data needs to be specified as the local neighborhood, and a simplified model is trained within this neighborhood. For SHAP, the appropriate interpreter needs to be selected according to the model type, such as TreeExplainer for tree models. Explanation generation layer: Use the interpreter to explain specific instances or the entire dataset, generate the contribution degree or importance score of each feature, and visualize the explanation results, such as feature importance charts, SHAP value charts. Result analysis layer: Analyze the explanation results to understand the decision-making process of the model and the importance of features.

[0044] In this example, by explaining the prediction results of the model, doctors and patients can better understand how the model works, thereby increasing trust in the model. Transparency helps ensure the fairness and rationality of medical decisions, especially when dealing with sensitive or critical decisions; Tools such as LIME and SHAP can reveal the contribution degree of each feature to the model prediction, helping doctors understand which factors have important impacts on the diagnosis results; By explaining the prediction results of the model, it can help doctors better understand the complexity and diversity of diseases, thereby promoting the inheritance and update of medical knowledge. Doctors can formulate more personalized treatment plans based on the explanation results of the model, combined with their professional knowledge and experience.

[0045] The present invention also discloses an online service device based on an AI large model, including a processor, an input device, an output device, and a memory. The processor, the input device, the output device, and the memory are interconnected. Among them, the memory is used to store a computer program, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the method of any of the above embodiments.

[0046] The present invention also discloses a computer-readable storage medium. The computer storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method of any of the above embodiments.

[0047] In summary: Through intelligent assisted diagnosis, the present invention can quickly process and analyze a large amount of medical data, shorten the diagnosis time. Doctors can rely on the real-time reasoning and analysis results of the system during the diagnosis process, improve work efficiency, reduce the waiting time of patients, optimize the allocation of medical resources. By reducing unnecessary invasive examinations and misdiagnoses, the accurate diagnosis suggestions provided by the system can reduce unnecessary medical interventions, thereby avoiding waste of medical resources and improving the overall quality of medical services; Drug recommendation can integrate multi-omics data such as the genes, physiology, and pathology of patients, and provide personalized drug treatment plans for patients. For example, by analyzing the gene characteristics and clinical data of patients, drug recommendation can recommend the most suitable drugs and dosages for patients, improve the treatment effect. Drug recommendation technology can provide comprehensive and accurate diagnostic bases and treatment suggestions for doctors, and assist doctors in making more scientific decisions; By precisely simulating real surgical scenarios, including mechanical and optical characteristics, and providing doctors and medical students with a more realistic training experience, simulation technology can significantly improve the mastery of surgical skills. In a highly realistic virtual surgical environment, doctors can perform a large number of surgical operation exercises in the simulation, thereby improving surgical skills, helping doctors become familiar with the use of various surgical tools and master the surgical process. Doctors can better prepare to handle complex surgical situations, thereby reducing the risks in actual surgeries.

[0048] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An intelligent medical diagnosis engine based on AI big model, characterized in that: The medical diagnosis engine comprises: Auxiliary diagnosis module: collects patient electronic medical record data, medical imaging data, real-time monitoring data and genomic data, and pre-processes and extracts features from the data. It uses a learning model to train the extracted data set and uses an integrated learning method to predict the results. Medical literature assistant: Doctors can search for relevant medical literature and research results through voice or text input, and recommend treatment plans or diagnostic examinations; Drug recommendation module: Use AI big models to analyze drug recommendation data, including compound structure, biological activity, and clinical trial results. Use deep learning algorithms to predict the potential efficacy and side effects of new drugs. At the same time, combine the patient's genetic data to recommend drug treatment plans for patients. Surgery simulation module: Combining medical imaging data and surgical operation knowledge with virtual reality technology to build a surgical environment and human body model, doctors practice surgery in a virtual environment. At the same time, the system can provide real-time feedback and suggestions based on the doctor's operating performance.

2. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The electronic medical record data in the auxiliary diagnosis module includes the patient's medical history, diagnosis records, treatment process and laboratory test results; The medical imaging data include X-rays, CT scans, and MRIs, and the real-time monitoring data include heart rate, blood pressure, and blood sugar physiological indicators; The genomic data includes genetic information of the patient; The preprocessing includes removing noise data, filling missing values ​​and standardizing data formats, and feature extraction includes extracting some features from the original data.

3. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The specific unit structure of the medical literature assistant includes: Database: including medical literature, disease database and patient medical record information data. The data is collected and processed through the data interface layer to provide a basis for subsequent analysis and matching; Auxiliary model unit: Use natural language processing technology to perform text analysis on speech description, extract text keywords and sentences, and match them with the existing medical knowledge base. At the same time, combine machine learning algorithms to analyze large-scale case databases, find similarities and patterns between patients, and display similar medical literature and research results; Intelligent recommendation unit: The system will recommend treatment plans or suggest further diagnosis and examination for patients based on medical guidelines and practical experience.

4. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The specific unit structure of the drug recommendation module includes: Data acquisition unit: responsible for collecting and organizing multi-source heterogeneous data and gene expression data related to drug recommendations, including compounds, drug targets, biological activities, and clinical trial data. The compound database contains information on the chemical structure, biological activity, and pharmacological properties of known compounds. Gene expression data includes the gene expression profiles of patients. Clinical trial data includes the results of historical clinical trials and patient response data. Data processing unit: cleans, removes duplicates, and standardizes the data, converts it into a format suitable for machine learning model input, and uses chemical informatics methods to convert compound molecules into computer-recognizable feature vectors, including the compound's molecular fingerprint, gene mutation site, SMILES string, and fingerprint descriptor; Drug Analysis Unit: Apply deep neural networks for unsupervised pre-training to learn the potential efficacy and side effects of compounds, and combine network pharmacology models to predict the interaction between drugs and targets.

5. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The specific operation steps of the surgical simulation module are as follows: S1: Data acquisition: including medical image data, surgical tools, human anatomy data and surgical operation data. The data is collected and processed through data acquisition technology to provide a basis for subsequent modeling and simulation; S2: Model construction: Use deep learning algorithms to analyze and process medical images to extract feature information. At the same time, combine virtual reality technology to build a realistic surgical environment and human model to provide learners with an immersive surgical experience. Use computer graphics models and physical simulation technology to build three-dimensional models of elements such as operating rooms, surgical instruments, and patient models, and simulate various mechanical reactions and interactive behaviors during surgery. S3: Application operation: Apply the constructed virtual surgical scene to actual surgical training, and the doctor operates through the virtual reality helmet and controller device; S4: Evaluation and feedback: The system evaluates the operator's operating ability in real time and provides feedback.

6. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: A generative pre-training model is introduced into the auxiliary diagnosis module and the drug recommendation module to capture long-distance dependencies in the input sequence. By calculating the similarity between the query, key and value, a weighted sum representation is generated to capture contextual information. Each word has its corresponding query, key and value vectors, which are generated through linear transformation. By calculating the dot product of the query and the key, and scaling and normalizing the result, the correlation between each word and other words is obtained to achieve complex natural language processing and deep text analysis capabilities.

7. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: Apply self-supervised learning and transfer learning methods to the processing of medical images and multimodal data in intelligent medical diagnosis engines. The specific framework of self-supervised learning is as follows: Data preprocessing: cleaning and normalizing the raw data to make it suitable for model processing; Feature learning: select a self-supervised learning strategy and train a self-supervised learning model; The specific framework of transfer learning: Parameter transfer: directly apply the trained model parameters to new tasks for fine-tuning; Knowledge transfer: Applying trained model knowledge to new tasks; Structural transfer: Applying the trained model structure to new tasks; Model transfer: Apply the learned features to the model for a new task and perform fine-tuning.

8. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The knowledge graph (KG) is introduced into the auxiliary diagnosis module and the drug recommendation module. The specific structure of the knowledge graph is as follows: Data collection and preprocessing: Collect data from various data sources and perform preprocessing operations such as cleaning, deduplication, and standardization on the data; Entity recognition and relationship extraction: Use natural language processing (NLP) technology to identify entities in text, extract relationships between entities, and form triples; Knowledge graph construction: Store the extracted entities and relationships in a graph database to build a knowledge graph. The knowledge graph is usually represented as a directed graph, where nodes represent entities and edges represent relationships between entities. Reasoning and application: The graph neural network (GNN) algorithm is used to perform reasoning on the knowledge graph. Reasoning can be used to discover new entity relationships, predict drug interactions, and assist in diagnosis.

9. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: In the surgical simulation module and the auxiliary diagnosis module, multimodal data fusion and cross-modal learning are introduced, and the specific architecture is as follows: Data preprocessing layer: Image data processing: standardize medical images; text data processing: clean, segment, remove stop words and perform natural language processing on electronic medical records and diagnostic report text data; gene data processing: perform quality control, variant detection, and functional annotation analysis on gene sequence data to extract disease-related genetic information; Feature extraction and representation layer: Use deep learning models to extract features from data of different modalities. These features are visual features of images, semantic features of text, and biological features of genetic data. The features are further processed through feature transformation or dimensionality reduction techniques to reduce the dimensions and retain key information. Cross-modal fusion layer: early fusion, late fusion or hybrid fusion strategies are used to fuse features of different modalities; Decision and reasoning layer: Based on the fused feature vector, use machine learning or deep learning models to make decisions and reason, and output diagnosis results, surgical planning suggestions or other relevant information.

10. The intelligent medical diagnosis engine based on AI big model according to claim 1, characterized in that: The medical diagnosis engine adds an explainable model (SHAP or LIME tool) to explain the prediction results of the model through SHAP or LIME tool. The specific structure is as follows: Data preparation layer: collects and organizes the data sets that need to be explained, including input features and target variables, and preprocesses the data; Model training layer: Use the selected machine learning algorithm to train the model on the training set, evaluate the performance of the model, and ensure that it has good generalization ability on the test set; Explainer construction layer: According to the selected interpretive tool (SHAP or LIME), the corresponding explainer object is constructed. For LIME, a subset of the training data needs to be specified as a local neighborhood and a simplified model is trained in this neighborhood. For SHAP, an appropriate explainer needs to be selected according to the model type. Explanation generation layer: Use the explainer to explain a specific instance or the entire dataset, generate the contribution or importance score of each feature, and visualize the explanation results; Result analysis layer: Analyze and interpret the results to understand the model’s decision-making process and the importance of features.

Images