Intelligent medical remote diagnosis and treatment system based on artificial intelligence

Abstract

The invention discloses an intelligent medical remote diagnosis and treatment system based on artificial intelligence, which adopts a convolutional neural network to extract ultrasonic, computed tomography and magnetic resonance imaging characteristics, adopts a long-short term memory network to extract electrocardiogram, blood pressure and blood sugar characteristics, and subsequently fuses the characteristics with an electronic medical record. And finally, the multi-modal local large model is utilized to diagnose the structured data. The specific implementation steps of the system are as follows: S1: data acquisition; s2, data preprocessing; s3, feature extraction; s4, data transmission; s5, performing diagnosis; s6, remote interaction; s7, data analysis; s8, manual verification is carried out; s9, updating the model; the defects that traditional medical image and physiological signal detection is long in consumed time and large in error are overcome, the problem that medical resources are not evenly distributed is effectively solved, and improvement of multi-mechanism cooperation and the overall diagnosis capacity is promoted.

Description

Technical Field

[0001] The present invention relates to the field of smart medical care, and in particular to a smart medical remote diagnosis and treatment system based on artificial intelligence. Background Art

[0002] In recent years, remote diagnosis and treatment has integrated advanced technologies such as blockchain, big data, the Internet of Things, and artificial intelligence, and has been widely used in various medical institutions such as hospitals, clinics, and telemedicine centers. This technology not only significantly improves the efficiency of diagnosis and treatment, optimizes resource allocation, but also greatly improves the patient's medical experience. However, remote diagnosis and treatment still faces major challenges in terms of accuracy and real-time performance, which directly affects the sustainable development of related technologies. With the development of artificial intelligence technology, the emergence of multimodal large models has provided new opportunities for remote diagnosis and treatment.

[0003] Current remote diagnosis and treatment relies on human participation, and the diagnosis of medical images and physiological data mainly relies on manual analysis. This method is time-consuming and inconsistent, and it is difficult to meet the needs of modern medical care. For example, after doctors obtain medical images such as ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI), they combine prior knowledge to determine the patient's physiological state; at the same time, they also need to interpret physiological data such as electrocardiograms, blood pressure, and blood sugar. This process not only increases the workload, but is also easily affected by subjective factors, making it difficult to ensure the accuracy and real-time nature of the diagnosis results. Therefore, it is urgent to introduce intelligent technology to optimize data processing and diagnosis processes to improve the efficiency and accuracy of remote diagnosis and treatment and meet the growing demand for medical services.

[0004] At present, there are also some patent technologies related to smart medical treatment based on artificial intelligence on the market. For example, Chinese patent CN202210487424X discloses an intelligent diagnosis and treatment system for cervical spondylosis based on a multimodal database. Through machine learning and combined with clinical evidence-based evidence, multimodal data is mapped with the diagnostic classification and degree of each type of cervical spondylosis, and a neural network model for multimodal data decision reasoning is constructed to form an intelligent diagnosis algorithm to assist doctors in improving diagnostic efficiency and accuracy, risk prediction, and disease severity assessment. However, this system has obvious drawbacks, which are as follows: 1. Limited accuracy: Traditional machine learning algorithms have poor generalization capabilities, and their accuracy cannot meet the demands of medical scenarios; 2. Poor data reliability: The lack of necessary privacy protection modules makes it impossible to achieve remote sharing of data and prevention of epidemics.

[0005] In order to improve the accuracy, efficiency and data reliability of remote diagnosis and treatment, we have developed an intelligent medical remote diagnosis and treatment system based on artificial intelligence. The system uses convolutional neural networks to extract features of ultrasound, CT and MRI, and uses long short-term memory networks to extract features of electrocardiograms, blood pressure and blood sugar, and then integrates these features with electronic medical records. Finally, a multimodal local large model is used to accurately diagnose structured data. The system effectively solves the problem of time-consuming and large errors in traditional medical imaging and physiological signal detection, significantly improves the accuracy and real-time nature of diagnosis, alleviates the problem of uneven distribution of medical resources, and promotes multi-institutional collaboration and overall diagnostic capabilities. Summary of the invention

[0006] In view of the above problems, the present invention provides an intelligent medical remote diagnosis and treatment system based on artificial intelligence, which uses a convolutional neural network to extract the features of ultrasound, computer tomography and magnetic resonance imaging, and uses a long short-term memory network to extract the features of electrocardiogram, blood pressure and blood sugar. The features are subsequently integrated with electronic medical records, and finally a multimodal local large model is used to diagnose structured data. The system is characterized in that the implementation steps of the system are:

[0007] S1: Data acquisition: Obtain medical image data collected by ultrasound equipment, computer tomography scanners and magnetic resonance imaging machines; obtain time series data collected by electrocardiographs, blood pressure monitors and blood glucose meters;

[0008] S2: Data preprocessing: Preprocess the medical image data and time series data; the preprocessing methods of the medical image data are rotation, flipping, and scaling, thereby improving the data quality and enhancing the generalization ability of the model; the preprocessing methods of the time series data are abnormal data point removal and data standardization, specifically:

[0009] Removing abnormal data points can improve the accuracy and robustness of the model, specifically:

[0010] >

[0011] in, Expressed as data points; It is expressed as a threshold value, provided by the multimodal local large model; Expressed as the mean of time series data, it can be expressed as:

[0012]

[0013] in, is the number of data points; It is expressed as the standard deviation of time series data, which measures the degree of dispersion of data. The calculation formula is:

[0014]

[0015] Data standardization can eliminate the impact of dimension, accelerate the process of model training, and improve the stability of the training process. Specifically:

[0016]

[0017] in, Expressed as standardized medical data, It is expressed as the original time series data obtained in step S1;

[0018] S3: Feature extraction: The image data obtained in step S2 is processed using a convolutional neural network, and the spatial features of the image are extracted through multi-layer convolution and pooling operations, and then fused with the electronic medical record; the time series data obtained in step S2 is processed using a long short-term memory network to capture the time-dependent features of the health status, and then fused with the electronic medical record;

[0019] S4: Data transmission: Protect the patient's electronic medical record data through identity authentication and authorization mechanisms to ensure the security of data transmission and avoid data leakage risks;

[0020] S5: Diagnosis: Analyze and diagnose the fused structured data using a multimodal local big model to obtain the patient's physiological state; the multimodal local big model is based on parameter fine-tuning technology; the parameter fine-tuning technology is described as producing a large number of data sets for the field of medical diagnosis, combining professional medical knowledge with the general capabilities of the big model, and obtaining a multimodal local big model for medical scenarios;

[0021] S6: Data analysis: Support vector machine is used to determine the status level of the epidemic; the support vector machine uses radial basis function kernel to achieve autonomous monitoring of the epidemic through hyperparameter optimization and model deployment; the radial basis function kernel It is expressed as:

[0022]

[0023] in, It is expressed as width, which represents the influence range of the kernel function; Representing data points in the input space;

[0024] S7: Real-time interaction: The system can transmit the diagnosis results back to the patient, realizing real-time two-way interaction between the user and the doctor; the system provides multidisciplinary joint consultation function for difficult cases, promoting collaborative sharing of medical resources;

[0025] S8: Manual verification: Physicians verify the diagnosis results of the multimodal local large model to improve the reliability of the diagnosis results;

[0026] S9: Model update: The system supports continuous learning and performance monitoring of the model, and regularly updates the model to incorporate the latest medical data and research results to ensure the advancement and adaptability of its diagnostic capabilities.

[0027] The advantages of the present invention are:

[0028] 1. High accuracy: The present invention uses convolutional neural networks to extract features of ultrasound, computed tomography and magnetic resonance imaging, and uses long short-term memory networks to extract features of electrocardiograms, blood pressure and blood sugar, and subsequently integrates the features with electronic medical records. This process effectively reduces manual dependence, reduces human errors, and improves diagnosis and treatment efficiency; the system is equipped with a manual verification module, which effectively improves the credibility of the diagnosis results;

[0029] 2. Strong real-time performance: The present invention uses a multimodal local large model to diagnose structured data and obtain the patient's physiological status in real time;

[0030] 3. High data security: The present invention uses encryption and privacy computing technology to ensure the security and reliability of patient data throughout its life cycle;

[0031] 4. Epidemic prediction: The system is equipped with a data analysis module that can determine the status level of an epidemic based on support vector machine analysis, thereby achieving early detection and prevention of an epidemic. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the present invention, and do not constitute a limitation of the embodiments of the present invention.

[0033] Figure 1 : An implementation architecture of a smart medical remote diagnosis and treatment system based on artificial intelligence.

[0034] Figure 2 : Implementation framework for data collection and data preprocessing.

[0035] Figure 3 : Implementation framework for feature extraction of ultrasound, CT and MRI image data.

[0036] Figure 4 : Feature extraction implementation framework for blood pressure, ECG and blood glucose time series data.

[0037] Figure 5 :Implementation method of data encryption framework.

[0038] Figure 6 : An implementation case of a multimodal local large model.

[0039] Figure 7 : Epidemic prediction-principle of support vector machine algorithm. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0041] Implementation Example 1

[0042] Attached Figure 1-2 The paper demonstrates the implementation architecture of a smart medical remote diagnosis and treatment system based on artificial intelligence. It uses convolutional neural networks to extract features of ultrasound, computed tomography and magnetic resonance imaging, and uses long short-term memory networks to extract features of electrocardiogram, blood pressure and blood sugar. The features are then integrated with electronic medical records, and finally a multimodal local large model is used to diagnose structured data. The system is characterized in that the implementation steps are as follows:

[0043] S1: Data acquisition: Obtain medical image data collected by ultrasound equipment, computer tomography scanners and magnetic resonance imaging machines; obtain time series data collected by electrocardiographs, blood pressure monitors and blood glucose meters;

[0044] S2: Data preprocessing: Preprocess the medical image data and time series data; the preprocessing methods of the medical image data are rotation, flipping, and scaling, thereby improving the data quality and enhancing the generalization ability of the model; the preprocessing methods of the time series data are abnormal data point removal and data standardization, specifically:

[0045] Removing abnormal data points can improve the accuracy and robustness of the model, specifically:

[0046] >

[0047] in, Expressed as data points; It is expressed as a threshold value, provided by the multimodal local large model; Expressed as the mean of time series data, it can be expressed as:

[0048]

[0049] in, is the number of data points; It is expressed as the standard deviation of time series data, which measures the degree of dispersion of data. The calculation formula is:

[0050]

[0051] Data standardization can eliminate the impact of dimension, accelerate the process of model training, and improve the stability of the training process. Specifically:

[0052]

[0053] in, Expressed as standardized medical data, It is expressed as the original time series data obtained in step S1;

[0054] S3: Feature extraction: The image data obtained in step S2 is processed using a convolutional neural network, and the spatial features of the image are extracted through multi-layer convolution and pooling operations, and then fused with the electronic medical record; the time series data obtained in step S2 is processed using a long short-term memory network to capture the time-dependent features of the health status, and then fused with the electronic medical record;

[0055] S4: Data transmission: Protect the patient's electronic medical record data through identity authentication and authorization mechanisms to ensure the security of data transmission and avoid data leakage risks;

[0056] S5: Diagnosis: Analyze and diagnose the fused structured data using a multimodal local big model to obtain the patient's physiological state; the multimodal local big model is based on parameter fine-tuning technology; the parameter fine-tuning technology is described as producing a large number of data sets for the field of medical diagnosis, combining professional medical knowledge with the general capabilities of the big model, and obtaining a multimodal local big model for medical scenarios;

[0057] S6: Data analysis: Support vector machine is used to determine the status level of the epidemic; the support vector machine uses radial basis function kernel to achieve autonomous monitoring of the epidemic through hyperparameter optimization and model deployment; the radial basis function kernel It is expressed as:

[0058]

[0059] in, It is expressed as width, which represents the influence range of the kernel function; Representing data points in the input space;

[0060] S7: Real-time interaction: The system can transmit the diagnosis results back to the patient, realizing real-time two-way interaction between the user and the doctor; the system provides multidisciplinary joint consultation function for difficult cases, promoting collaborative sharing of medical resources;

[0061] S8: Manual verification: Physicians verify the diagnosis results of the multimodal local large model to improve the reliability of the diagnosis results;

[0062] S9: Model update: The system supports continuous learning and performance monitoring of the model, and regularly updates the model to incorporate the latest medical data and research results to ensure the advancement and adaptability of its diagnostic capabilities.

[0063] Implementation Example 2

[0064] Attached Figure 3 The implementation framework of feature extraction of ultrasound, CT and MRI image data is presented. Taking ultrasound as an example, we demonstrate the process of extracting image features using a convolutional neural network, specifically:

[0065] S1: Data acquisition: Acquisition of ultrasound images;

[0066] S2: Data labeling: labelimg is used to label the data set, and the results are saved in YOLO format;

[0067] S3: Model training: Write the mytrain.yaml file and specify the location and type of the dataset. The specific content is as follows:

[0068] path: / home / ai / yolov8 / ultralytics / datasets / hongwai

[0069] train: images / train2017 # train images (relative to 'path') 4 images

[0070] val: images / val2017 # val images (relative to 'path') 4 images

[0071] test: # test images (optional)

[0072] names:

[0073] 0: Normal

[0074] 1: Abnormal

[0075] 2: Critical

[0076] After the data set is created, model training is performed. The specific execution command is:

[0077] yolo detect train data=datasets / mytrain.yaml model=runs / weights / best.pt batch=16 imgsz=640 workers=0 device=0 Submit the calculation task through the command line.

[0078] S4: Model deployment: During the project implementation, the trained model will be exported to ONNX format, and ONNXRuntime will be used for inference. ONNX (Open Neural Network Exchange) was jointly developed by Microsoft and Facebook and open sourced at the end of 2017. It is an open deep learning model exchange format with advantages such as framework independence and easy deployment. The specific code used is:

[0079] import onnxruntime as ort

[0080] def main(self):

[0081] session = ort.InferenceSession(self.onnx_model, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

[0082] model_inputs = session.get_inputs()

[0083] input_shape = model_inputs[0].shape

[0084] self.input_width = input_shape[2]

[0085] self.input_height = input_shape[3]

[0086] img_data = self.preprocess()

[0087] outputs = session.run(None, {model_inputs[0].name: img_data})

[0088] return self.postprocess(self.img, outputs)

[0089] Implementation Example 3

[0090] Attached Figure 4 The implementation framework of feature extraction of blood pressure, ECG and blood sugar time series data is demonstrated. The LSTM algorithm has a gating mechanism and memory unit, which can effectively capture and learn long-term dependencies in time series data. Related results are widely used in system identification, human posture prediction, pollutant concentration prediction, mechanical equipment life prediction, foundation deformation law prediction, and stock price rise and fall prediction.

[0091] The LSTM algorithm mainly consists of two parts: forget gate, input gate, output gate and memory cell. The mathematical expressions of forget gate, input gate and output gate are as follows:

[0092]

[0093]

[0094]

[0095] The update function of the cell state is:

[0096]

[0097]

[0098] The output of the system is:

[0099]

[0100] The LSTM algorithm implementation process includes two parts: model training and inference deployment. Model training uses a high-performance server. After training, the results are exported to the ONNX format and deployed to a private high-performance computing server.

[0101] Model structure: input_size is expressed as the number of channels, specifically 3, which are ECG, blood pressure and blood sugar respectively; output_size is expressed as the output parameter, specifically the features to be extracted; num_steps expresses the matrix dimension, which needs to be greater than the minimum length; max_epoch represents the parameters related to model training: Increasing max_epoch can improve accuracy, but setting it too large will cause overfitting of the model, increase training costs and bring resource consumption problems, etc. The specific model structure used in the project is:

[0102] def __init__(self, sess,

[0103] lstm_size=128,

[0104] num_layers = 1,

[0105] num_steps = 10,

[0106] input_size = 21,

[0107] output_size = 1,

[0108] logs_dir = "logs",

[0109] plots_dir = "figures",

[0110] max_epoch = 5):

[0111] self.sess = sess

[0112] self.lstm_size = lstm_size

[0113] self.num_layers = num_layers

[0114] self.num_steps = num_steps

[0115] self.input_size = input_size

[0116] self.output_size = output_size

[0117] self.logs_dir = logs_dir

[0118] self.plots_dir = plots_dir

[0119] self.max_epoch = max_epoch

[0120] self.build_graph()

[0121] Example 4

[0122] Appendix Figure 5The implementation method of the data encryption framework is demonstrated. In the remote diagnosis and treatment system, the transmission of patients' electronic medical records and physiological data must ensure a high degree of security and privacy. Therefore, data transmission needs to adopt an encrypted communication protocol to protect the patient's personal health information. During the implementation of the project, the AES-256 algorithm is used to encrypt ultrasound images, CT, MRI imaging data, electrocardiogram, blood pressure, and blood sugar characteristic data to ensure the confidentiality of the data. At the same time, a secure channel is established during the data transmission process through an encrypted network protocol (TLS / SSL) to prevent the data from being attacked or tampered with during the transmission process.

[0123] The system integrates the blockchain system and automatically verifies the integrity and source of the data through smart contracts. The data integrity verification process is to calculate the hash value of the data and compare it with the hash value stored when uploading to ensure that the data has not been tampered with. The data source verification process is to verify through the digital signature or other identity authentication information of the medical institution to ensure that the data is indeed from a legitimate medical institution. The system uses identity authentication and authorization mechanisms to ensure that only authorized medical personnel or systems can access or transmit data, thereby ensuring data security.

[0124] In addition, the blockchain system ensures that each data upload, verification and update operation has an unalterable timestamp and audit record, providing a transparent operation history for medical institutions, patients and regulatory authorities. The system uses distributed storage technology to store data on multiple nodes, effectively improving storage efficiency and reliability, and reducing the risk of data loss caused by centralized storage.

[0125] Implementation Example 5

[0126] The multimodal local big model can analyze and diagnose the fused structured data to obtain the patient's physiological state; the multimodal local big model is based on parameter fine-tuning technology; the parameter fine-tuning technology is described as producing a large number of data sets for the field of medical diagnosis, combining professional medical knowledge with the general capabilities of the big model, and obtaining a multimodal local big model for medical scenarios;

[0127] Figure 6 The implementation case of multimodal local big model is demonstrated. In recent years, artificial intelligence technology has achieved rapid development. AI big model has gradually become one of the core technologies in various industries, and has wide application potential in natural language processing, time series data processing and medical fields. The deployment process of multimodal local big model is as follows:

[0128] S1: Dataset construction: Establish a database for remote diagnosis and treatment, including feature data of ultrasound, CT, MRI, ECG, blood pressure and blood sugar. Establish data labels through manual annotation, specifically: [

[0130] {

[0131] "timestamp": "2024-08-12T08:00:00Z",

[0132] "heart_rate": 72,

[0133] "systolic_pressure": 120,

[0134] "diastolic_pressure": 80,

[0135] "fasting_glucose": 90,

[0136] "postprandial_glucose": 110,

[0137] "ultrasound_organ_shape": "Normal",

[0138] "ct_lesion_size": "Small",

[0139] "mri_signal_intensity": "Normal",

[0140] "label": "Healthy"

[0141] },

[0142] …..

[0143] {

[0144] "timestamp": "2024-09-12T08:00:00Z",

[0145] "heart_rate": 75,

[0146] "systolic_pressure": 125,

[0147] "diastolic_pressure": 85,

[0148] "fasting_glucose": 92,

[0149] "postprandial_glucose": 115,

[0150] "ultrasound_organ_shape": "Enlarged",

[0151] "ct_lesion_size": "Medium",

[0152] "mri_signal_intensity": "Mildly Abnormal",

[0153] "label": "At Risk"

[0154] } ]

[0156] S2: Model fine-tuning: fine-tune the AI ​​large model based on PEFT and perform quantitative operations;

[0157] S3: Multimodal local large model deployment: Build a high-performance server equipped with a high-end graphics card, and then install LM Studio software locally to load the model;

[0158] S4: Parameter adjustment and API service of multimodal local large model: adjust the batch size, GPU load and Temperature parameters, and open the API to call the multimodal local large model.

[0159] Implementation Example 6

[0160] Attached Figure 7 The implementation principle of epidemic prediction-support vector machine (SVM) is demonstrated. SVM has excellent pattern recognition ability and is suitable for processing small sample problems. It is a powerful supervised learning algorithm. Its core idea is to find an optimal hyperplane. The specific process is:

[0161] S1: Kernel function selection: The project uses the radial basis function kernel, specifically:

[0162]

[0163] S2: Construct optimization objective: Construct optimization objective function, specifically:

[0164]

[0165] S3: Training model: Use the training data set to train the model and obtain the model parameters through the gradient descent algorithm;

[0166] S4: Model deployment and application: The trained SVM algorithm is deployed on the server to predict epidemics by analyzing electronic medical records.

[0167] The specific implementation method described above provides a detailed description of the objectives, technical solutions and beneficial effects of the present invention. It should be understood that the above is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent medical remote diagnosis and treatment system based on artificial intelligence, which uses convolutional neural networks to extract features of ultrasound, computer tomography and magnetic resonance imaging, and uses long short-term memory networks to extract features of electrocardiogram, blood pressure and blood sugar, and then fuses the features with electronic medical records, and finally uses a multimodal local large model to diagnose structured data, characterized in that: The implementation steps of the system are: S1: Data acquisition: Obtain medical image data collected by ultrasound equipment, computer tomography scanners and magnetic resonance imaging machines; obtain time series data collected by electrocardiographs, blood pressure monitors and blood glucose meters; S2: Data preprocessing: Preprocess the medical image data and time series data; the preprocessing methods of the medical image data are rotation, flipping, and scaling, thereby improving the data quality and enhancing the generalization ability of the model; the preprocessing methods of the time series data are abnormal data point removal and data standardization; the abnormal data point removal can improve the accuracy and robustness of the model, specifically: > in, Expressed as data points; It is expressed as a threshold value, provided by the multimodal local large model; Expressed as the mean of time series data, it can be expressed as: in, is the number of data points; It is expressed as the standard deviation of time series data, which measures the degree of dispersion of data. The calculation formula is: The data standardization can eliminate the dimension effect, accelerate the model training process, and improve the stability of the training process. Specifically: in, Expressed as standardized medical data, It is expressed as the original time series data obtained in step S1; S3: Feature extraction: The image data obtained in step S2 is processed using a convolutional neural network, and the spatial features of the image are extracted through multi-layer convolution and pooling operations, and then fused with the electronic medical record; the time series data obtained in step S2 is processed using a long short-term memory network to capture the time-dependent features of the health status, and then fused with the electronic medical record; S4: Data transmission: Protect the patient's electronic medical record data through identity authentication and authorization mechanisms to ensure the security of data transmission and avoid data leakage risks; S5: Diagnosis: Analyze and diagnose the fused structured data using a multimodal local big model to obtain the patient's physiological state; the multimodal local big model is based on parameter fine-tuning technology; the parameter fine-tuning technology is described as producing a large number of data sets for the field of medical diagnosis, combining professional medical knowledge with the general capabilities of the big model, and obtaining a multimodal local big model for medical scenarios; S6: Data analysis: Support vector machine is used to determine the status level of the epidemic; the support vector machine uses radial basis function kernel to achieve autonomous monitoring of the epidemic through hyperparameter optimization and model deployment; the radial basis function kernel It is expressed as: in, It is expressed as width, which represents the influence range of the kernel function; Representing data points in the input space; S7: Real-time interaction: The system can transmit the diagnosis results back to the patient, realizing real-time two-way interaction between the user and the doctor; the system provides multidisciplinary joint consultation function for difficult cases, promoting collaborative sharing of medical resources; S8: Manual verification: Physicians verify the diagnosis results of the multimodal local large model to improve the reliability of the diagnosis results; S9: Model update: The system supports continuous learning and performance monitoring of the model, and regularly updates the model to incorporate the latest medical data and research results to ensure the advancement and adaptability of its diagnostic capabilities.

Images