Prescription multi-mode intelligent auditing system based on artificial intelligence

Abstract

The invention discloses an artificial intelligence-based prescription multi-modal intelligent auditing system, which comprises a multi-modal data acquisition module, a data preprocessing module, a feature fusion module, an intelligent auditing model and a result feedback and knowledge base updating module, the multi-modal data comprises basic information of a patient, diagnosis information, inspection report data, medical image data and prescription text information, the data is automatically collected through the multi-modal data collection module, a deep learning algorithm is used for rapid processing and analysis, rapid examination and verification of a prescription are completed, comprehensive analysis is conducted in combination with the multi-modal data, and the accuracy of the prescription is improved. Comprising basic information, diagnosis information, inspection report data, medical image data and the like of the patient, whether the prescription has the problems of drug interaction, dosage abnormity, mismatching of indications and the like or not can be judged more comprehensively and accurately, in this way, auditing errors are effectively reduced, the drug use risk of the patient is reduced, and the drug use safety of the patient is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of prescription review, and in particular relates to an artificial intelligence-based multimodal intelligent prescription review system. Background Art

[0002] In the medical process, prescriptions are important medical documents that doctors use to formulate treatment plans for patients. Their accuracy and rationality are directly related to the patient's treatment effect and medication safety.

[0003] Currently, prescription review relies primarily on manual methods, with pharmacists reviewing each prescription individually based on their professional knowledge and experience. However, this traditional manual review model is inefficient. During busy medical periods, pharmacists struggle to quickly process a large number of prescriptions, which can easily lead to review backlogs and affect patients' medication pickup time and treatment progress. Furthermore, manual review is susceptible to factors such as pharmacists' individual knowledge and fatigue, making it difficult to ensure accuracy and consistency. Potential prescription issues, such as drug interactions, inappropriate dosages, and mismatches with indications, may be missed, posing medication risks to patients. With the development of medical information technology, rule-based prescription review systems have emerged. These systems compare and judge prescription information based on pre-set rules. However, these systems lack flexibility and struggle to cope with complex clinical situations and the constant flow of updated medical knowledge. They are unable to effectively review special cases or newly emerging drugs. Furthermore, existing review systems often focus solely on the textual content of prescriptions, ignoring other relevant patient information, such as medical imaging and laboratory reports, and are unable to comprehensively assess the rationality of prescriptions from multiple dimensions. Summary of the Invention

[0004] The purpose of the present invention is to provide an artificial intelligence-based multimodal intelligent prescription review system to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solutions: a multimodal intelligent review system for prescriptions based on artificial intelligence, comprising:

[0006] A multimodal data acquisition module, which is used to collect multimodal data related to the prescription, including basic patient information, diagnosis information, test report data, medical imaging data, and prescription text information;

[0007] A data preprocessing module, which is connected to the multimodal data acquisition module through a data interface and preprocesses the collected multimodal data;

[0008] A feature fusion module, which is connected to the data preprocessing module through a data interface and uses a deep learning algorithm to fuse the preprocessed multimodal data features;

[0009] An intelligent audit model is connected to the feature fusion module through a data interface. A deep neural network model is constructed based on the fused comprehensive feature vector to perform prescription audits. The intelligent audit model is trained with a large amount of historical prescription data and corresponding audit results. It learns the characteristic patterns of normal and abnormal prescriptions to determine whether there are problems with the prescriptions and outputs detailed audit results.

[0010] The result feedback and knowledge base update module is connected to the intelligent audit model through a data interface to feedback the audit results to doctors and pharmacists. If there are any problems with the prescription, detailed modification suggestions and references are provided. At the same time, the audit knowledge base is automatically updated based on clinical feedback and the latest medical research results, and the intelligent audit model is retrained based on the updated knowledge base.

[0011] Preferably, the multimodal data acquisition module realizes automatic data collection and integration by connecting with the hospital information system.

[0012] Preferably, the hospital information system includes an electronic medical record system, a testing system and an imaging system, and the multimodal data acquisition module interacts with the electronic medical record system, the testing system and the imaging system through a network interface to obtain the patient's multimodal data.

[0013] Preferably, in the data preprocessing module, when performing word vector conversion on text data, a word vector model such as Word2Vec, GloVe or BERT is used to convert the text information into a numerical form.

[0014] Preferably, in the feature fusion module, medical image features are extracted through a convolutional neural network, text data features are learned using a Transformer model, and then feature splicing and weighted fusion are used to fuse features of different modalities into a comprehensive feature vector.

[0015] Preferably, during the training process, the intelligent audit model adopts the cross entropy loss function as the loss calculation method, and uses the stochastic gradient descent, Adagrad, Adadelta or Adam optimization algorithm to update the model parameters.

[0016] Preferably, the audit results include drug interaction prompts, dosage abnormality information, indication mismatch instructions, and potential prescription problem prompts, and the audit results are presented to doctors and pharmacists in the form of a visual interface.

[0017] Preferably, when fusing multimodal data features, the feature fusion module assigns different weights to each modal feature according to the importance of different modal data to prescription review, so as to achieve more accurate feature fusion.

[0018] Preferably, when updating the audit knowledge base, the result feedback and knowledge base update module uses a combination of manual review and automated verification to verify the accuracy of newly added medical knowledge and drug information to ensure the quality of the audit knowledge base.

[0019] Preferably, when performing preprocessing in the data preprocessing module, word segmentation, stop word removal, and word vector conversion operations are performed on text data, numerical test report data is normalized, and image enhancement, noise reduction, and segmentation are performed on medical image data to extract key feature information.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) Data is automatically collected through the multimodal data acquisition module, and deep learning algorithms are used for rapid processing and analysis to complete the rapid review of prescriptions. During busy business hours, the system can also dynamically adjust the data collection frequency according to the load to ensure that the latest patient data is obtained in a timely manner for review. This greatly shortens the time patients wait for medication, optimizes the medical process, and improves patient satisfaction with medical treatment.

[0022] (2) The constructed intelligent audit model is trained with a large amount of historical prescription data and corresponding audit results, and learns the characteristic patterns of normal prescriptions and abnormal prescriptions. At the same time, it combines multimodal data for comprehensive analysis, including the patient's basic information, diagnostic information, test report data and medical imaging data, etc., to more comprehensively and accurately judge whether there are problems such as drug interactions, abnormal dosage, mismatch of indications, etc. in the prescription. In this way, audit errors are effectively reduced, the patient's medication risk is reduced, and the patient's medication safety is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a module composition diagram of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0025] The present invention provides Figure 1 An artificial intelligence-based multimodal intelligent review system for prescriptions is shown below:

[0026] 1. System deployment and environment construction

[0027] Hardware selection: We selected high-performance servers equipped with NVIDIA A100 GPUs. Their powerful parallel computing capabilities can accelerate the training and inference of deep learning models. Paired with Intel Xeon Platinum 8380 processors with 64 cores, they provide ample computing resources for data processing and system operation.

[0028] Software Configuration: CentOS 7.9 was chosen as the server operating system, which offers high stability and compatibility. The Anaconda environment manager was installed on this system to facilitate the management of Python virtual environments and the installation of various dependent libraries. A dedicated project environment was created based on Anaconda. The deep learning framework PyTorch 1.12.1 was installed, which excels in building and training neural network models. CUDA 11.3 and cuDNN 8.2 were also installed to enable GPU-accelerated computing, significantly improving model training efficiency.

[0029] Network connection: Connect the server to the hospital's internal LAN via 10 Gigabit Ethernet to ensure stable connection with the electronic medical record system, testing system, and imaging system. Configure a static IP address, set an appropriate subnet mask, gateway, and DNS server to ensure the reliability and security of data transmission. Use firewall and data encryption technology to prevent data leakage and network attacks, and ensure the security of the hospital information system.

[0030] 2. Data Collection and Preprocessing

[0031] 1. Data Collection

[0032] Collection time and frequency: The system is set to automatically collect data every 15 minutes. During peak business hours, such as 9:00-11:00 am and 2:00-4:00 pm, the collection frequency can be dynamically adjusted to every 10 minutes based on system load to ensure timely acquisition of the latest patient data;

[0033] Data Sources and Interfaces: Retrieve patient basic information (name, age, gender, ethnicity, contact information, etc.), diagnosis information (detailed disease diagnosis descriptions, such as "acute bronchitis, with symptoms of cough and expectoration"), allergy history, and other text data from the Electronic Medical Record System (EMR). Transmit the data through the HL7 interface (i.e., the Health Level Seven interface, a standard protocol for electronic data exchange between medical information systems, aiming to achieve seamless communication and data sharing between different medical devices and information systems) to ensure unified data format specifications. Retrieve various types of test report data, such as blood routine (white blood cell count, red blood cell count, hemoglobin concentration, platelet count, etc.), biochemical indicators (alanine aminotransferase, aspartate aminotransferase, bilirubin of liver function, creatinine, urea nitrogen of kidney function, blood glucose, blood lipids, etc.) from the Laboratory Information System (LIS), and use the DICOM standard interface for data collection. Retrieve medical imaging data, such as X-ray films, CT scan images, MRI images, etc. from the Picture Archiving and Communication System (PACS), and achieve the transmission and reception of imaging data through the DICOM network protocol.

[0034] 2. Data Preprocessing

[0035] Text Data Processing: Use the spaCy library in Python for word segmentation of text data. This library is efficient and accurate in processing natural language text. Through functions such as词性标注 (part-of-speech tagging) and named entity recognition, accurately identify key information in the text, remove stop words (such as words with no actual semantic meaning like "的", "了", "在", etc.) to reduce data redundancy, and use the pre-trained BERT-Base-Chinese model (for Chinese text) or BERT-Base-Uncased model (for English text) for word vector conversion, converting each word into a 768-dimensional word vector to enable the computer to effectively process and understand the text data;

[0036] Numerical Data Processing: For the numerical data in the test reports, first perform outlier detection using the statistical 3σ principle. That is, if a data point is more than 3 times the standard deviation away from the mean, it is determined as an outlier. For outliers, handle them according to the specific situation. If it is an obviously incorrect data, correct or delete it. If it is an outlier under special physiological conditions, retain and mark it. Then, use the Z-Score normalization method to normalize the numerical data. The formula is:

[0037]

[0038] Where X is the original data, μ is the data mean, and σ is the standard deviation. Map the data to the standard normal distribution interval with a mean of 0 and a standard deviation of 1 for subsequent model training and comparison;

[0039] Image data processing: Medical images are preprocessed using the OpenCV library. First, histogram equalization technology is used to enhance image contrast and make image details clearer. For X-rays, the contrast between bones and soft tissues is enhanced by adjusting the grayscale histogram. For CT images, the distinction between different tissues and organs is improved. Then, the Gaussian filtering algorithm is used to remove image noise. This algorithm is based on the Gaussian kernel function and can retain edge information while smoothing the image. Finally, the U-Net convolutional neural network model is used for image segmentation. For different types of medical images (such as lung images and brain images), corresponding segmentation models are trained to accurately segment regions of interest (ROIs), such as lung tissue in lung images and brain tissue in brain images, providing a basis for subsequent feature extraction and analysis.

[0040] 3. Model Training and Optimization

[0041] Building an intelligent audit model: Based on the Transformer architecture, the intelligent audit model consists of an input layer, multiple Transformer blocks, a fully connected layer, and an output layer. The input layer receives the fused multimodal feature vector; the Transformer block contains a multi-head attention mechanism, a feedforward neural network, and layer normalization operations. The multi-head attention mechanism can simultaneously focus on different parts of the input data and capture complex feature relationships. The feedforward neural network performs further nonlinear transformations on the features output by the attention mechanism, and layer normalization standardizes the data to accelerate model convergence; the fully connected layer integrates and maps the features output by the Transformer block; the output layer uses the Softmax function to output the probability distribution of the prescription audit results, such as normal prescriptions, drug interactions, abnormal dosages, and other different categories;

[0042] Data preparation: 500,000 prescriptions containing complete multimodal data were screened from the hospital's electronic medical records over the past five years. These data were cleaned and annotated, removing samples with serious data missing, incorrect formatting, or illogical logic. The annotation work was completed by experienced pharmacists and doctors. Each prescription was reviewed and the types of problems (such as drug interactions, unreasonable dosage, inconsistent indications, etc.) and the corresponding correct review results were marked. The annotated data was divided into training set, validation set, and test set in a ratio of 8:1:1. The training set was used to train the model, allowing the model to learn the characteristic patterns of normal and abnormal prescriptions; the validation set was used to evaluate model performance during training, adjust model hyperparameters, and prevent overfitting; and the test set was used to finally evaluate the generalization ability and accuracy of the model.

[0043] Model training: During the training process, the training set data is input into the model in batches, each batch of data contains 64 samples, the number of training rounds is set to 100, the initial learning rate is 0.0001, and the cross-entropy loss function is used as the optimization target of the model. This function can effectively measure the difference between the model prediction results and the true labels. The Adam optimization algorithm is used to update the model parameters. The Adam algorithm combines the advantages of the momentum method and the adaptive learning rate. It can converge quickly during the training process and avoid falling into the local optimal solution. In each round of training, the model performs forward propagation calculations based on the input multimodal feature vector to obtain the predicted audit results. Then, the gradient of the loss function with respect to the model parameters is calculated through the backpropagation algorithm, and the parameters are updated using the Adam optimization algorithm. After every 5 rounds of training, the model performance is evaluated on the validation set, and indicators such as accuracy, recall rate, and F1 value are calculated. If the F1 value on the validation set does not improve after 10 consecutive rounds, the training is stopped and the current optimal model is saved.

[0044] 4. Real-time review and result feedback

[0045] Real-time review: When a doctor writes a prescription in the electronic medical record system and clicks the Save button, the system immediately triggers the review process. The multimodal data acquisition module quickly obtains the patient's latest data from relevant information systems, including basic information, diagnosis information, test report data, and medical imaging data. After data preprocessing and feature fusion, it is input into the trained intelligent review model for inference. The model outputs the review results within seconds, determines whether there are any problems with the prescription, and provides a detailed classification and description of any existing problems.

[0046] Result feedback: The audit results are fed back to the doctor in the form of pop-up windows and message reminders. The pop-up window clearly displays the problems with the prescription, such as "Drug A and Drug B interact with each other, which may lead to increased liver toxicity, please use with caution", "The dosage of drug C exceeds the normal range, it is recommended to adjust the dosage according to the patient's weight and renal function", etc. At the same time, relevant medical literature links are provided as a reference to help doctors understand the severity of the problem and how to deal with it. The audit results will also be synchronized to the pharmacist workstation, and the pharmacist can review and provide additional comments on the audit results. If the doctor has any questions about the audit results, he can click the "View Details" button in the pop-up window to view the basis and relevant data of the system analysis, including the patient's multimodal data, model reasoning process and other information, for better communication and decision-making.

[0047] 5. Knowledge Base Update

[0048] Information Collection: The system automatically collects medical information from multiple channels at 2 a.m. every day. It uses web crawler technology to obtain the latest medical research papers from medical journal databases (such as PubMed, Wanfang Medical Network, and CNKI Medicine), screening out research results related to prescription review, such as drug interactions, drug dosage adjustments, and disease indications. It also obtains updated information on drug instructions and adverse drug reaction notifications from drug regulatory agency websites (such as the National Medical Products Administration and the FDA). It also collects problems and feedback from doctors and pharmacists in clinical practice from the hospital's internal case discussion system and medical quality monitoring system.

[0049] Update and review the knowledge base: classify and organize the collected information, use natural language processing technology to perform text analysis on medical research papers and drug instructions, and extract key information, such as new discoveries about drug interactions, the basis for drug dosage adjustments, changes in disease indications, etc. For information fed back from within the hospital, manually review and confirm it, add the sorted new information to the review knowledge base, and update the rules and knowledge entries in the knowledge base. During the update process, adopt a version management mechanism to record the update history of the knowledge base for easy traceability and management. After the update is completed, use the new knowledge base to retrain the intelligent review model so that the model can adapt to the updates of medical knowledge and changes in clinical practice in a timely manner. At the same time, regularly conduct comprehensive reviews and optimizations on the knowledge base, delete expired or erroneous knowledge entries, and ensure the accuracy and effectiveness of the knowledge base.

[0050] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal intelligent prescription review system based on artificial intelligence, characterized by: include: A multimodal data acquisition module, which is used to collect multimodal data related to the prescription, including basic patient information, diagnosis information, test report data, medical imaging data, and prescription text information; A data preprocessing module, which is connected to the multimodal data acquisition module through a data interface and preprocesses the collected multimodal data; A feature fusion module, which is connected to the data preprocessing module through a data interface and uses a deep learning algorithm to fuse the preprocessed multimodal data features; An intelligent audit model, which is connected to the feature fusion module via a data interface and constructs a deep neural network model based on the fused comprehensive feature vector to perform prescription audits. The intelligent audit model is trained using a large amount of historical prescription data and corresponding audit results; The result feedback and knowledge base update module is connected to the intelligent audit model through a data interface and is used to feed back the audit results to doctors and pharmacists.

2. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: The multimodal data acquisition module realizes automatic data collection and integration by connecting with the hospital information system.

3. The artificial intelligence-based multimodal intelligent prescription review system according to claim 2, characterized in that: The hospital information system includes an electronic medical record system, a testing system and an imaging system. The multimodal data acquisition module interacts with the electronic medical record system, the testing system and the imaging system through a network interface to obtain multimodal data of patients.

4. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: In the data preprocessing module, when performing word vector conversion on text data, Word2Vec, GloVe or BERT word vector model is used to convert text information into numerical form.

5. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: In the feature fusion module, medical image features are extracted through a convolutional neural network, text data features are learned using a Transformer model, and then features of different modalities are fused into a comprehensive feature vector using feature splicing and weighted fusion.

6. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: During the training process, the intelligent audit model adopts the cross entropy loss function as the loss calculation method, and uses the stochastic gradient descent, Adagrad, Adadelta or Adam optimization algorithm to update the model parameters.

7. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: The review results include drug interaction prompts, dosage abnormality information, indication mismatch instructions, and potential prescription problem prompts, and the review results are presented to doctors and pharmacists in the form of a visual interface.

8. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: When fusing multimodal data features, the feature fusion module assigns different weights to each modal feature according to the importance of different modal data to prescription review, so as to achieve more accurate feature fusion.

9. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: When updating the audit knowledge base, the result feedback and knowledge base update module uses a combination of manual review and automated verification to verify the accuracy of newly added medical knowledge and drug information to ensure the quality of the audit knowledge base.

10. The artificial intelligence-based multimodal intelligent prescription review system according to claim 1, characterized in that: During preprocessing, the data preprocessing module performs word segmentation, stop word removal, and word vector conversion operations on text data, normalizes numerical test report data, and performs image enhancement, noise reduction, and segmentation on medical image data to extract key feature information.

Images