Quality monitoring method and device

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a quality monitoring method and device based on a medical large model, and the method comprises the steps: a basic layer carries out the collection, storage, processing and safety guarantee of data; the model layer is used for constructing and maintaining a large medical model, including pre-training and fine tuning; the application layer converts a result generated by the large medical model into practical application to serve different medical scenes; and the public module provides safety guarantee and management support. Compared with the prior art, the method can ensure the accuracy and reliability in practical application, thereby improving the quality and efficiency of medical services.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically provides a quality monitoring method and device. Background Art

[0002] With the rapid development of artificial intelligence (AI), especially the rise of deep learning, AI has demonstrated strong application potential in many fields. In the healthcare sector, using AI to improve the quality and efficiency of medical services has become a research hotspot.

[0003] AI can help doctors make more accurate diagnoses and provide personalized treatment recommendations by analyzing large amounts of medical data. However, traditional AI models are often limited to specific tasks and lack sufficient generalization capabilities to cope with diverse medical application scenarios.

[0004] In recent years, large models have become a key trend in AI development due to their powerful generalization and versatility. Through pre-training and fine-tuning, large medical models can excel in various medical tasks, such as medical text generation, medical image recognition, drug development, and medical education.

[0005] However, existing medical big models still face certain challenges in terms of data security, technology maturity, and application scenario implementation, especially how to ensure the accuracy and reliability of medical big models in practical applications. Summary of the Invention

[0006] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a quality monitoring method with strong practicality.

[0007] A further technical task of the present invention is to provide a quality monitoring device that is rationally designed, safe and applicable.

[0008] The technical solution adopted by the present invention to solve its technical problem is:

[0009] A quality monitoring method based on a medical big model, including:

[0010] The basic layer collects, stores, processes and ensures data security;

[0011] The model layer builds and maintains large medical models, including pre-training and fine-tuning;

[0012] The application layer converts the results generated by the medical large model into practical applications to serve different medical scenarios;

[0013] Common modules provide security and management support.

[0014] Furthermore, the base layer includes:

[0015] The data collection and annotation module collects medical data from multiple channels and annotates the data;

[0016] The data storage and management module uses cloud computing, big data technologies and related data governance strategies to store and manage data securely;

[0017] The data security and privacy protection module uses encryption technology and access control measures to ensure the security and privacy of medical data.

[0018] Furthermore, the model layer includes:

[0019] Pre-training module: The pre-training module is based on the unlabeled dataset of the medical model to pre-train the model so that the model can obtain medical knowledge background;

[0020] The fine-tuning module uses a small-scale labeled dataset to fine-tune the model for specific medical tasks and optimize its performance on specific tasks;

[0021] The model architecture module adopts the Transformer architecture and uses the self-attention mechanism to capture long-distance dependencies and contextual information in the input data.

[0022] Furthermore, the application layer includes:

[0023] Medical text generation module, which automatically generates structured electronic medical records, case reports, and research abstracts;

[0024] Medical image analysis module, which automatically analyzes medical images to assist doctors in disease diagnosis and treatment decisions;

[0025] Drug R&D support module, which uses large models for drug design, screening and optimization to accelerate the development of new drugs;

[0026] The medical education module provides virtual patient simulation and medical knowledge question-and-answer functions to support the training and education of medical students.

[0027] Furthermore, the common module includes:

[0028] Regulatory governance module, formulating relevant policies and regulations to standardize the development and application of medical big models;

[0029] Security capability modules, including data encryption and identity authentication, as well as specific security solutions targeting the sensitivity and confidentiality of medical data;

[0030] Performance monitoring module: monitors the performance and security of the model in real time, and promptly identifies and resolves problems;

[0031] The user interface module provides a user interface for easy use by medical staff and patients.

[0032] Furthermore, the common module also includes:

[0033] Data quality monitoring module, which verifies input data and regularly checks the quality of data annotation;

[0034] The model performance monitoring module provides real-time monitoring and regular evaluation, and sets performance thresholds and performance early warning mechanisms;

[0035] Security monitoring module, which monitors the security of data transmission and storage as well as the security of monitoring models;

[0036] The ethics monitoring module ensures the fairness and unbiasedness of model output through bias detection, content review, and user feedback.

[0037] Furthermore, the model performance monitoring module includes:

[0038] Real-time monitoring to continuously track the performance of large medical models, including accuracy, specificity, sensitivity, precision, F1 score, and AUC-ROC;

[0039] Regular evaluation: Evaluate the model regularly, using standardized test data sets and evaluation metrics;

[0040] Performance warning: Set performance thresholds. When the model performance is lower than the threshold, the system automatically issues a warning to remind relevant personnel to adjust or optimize the model.

[0041] Furthermore, the security monitoring module includes:

[0042] Data security, monitoring the security of data transmission and storage;

[0043] Model security, monitoring the security of the model;

[0044] Access control: set up strict access control mechanisms to ensure that only authorized personnel can access the system and data.

[0045] Furthermore, the ethics monitoring module includes:

[0046] Bias detection: Regularly review medical data and model outputs to identify and correct bias and unfairness in the application of large medical models;

[0047] Content review: establish content review and filtering mechanisms, and clarify the definition and regulatory strategy of harmful content;

[0048] User feedback: Encourage patients and medical staff to provide feedback, monitor errors and problems in the application of large medical models, and make timely model corrections and updates.

[0049] A quality monitoring device, comprising: at least one memory and at least one processor;

[0050] The at least one memory is configured to store a machine-readable program;

[0051] The at least one processor is configured to call the machine-readable program to execute a quality monitoring method.

[0052] Compared with the prior art, the quality monitoring method and device of the present invention have the following outstanding beneficial effects:

[0053] (1) Improve data quality. Ensure the integrity and consistency of input data through data validation and data annotation quality checks, thereby improving the training and application of large medical models. Improve data quality and reduce the impact of noise and irrelevant data on model performance by performing data cleaning, deduplication, and outlier detection.

[0054] (2) Ensure model performance. Through real-time monitoring and regular evaluation, ensure the accuracy and reliability of large medical models in practical applications. Real-time monitoring can promptly detect any degradation in model performance, and regular evaluation can comprehensively assess the model's performance to ensure its long-term stable operation. Set performance thresholds and performance warning mechanisms. When model performance falls below the threshold, the system automatically issues an alert, reminding relevant personnel to adjust or optimize the model, thereby ensuring that the model is always in optimal condition.

[0055] (3) Ensure data and model security, and ensure the security and privacy of data during transmission and storage through data encryption, access control, and other means to prevent data leakage and abuse;

[0056] By integrating an adversarial attack detection system, potential attack behaviors can be monitored and identified in real time to prevent adversarial attacks and malicious use, thus ensuring the security of the model.

[0057] (4) Ensure ethics and regularly review medical data and model outputs through bias detection tools to identify and correct bias and unfairness in the application of large medical models and ensure the fairness and non-bias of model outputs.

[0058] Establish content review and filtering mechanisms, clarify the definition and regulatory strategies for harmful content, and prevent medical big models from generating harmful, false, or misleading information.

[0059] Encourage patients and medical staff to provide feedback, monitor errors and problems in the application of large medical models, and make timely model corrections and updates to ensure the accuracy and reliability of model output.

[0060] (5) Improve the quality and efficiency of medical services. By automatically generating structured electronic medical records, case reports, research summaries, and other functions, the workload of medical staff can be reduced and work efficiency can be improved. By automatically analyzing medical images, doctors can be assisted in disease diagnosis and treatment decision-making, improving the accuracy and speed of diagnosis. Through drug design, screening, and optimization systems, the research and development process of new drugs and medical devices can be accelerated, costs can be reduced, and success rates can be increased. Through virtual patient simulation, medical knowledge Q&A, and other functions, the training and education of medical students can be supported, improving the quality and effectiveness of medical education. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] Attachment Figure 1 It is a flow chart of a quality control method. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention will be further described in detail below in conjunction with specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0064] A best embodiment is given below:

[0065] like Figure 1 As shown, a quality monitoring method in this embodiment, a quality monitoring method based on a medical big model, includes:

[0066] The basic layer collects, stores, processes and ensures data security;

[0067] Further including:

[0068] Data collection and annotation module, used to collect medical data through multiple channels and perform high-quality data annotation;

[0069] Data storage and management module, which uses cloud computing, big data technologies, and related data governance strategies to ensure secure data storage and efficient management;

[0070] The data security and privacy protection module is used to use encryption technology, access control and other means to ensure the security and privacy of medical data and prevent data leakage and abuse.

[0071] The model layer builds and maintains large medical models, including pre-training and fine-tuning;

[0072] Further including:

[0073] Pre-training module, used for pre-training based on large-scale unlabeled datasets, so that the model can acquire a wide range of medical knowledge background;

[0074] The fine-tuning module is used to fine-tune the model for specific medical tasks using a high-quality, small-scale annotated dataset to optimize the model's performance on specific tasks;

[0075] The model architecture module adopts the Transformer architecture and uses the self-attention mechanism to capture long-distance dependencies and contextual information in the input data, thereby improving the generalization ability and accuracy of the model.

[0076] The application layer converts the results generated by the medical large model into practical applications to serve different medical scenarios;

[0077] Further including:

[0078] Medical text generation module, used to automatically generate structured electronic medical records, case reports, research summaries, etc., reducing the workload of medical staff;

[0079] Medical image analysis module, used to automatically analyze medical images and assist doctors in disease diagnosis and treatment decisions;

[0080] Drug R&D support module, used to use large models for drug design, screening and optimization, accelerating the development of new drugs;

[0081] The medical education module is used to provide virtual patient simulation, medical knowledge question and answer and other functions to support the training and education of medical students.

[0082] The public module provides security and management support to ensure the stable operation of the system;

[0083] Further including:

[0084] A regulatory governance module, used to formulate relevant policies and regulations, standardize the development and application of large medical models, and ensure data privacy and security;

[0085] Security capability modules, which include general security measures such as data encryption and identity authentication, as well as specific security solutions tailored to the sensitivity and confidentiality of medical data;

[0086] Performance monitoring module, used to monitor the performance and security of the model in real time, and to promptly identify and resolve problems;

[0087] A user interface module, used to provide a friendly user interface;

[0088] Data quality monitoring module, used to verify input data, ensure data integrity and consistency, and regularly check the quality of data annotation;

[0089] The model performance monitoring module is used to ensure the accuracy and reliability of the model through real-time monitoring and regular evaluation, and to set performance thresholds and performance early warning mechanisms.

[0090] These include real-time monitoring: continuous tracking of the performance of large medical models, monitoring indicators including accuracy, specificity, sensitivity, precision, F1 score, AUC-ROC, etc.;

[0091] Regular evaluation: Regularly evaluate the model using standardized test data sets and evaluation metrics to ensure the accuracy and reliability of the model;

[0092] Performance warning: Set a performance threshold. When the model performance is lower than the threshold, the system automatically issues a warning to remind relevant personnel to adjust or optimize the model.

[0093] Security monitoring module, used to monitor the security of data transmission and storage to prevent data leakage and abuse, as well as monitor the security of models to prevent adversarial attacks and malicious use;

[0094] Among them, include:

[0095] Data security: monitor the security of data transmission and storage to prevent data leakage and abuse;

[0096] Model security: monitor the security of the model to prevent adversarial attacks and malicious use;

[0097] Access control: Set up strict access control mechanisms to ensure that only authorized personnel can access systems and data.

[0098] An ethics monitoring module to ensure the fairness and unbiasedness of model outputs through bias detection, content review, and user feedback;

[0099] Among them, include:

[0100] Bias detection: Regularly review medical data and model outputs to identify and correct bias and unfairness in the application of large medical models;

[0101] Content review: Establish content review and filtering mechanisms, clarify the definition and regulatory strategy of harmful content, and prevent medical big models from generating harmful, false, or misleading information;

[0102] User feedback: Encourage patients and medical staff to provide feedback, monitor errors and problems in the application of large medical models, and make timely model corrections and updates.

[0103] Based on the above method, a quality monitoring device in this embodiment includes: at least one memory and at least one processor;

[0104] The at least one memory is configured to store a machine-readable program;

[0105] The at least one processor is configured to call the machine-readable program to execute a quality monitoring method.

[0106] The above-mentioned specific implementation methods are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementation methods. Any technical solutions that conform to the above-mentioned specific implementation methods of the present invention and any appropriate changes or substitutions made thereto by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0107] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A quality monitoring method, characterized in that: Based on the medical big model, including: The basic layer collects, stores, processes and ensures data security; The model layer builds and maintains large medical models, including pre-training and fine-tuning; The application layer converts the results generated by the medical big model into practical applications to serve different medical scenarios; Common modules provide security and management support.

2. A quality monitoring method according to claim 1, characterized in that: The base layer includes: The data collection and annotation module collects medical data from multiple channels and annotates the data; The data storage and management module uses cloud computing, big data technology and related data governance strategies to store and manage data securely; The data security and privacy protection module uses encryption technology and access control methods to ensure the security and privacy of medical data.

3. A quality monitoring method according to claim 2, characterized in that: The model layer includes: Pre-training module: The pre-training module is based on the unlabeled data set of the medical large model to pre-train the model so that the model can obtain medical knowledge background; The fine-tuning module uses a small-scale labeled dataset to fine-tune the model for specific medical tasks and optimize the performance of the model on specific tasks; The model architecture module adopts the Transformer architecture and uses the self-attention mechanism to capture long-distance dependencies and contextual information in the input data.

4. A quality monitoring method according to claim 3, characterized in that: The application layer includes: Medical text generation module, which automatically generates structured electronic medical records, case reports and research abstracts; Medical image analysis module, which automatically analyzes medical images to assist doctors in disease diagnosis and treatment decision-making; Drug R&D support module, which uses large models for drug design, screening and optimization to accelerate the development of new drugs; The medical education module provides virtual patient simulation and medical knowledge question-and-answer functions to support the training and education of medical students.

5. A quality monitoring method according to claim 4, characterized in that: The common modules include: Regulatory governance module, formulating relevant policies and regulations to regulate the development and application of medical big models; Security capability modules, including data encryption and identity authentication, as well as specific security solutions for the sensitivity and confidentiality of medical data; Performance monitoring module, which monitors the performance and security of the model in real time, and promptly discovers and solves problems; The user interface module provides a user interface for easy use by medical staff and patients.

6. A quality monitoring method according to claim 5, characterized in that: The common module also includes: Data quality monitoring module, which verifies input data and regularly checks the quality of data annotation; The model performance monitoring module monitors in real time and conducts regular evaluations, and sets performance thresholds and performance warning mechanisms; Security monitoring module, which monitors the security of data transmission and storage as well as the security of monitoring models; The ethics monitoring module ensures the fairness and unbiasedness of the model output through bias detection, content review and user feedback.

7. A quality monitoring method according to claim 6, characterized in that: The model performance monitoring module includes: Real-time monitoring, which continuously tracks the performance of large medical models, including accuracy, specificity, sensitivity, precision, F1 score, and AUC-ROC; Regular evaluation: Evaluate the model regularly, using standardized test data sets and evaluation metrics; Performance warning: Set performance thresholds. When the model performance is lower than the threshold, the system automatically issues a warning to remind relevant personnel to adjust or optimize the model.

8. A quality monitoring method according to claim 7, characterized in that: The security monitoring module includes: Data security, monitoring the security of data transmission and storage; Model security, monitoring the security of the model; Access control: set up strict access control mechanisms to ensure that only authorized personnel access the system and data.

9. A quality monitoring method according to claim 8, characterized in that: The ethics monitoring module includes: Bias detection: Regularly review medical data and model outputs to identify and correct bias and unfairness in the application of large medical models; Content review: establish content review and filtering mechanisms, and clarify the definition and supervision strategy of harmful content; User feedback: Encourage patients and medical staff to provide feedback, monitor errors and problems in the application of large medical models, and make timely model corrections and updates.

10. A quality monitoring device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 9.

Images