Medical staff AI competency gatekeeping evaluation system and method under digital MDT mode

Abstract

The system obtains target MDT cases, AI suggestions and medical staff data, standardizes the generation of basic data packages, and constructs AI-MDT interaction event sequences; generates medical staff AI system application competency portraits and AI function permission states based on case risks, AI suggestion types, historical portraits, training and examination, and model version learning states; generates a set of flow control conditions in combination with model version states and the integrity of traditional Chinese medicine characteristic data chains, performs gate control judgment before AI suggestions are submitted as MDT discussion opinions or MDT conclusion candidates, and outputs release, prevention, correction, review, superior confirmation, training or re-authorization control actions. The system also evaluates traditional Chinese medicine characteristic data chains, difference reasons, AI suggestion closed loops and model version changes, and feeds back updated portraits, AI function permission states and subsequent gate control conditions, thereby improving the safety, standardization, traceability and continuous improvement capability of AI suggestion flow.

Description

Technical Field

[0001] This invention relates to the fields of medical AI safety control and dynamic evaluation of the digital capabilities of medical personnel, and in particular to a gating evaluation system and method for the AI ​​competence of medical personnel under a digital MDT model. Background Technology

[0002] Multidisciplinary team (MDT) care is a treatment model in which medical professionals from multiple disciplines collaborate on case discussions, treatment plan analysis, and follow-up management for difficult, complex, and critical cases. With the development of medical informatics and artificial intelligence technologies, digital MDT platforms are increasingly connecting with AI-assisted diagnostic systems, electronic medical record systems, examination and testing systems, traditional Chinese medicine diagnostic information collection systems, access control systems, and training and assessment systems to improve the efficiency of case data processing, risk alerts, and diagnostic references.

[0003] Existing AI-assisted diagnosis and treatment systems can typically generate auxiliary diagnoses, medication suggestions, treatment plan recommendations, risk predictions, or TCM-assisted syndrome differentiation results. Existing digital multidisciplinary team (MDT) platforms can typically complete consultation requests, data collection, expert discussions, opinion recording, and conclusion generation. However, most existing systems only display AI suggestions as reference information or as objects of post-analysis, lacking a gating mechanism whereby the server, based on the medical staff's actual operational trajectory, case risk level, AI suggestion type, and permission status, performs actions such as granting, blocking, correcting, reviewing, obtaining higher-level confirmation, or re-authorizing before the AI ​​suggestions enter the MDT discussion or MDT conclusion candidates.

[0004] On the other hand, current evaluations of medical staff's AI application capabilities or job competence largely rely on questionnaires, self-assessments, exam scores, training completion records, or expert ratings. These evaluations typically focus on static knowledge acquisition or training results, failing to accurately reflect the actual operational behaviors of medical staff in the AI-MDT process, such as reviewing AI suggestions, tracing evidence, manual review, recording reasons for discrepancies, reviewing TCM diagnostics, and learning model versions. Existing evaluation results usually remain at the post-event statistical level and are difficult to directly translate into executable control parameters within a digital MDT platform, such as reviewing, reviewing, submitting, receiving superior confirmation, enabling high-risk AI functions, and restricting access.

[0005] For TCM hospitals or integrated TCM and Western medicine MDT scenarios, existing TCM AI systems can use data from observation, auscultation, inquiry, and palpation to assist in diagnosis, but they lack a mechanism to construct a computable data chain from the original data of the four diagnostic methods, syndrome elements, AI diagnosis results, human diagnosis review opinions, TCM treatment plans, and follow-up feedback, and to use the integrity of this data chain as a pre-control condition for AI diagnosis suggestions to enter into MDT discussion opinions or MDT conclusion candidates.

[0006] Furthermore, after an AI model is updated, existing systems typically only indicate the version change, lacking a control process to trigger version learning, simulated case assessments, access restrictions, and re-authorization for medical personnel based on changes in output fields, scope of application, risk warning rules, model logic, or TCM syndrome differentiation labeling system.

[0007] Therefore, there is an urgent need for a technical solution that can generate a competency profile of medical staff in applying AI systems based on real AI-MDT interaction event sequences, and transform this profile, along with case risk, AI suggestion type, model version status, and the integrity of the TCM-specific data chain, into pre-submission gating conditions for AI suggestions, so as to achieve pre-control, permission feedback, and closed-loop traceability of the AI ​​suggestion flow process. Summary of the Invention

[0008] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is to provide a gating evaluation system and method for the AI ​​competence of medical staff in a digital MDT mode. Based on the real AI interaction events of medical staff in the digital MDT process, the system can perform system-level gating control on the AI ​​suggestions before they enter the MDT discussion opinions or MDT conclusion candidates, and simultaneously realize the dynamic evaluation of the medical staff's AI system application competence, permission feedback update and model version re-authorization.

[0009] To achieve the above objectives, this invention provides a gating assessment system for AI competence of medical personnel in a digital MDT (Multidisciplinary Team) model, comprising:

[0010] The multi-source AI-MDT data acquisition and governance module is used to acquire data related to target MDT cases, target AI suggestions, and participating medical personnel, and to perform data anonymization, field standardization, and data quality verification to generate a standardized basic data package.

[0011] The AI-MDT interaction event sequence generation module is used to convert the operation logs and business data in the standardized basic data package into AI-MDT interaction events, and associate them according to the target medical staff, target cases, target MDT consultations and target AI suggestions to form a time-ordered AI-MDT interaction event sequence corresponding to the target AI suggestions;

[0012] The case risk and scenario determination module is used to generate the risk level and AI application scenario category of the target MDT case based on the target MDT case data, AI system risk prompts, current MDT stage and consultation topic;

[0013] The AI ​​system application competency profile module for medical personnel is used to generate or update the AI ​​system application competency profile of participating medical personnel based on one or any combination of the AI-MDT interaction event sequence, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results, and to determine the AI ​​function permission status of medical personnel.

[0014] The competency-related control condition generation module is used to generate a set of flow control conditions that the target AI suggestion must meet before entering the MDT discussion opinions or MDT conclusion candidates, based on the risk level of the target MDT case, the category of AI application scenario, the type of target AI suggestion, the competency profile of medical staff in applying the AI ​​system, the AI ​​function permission status of medical staff, the AI ​​model version status, and the integrity of the TCM characteristic data chain when the target AI suggestion belongs to the TCM auxiliary diagnosis type AI suggestion or the TCM treatment plan recommendation type AI suggestion.

[0015] The AI ​​suggestion flow gating judgment module is used to read the AI-MDT interaction event sequence, the medical staff's AI system application competence profile, the medical staff's AI function permission status, and the flow control condition set when medical staff intend to submit the target AI suggestion as an MDT discussion opinion or MDT conclusion candidate. Based on the flow control condition set, it determines whether the target AI suggestion meets the pre-submission gating requirements and outputs the release result, blocking result, correction prompt, review reminder, superior confirmation request, training task, model version re-authorization request, or permission restriction result.

[0016] The competency evaluation result feedback control module is used to update the medical staff's AI system application competency profile, the medical staff's AI function permission status, and subsequent flow control conditions based on one or any combination of the gating results, review results, reasons for discrepancies, closed-loop processing results, training completion status, model version learning results, and follow-up feedback results from the AI ​​suggestion flow gating judgment module.

[0017] As a further improvement of the present invention, it also includes one or any combination of the following modules: threshold weight and flow rule configuration module, traditional Chinese medicine characteristic data chain evaluation module, AI suggestion difference reason evaluation module, AI suggestion closed-loop processing evaluation module, AI model version adaptability evaluation and re-authorization module, and abnormal data processing module.

[0018] The threshold weight and flow rule configuration module is used to read the threshold configuration table, weight configuration table and flow control rule configuration table of the current effective version, and provide the current effective configuration version to one or any combination of the case risk and scenario judgment module, medical staff AI system application competency profile module, competency association control condition generation module, AI suggestion flow gating judgment module, traditional Chinese medicine characteristic data chain evaluation module, AI suggestion closed-loop processing evaluation module and AI model version adaptability evaluation and re-authorization module;

[0019] The TCM-featured data chain evaluation module is used to evaluate the integrity of the data chain between the original data of the four diagnostic methods, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plans and follow-up feedback when the target AI suggestion belongs to the TCM-assisted syndrome differentiation AI suggestion or the TCM treatment plan recommendation AI suggestion.

[0020] The AI ​​suggestion difference evaluation module is used to generate a structured record of the reasons for the difference when there are differences between the target AI suggestion, the medical staff's review opinion, the opinion to be submitted to the MDT discussion, or the final conclusion of the MDT.

[0021] The AI ​​suggestion closed-loop processing and evaluation module is used to construct the AI ​​suggestion closed-loop object based on the target AI suggestion, medical staff review opinions, reasons for differences, MDT final conclusions and follow-up feedback, and to determine the AI ​​suggestion processing result;

[0022] The AI ​​model version adaptability evaluation and re-authorization module is used to generate version learning tasks, field explanation and confirmation tasks, simulated case assessment tasks, or permission re-authorization requirements when the AI ​​model version undergoes changes in output fields, scope of application, model logic, risk warning rules, or TCM syndrome differentiation labeling system.

[0023] The abnormal data processing module is used to generate abnormal status markers, alternative processing results, or correction tasks according to preset abnormal processing rules when there are missing AI suggestions, missing AI confidence fields, missing data from the four diagnostic methods of traditional Chinese medicine, missing follow-up data, unstructured MDT conclusions, missing case risk fields, unknown AI model versions, missing historical competency profiles, or missing permission audit data.

[0024] This invention also discloses a gating evaluation method for AI competence of medical staff in a digital MDT model, comprising the following steps:

[0025] S100. Obtain and standardize basic data:

[0026] Acquire data on target MDT cases, target AI suggestions, and relevant medical staff, perform anonymization, field standardization, and data quality verification, and generate a standardized basic data package;

[0027] S200, Constructing the AI-MDT Interaction Event Sequence:

[0028] Operation logs and business data are converted into AI-MDT interaction events and then sorted and associated according to target medical personnel, target cases, target MDT consultations, and target AI suggestions.

[0029] S300: Generate a competency profile of medical personnel for AI system applications.

[0030] Based on the case risk level, AI suggestion type, historical profile, training and assessment records, model version learning status, and the AI-MDT interaction event sequence, generate or update the AI ​​system application competency profile of medical staff, and determine the AI ​​function permission status of medical staff.

[0031] S400, AI suggestion flow control conditions based on competency profile generation:

[0032] Based on the case risk level, AI application scenario category, AI suggestion type, AI system application competence profile of medical staff, AI function permission status of medical staff, and AI model version status, a set of flow control conditions that the target AI suggestion must meet before entering the MDT discussion or MDT conclusion candidate is generated; when the target AI suggestion belongs to the TCM auxiliary diagnosis type AI suggestion or the TCM treatment plan recommendation type AI suggestion, the evaluation results of the TCM characteristic data chain are called or the integrity of the generated TCM characteristic data chain is read, and it is used as the basis for generating TCM-related flow control conditions;

[0033] S500, Gating before submitting AI recommendations:

[0034] When medical staff intend to submit a target AI suggestion, the system reads the AI-MDT interaction event sequence, the medical staff's AI system application competency profile, the medical staff's AI function permission status, and the set of flow control conditions. It then judges the medical staff's AI function permission status, the completion status of necessary events, mandatory control conditions, event completeness, superior confirmation requirements, and model version re-authorization requirements, and outputs a release result or a blocking result. If blocked, it triggers one or any combination of control actions such as correction prompts, review reminders, superior confirmation requests, training tasks, model version re-authorization, or permission restrictions.

[0035] S600, Evaluation of Competency in Implementing AI Applications with Traditional Chinese Medicine Characteristics:

[0036] When the target AI suggestion belongs to the category of TCM auxiliary diagnosis AI suggestion or TCM treatment plan recommendation AI suggestion, the integrity of the data chain between the four diagnostic methods data, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plan and follow-up feedback is evaluated; S600 is executed as needed before, during or when S400 generates TCM-related circulation control conditions, or when S500 makes gating judgment.

[0037] S700, Evaluation of AI suggestion discrepancy handling capability:

[0038] When there are discrepancies between the target AI suggestions, review opinions, proposed MDT discussion opinions, or the final conclusion of the MDT, generate a structured record of the reasons for the discrepancies.

[0039] S800's ability to execute AI suggestions in a closed-loop processing capability evaluation:

[0040] Based on the target AI suggestions, review opinions, reasons for discrepancies, final conclusions of the MDT, and follow-up feedback, construct a closed-loop object for AI suggestions and determine the processing results of AI suggestions;

[0041] S900, Implement AI Model Version Adaptive Re-authorization and Special Adjustment of Permission Status:

[0042] When the AI ​​model version changes in output fields, scope of application, model logic, risk warning rules, or TCM syndrome differentiation labeling system, version learning tasks, field explanation confirmation tasks, simulated case assessment tasks, or permission re-authorization requirements are generated, and the corresponding AI function permission status is adjusted according to the completion status.

[0043] As a further improvement of the present invention, S300 further includes:

[0044] The risk level of the target MDT case is determined based on the target MDT case data, AI system risk alerts, the current MDT stage, and the consultation topic.

[0045] Read the historical profiles, historical ability dimension scores, historical AI function permission status, historical risk events, training completion status, and model version learning status of the participating medical personnel;

[0046] When no historical profile exists, an initial profile is generated based on the medical staff's job role, professional title, authorized AI function scope, training records and / or simulated case assessment results, and initial AI function permissions are set.

[0047] Based on the AI-MDT interaction event sequence, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results, update one or any combination of the following in the medical staff's AI system application competency profile: competency dimension score, comprehensive competency score, AI function permission status, scope of usable AI functions, historical risk events, training completion status, and model version learning status.

[0048] As a further improvement of the present invention, S400 further includes:

[0049] Basic control conditions are generated based on the risk level of the target MDT cases;

[0050] Adjust the basic control conditions according to the target AI suggestion type, the current MDT stage, the AI ​​model version status, and the AI ​​function permission status of medical staff;

[0051] When at least one competency dimension in the competency profile of medical staff in the AI ​​system is lower than the preset threshold, additional requirements will be added for viewing, reviewing, tracing evidence, recording the reasons for the discrepancy, confirming with superiors, training, or re-authorizing the model version corresponding to that competency dimension.

[0052] When the target AI suggestion belongs to the category of AI suggestions for TCM-assisted syndrome differentiation or AI suggestions for TCM treatment plan recommendations, additional requirements are added based on the integrity of the TCM-specific data chain, including supplementary recording of the four diagnostic methods, verification of AI syndrome differentiation results, completion of manual syndrome differentiation verification opinions, or association with TCM treatment plans. The integrity of the TCM-specific data chain is determined by at least a portion of the data from the original data of the four diagnostic methods, syndrome elements, AI syndrome differentiation results, manual syndrome differentiation verification opinions, TCM treatment plans, and follow-up feedback in the standardized basic data package. When the target AI suggestion does not belong to the category of AI suggestions for TCM-assisted syndrome differentiation or AI suggestions for TCM treatment plan recommendations, the integrity of the TCM-specific data chain is not a mandatory transfer control condition.

[0053] When multiple control conditions apply to the same target AI recommendation, the control conditions are integrated according to the strictest principle; if there is a conflict between the release and restriction between different control conditions, the restriction condition is used as the final control condition; if different control conditions correspond to different confirmation levels, the higher confirmation level is used as the final confirmation requirement.

[0054] As a further improvement of the present invention, S500 further includes:

[0055] Read the set of flow control conditions corresponding to the target AI suggestion. The set of flow control conditions includes at least mandatory control conditions, correctable control conditions, superior confirmation conditions, model version re-authorization conditions, and permission level conditions.

[0056] Based on the necessary event types in the set of flow control conditions, retrieve whether the corresponding interaction event has been completed in the AI-MDT interaction event sequence;

[0057] When a necessary event type is marked as a mandatory control condition and the corresponding interactive event is not completed, it is determined that the target AI suggestion does not meet the pre-submission gating requirements.

[0058] When the AI ​​function permission status of medical staff does not meet the permission level conditions corresponding to the target AI suggestion, it is determined that the target AI suggestion does not meet the pre-submission gating requirements.

[0059] When the event completeness does not reach the event completeness threshold corresponding to the target AI suggestion, the target AI suggestion is deemed not to meet the pre-submission gating requirements.

[0060] For target AI suggestions that do not meet the pre-submission gating requirements, output the blocking result, and trigger one or any combination of control actions based on the incomplete control conditions, such as correction prompts, review reminders, superior confirmation requests, training task pushes, model version re-authorization, or permission restrictions;

[0061] For target AI suggestions that meet the pre-submission gating requirements, the release result is output, and the gating result, operation time, operator anonymity identifier, target AI suggestion identifier, medical staff AI function permission status and configuration version are written into the AI-MDT interaction event sequence.

[0062] As a further improvement of the present invention, S600 can be invoked when generating TCM-related circulation control conditions in S400, or when performing pre-submission gating judgment in S500; S600 also includes:

[0063] When the target AI suggestion belongs to the category of AI suggestion for TCM auxiliary diagnosis or AI suggestion for TCM treatment plan recommendation, the TCM characteristic data chain nodes that should be completed should be determined according to the current MDT stage.

[0064] Before the consultation, at least the completeness of the original data of the four diagnostic methods, syndrome elements and AI diagnosis results should be assessed.

[0065] During the consultation phase, at least the completeness of the manual diagnostic review opinion should be assessed;

[0066] During the MDT conclusion formation stage, at least the correlation between the TCM treatment plan and the final syndrome diagnosis should be determined.

[0067] In the post-consultation phase, the follow-up feedback will serve as the basis for subsequent profile updates and closed-loop evaluation;

[0068] When the TCM-specific data chain nodes required for the current MDT stage are missing, tasks such as supplementing TCM four diagnostic methods data, supplementing syndrome elements, manual syndrome differentiation review, or associating TCM treatment plans are generated. Before the correction is completed, the corresponding TCM-related AI suggestions are restricted from entering the MDT discussion opinions or MDT conclusion candidates.

[0069] As a further improvement of the present invention, S700 further includes:

[0070] Before medical staff submit the target AI suggestion as an opinion in the MDT discussion or as a candidate for the MDT conclusion, compare the differences between the target AI suggestion and the opinion that the medical staff intends to submit; and after the final conclusion of the MDT is formed, compare the differences between the target AI suggestion, the medical staff's review opinion and the final conclusion of the MDT.

[0071] When the difference reaches a preset threshold, a task is generated to generate a structured record of the reason for the difference.

[0072] The reasons for the discrepancies include one or any combination of the following: insufficient AI basis, insufficient data, individual mismatch, TCM syndrome correction, coverage of MDT expert opinions, risk control, patient factors, and other reasons confirmed manually.

[0073] The results of recording the reasons for the differences are written into the AI-MDT interaction event sequence and used as the basis for updating the medical staff's ability to review AI suggestions, identify differences, or process closed loops.

[0074] As a further improvement of the present invention, S800 further includes:

[0075] The AI ​​suggestion closed-loop object is constructed based on the target AI suggestions, the review opinions of medical staff, the reasons for the differences, the final conclusion of the MDT, and the follow-up feedback;

[0076] The processing result of the AI ​​suggestion is determined, including adoption, partial adoption, correction, rejection, omission, or no processing;

[0077] The credibility of AI-recommended processing results is determined based on the completeness of human review opinions, the completeness of records of reasons for differences, the degree of support for the final conclusion of the MDT, and the degree of support for follow-up feedback.

[0078] When follow-up feedback has not yet been generated or is not applicable to the target AI suggestion, the corresponding AI suggestion closed-loop object is marked as pending follow-up update status, and the remaining review opinions, reasons for differences and final conclusions of MDT are used for phased evaluation;

[0079] The results of AI suggestion processing and the credibility of AI suggestion processing are fed back to the AI ​​system application competency profile of medical staff, so as to update the clinical translation ability, AI suggestion review ability, closed-loop processing ability, or continuous learning and improvement ability of AI suggestions.

[0080] As a further improvement to the present invention, it also includes S1000, performing abnormal data processing and competency profile feedback update:

[0081] When AI suggestions are missing, AI confidence field is missing, TCM four diagnostic methods data is missing, follow-up data is missing, MDT conclusions are unstructured, case risk field is missing, AI model version is unknown, historical competency profile is missing, or permission audit data is missing, the system generates an abnormal status marker, alternative processing results, or correction tasks; among them, when the AI ​​confidence field is missing, the system uses AI risk level, AI evidence completeness, or AI scope of application prompts as alternative parameters;

[0082] When data from the four diagnostic methods of TCM is missing, the system generates a task to supplement the data and restricts the corresponding AI suggestions for TCM auxiliary diagnosis or TCM treatment plan recommendation from directly entering the MDT conclusion candidate before the supplementation is completed.

[0083] When the AI ​​model version is unknown, the system marks the AI ​​model version status as unknown and restricts the submission of high-risk AI functions;

[0084] When historical competency profiles are missing, the system generates an initial profile and sets basic AI function permissions.

[0085] The S1000 is also used to distinguish between abnormal data on the system side and missing operations by medical staff, so as to avoid directly including data missing caused by the AI ​​system not returning, follow-up data not being generated, or external system fields not being synchronized into the negative evaluation of medical staff. In addition, the system updates the AI ​​system application competency profile of medical staff, the AI ​​function permission status of medical staff, and subsequent flow control conditions based on the gating results of S500, the difference reason recording results of S700, the AI ​​suggestion closed-loop processing results of S800, and the model version learning results or re-authorization results of S900.

[0086] Compared with the prior art, the present invention has at least the following beneficial effects:

[0087] 1. This invention is based on AI-MDT interactive event sequence recording of real operational behaviors such as AI suggestion generation, viewing, evidence tracing, manual review, recording of reasons for differences, superior confirmation, model version learning, and follow-up feedback. This makes the evaluation of medical staff's AI system application competence no longer dependent on questionnaires, self-assessment, or single exams, but rather derived from traceable and calculable system event data.

[0088] 2. This invention transforms the competency profiles and permission status of medical personnel in the AI ​​system into AI suggestion flow control conditions, enabling the evaluation results to directly affect the viewing, review, submission, superior confirmation, model version re-authorization, and opening of high-risk AI functions. This transforms competency evaluation from a post-event statistical result into a real-time control parameter in the AI ​​suggestion flow process.

[0089] 3. This invention generates a set of flow control conditions based on the case risk level, AI suggestion type, current MDT stage, AI function permission status of medical staff, model version status, and the integrity of the TCM characteristic data chain. When multiple conditions are applicable at the same time, a strict principle is adopted to avoid using the same gating standard for different risk cases, different AI suggestion types, and different personnel with different permissions.

[0090] 4. When the target AI suggestion has not completed the review, evidence tracing, manual review, four diagnostic data supplementation, AI diagnosis review, difference reason recording, superior confirmation, or model version learning, the present invention can prevent it from entering the MDT discussion opinions or MDT conclusion candidates and trigger the correction, review, confirmation, training or re-authorization tasks, thereby realizing the pre-control of the AI ​​suggestion submission interface, candidate interface or high-risk AI function.

[0091] 5. This invention sets up a data chain integrity evaluation for AI suggestions in TCM auxiliary diagnosis and AI suggestions in TCM treatment plan recommendation, which includes the original data of the four diagnostic methods, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plan and follow-up feedback, to improve the data support integrity and subsequent traceability of TCM AI suggestions before they enter the MDT process.

[0092] 6. When the AI ​​model version changes in output fields, scope of application, risk warning rules, model logic, or TCM syndrome differentiation labeling system, this invention can trigger version learning, field interpretation confirmation, simulated case assessment, and permission re-authorization, reducing the situation where medical staff continue to use high-risk AI functions independently without understanding the changes in the output logic or scope of application of the new version of AI.

[0093] 7. This invention enables the system to continue operating even when there is a lack of AI confidence, missing data from the four diagnostic methods, unstructured MDT conclusions, delayed follow-up, or unknown model version, through mechanisms such as abnormal state marking, alternative parameters, correction tasks, follow-up updates, and permission restrictions. Furthermore, it updates the competency profile and permission status in reverse through gating results, reasons for differences, and closed-loop results, forming a continuous control system of "use—gating—correction—closed-loop—feedback—re-gating".

[0094] This invention can be deployed in tertiary TCM hospitals, integrated TCM and Western medicine hospitals, general hospitals, regional TCM medical consortia, digital MDT platforms of medical groups, and medical AI system monitoring platforms. It can utilize real-world operational data from existing hospital information systems to dynamically evaluate the AI ​​system application behavior of medical staff in the digital MDT process. Based on the evaluation results, it can implement risk control and process gating for the process of medical AI suggestions entering MDT discussion opinions or MDT conclusion candidates. Furthermore, this invention can improve the security, standardization, traceability, and continuous improvement capabilities of medical AI systems in digital MDT scenarios. Attached Figure Description

[0095] Figure 1 This is a schematic block diagram of the system configuration of the present invention;

[0096] Figure 2 This is a schematic diagram of the operation flow of the method of the present invention. Detailed Implementation

[0097] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0098] The AI ​​competency gating evaluation system for medical staff under the digital MDT model in this embodiment can be deployed on the hospital's intranet server, hospital data platform server, digital MDT platform server, edge computing server, or cloud-edge collaborative medical information platform, and can interact with the hospital's existing digital MDT platform, AI-assisted diagnosis and treatment system, electronic medical record system, traditional Chinese medicine four diagnostic information collection system, examination and testing system, training and assessment system, access audit system, and follow-up management system.

[0099] The system in this embodiment does not require a complete replacement of the hospital's existing information system. Instead, it operates as a risk gating and quality traceability system positioned between the digital MDT platform and the medical AI system. The system determines and controls whether AI suggestions can be included in MDT discussion opinions or MDT conclusion candidates by reading AI suggestions, case data, medical staff operation records, MDT process records, traditional Chinese medicine diagnostic data, model version information, and permission audit information.

[0100] See Figure 1 The AI ​​competency gating evaluation system for medical staff under the digital MDT model in this embodiment includes:

[0101] The multi-source AI-MDT data acquisition and governance module is used to acquire data related to target MDT cases, target AI suggestions, and participating medical staff from digital MDT platforms, AI-assisted diagnosis and treatment systems, electronic medical record systems, TCM four diagnostic information acquisition systems, examination and testing systems, training and assessment systems, access control systems, model management systems, and follow-up management systems. It also performs anonymization processing on patient identities, medical staff identities, and case numbers, converts data from different systems into a unified field format, and generates standardized basic data packages, which are then provided to the AI-MDT interactive event sequence generation module, case risk and scenario determination module, medical staff AI system application competency profiling module, and AI suggestion flow gating judgment module.

[0102] The threshold weight and flow rule configuration module is used to read the threshold configuration table, weight configuration table, and flow control rule configuration table of the current valid version. It is also used to provide the current valid configuration version to the case risk and scenario judgment module, the medical staff AI system application competency profile module, the competency association control condition generation module, the AI ​​suggestion flow gating judgment module, the traditional Chinese medicine characteristic data chain evaluation module, the AI ​​suggestion closed-loop processing evaluation module, and the AI ​​model version adaptability evaluation and re-authorization module within the same evaluation cycle, and record the configuration version number.

[0103] The AI-MDT interaction event sequence generation module converts the operation logs and business data output by the multi-source AI-MDT data acquisition and governance module into a time-ordered AI-MDT interaction event sequence. It also merges duplicate logs, unifies timestamps from different system sources, and generates events to be associated when AI suggestion identifiers are missing or event attribution is unclear. The generated AI-MDT interaction event sequence is used for subsequent competency profile calculations, flow control condition generation, and pre-submission gating judgments.

[0104] The case risk and scenario determination module is used to generate risk scores, risk levels, and application scenario categories for target MDT cases based on basic case data, AI system risk alerts, the current MDT stage, and consultation topics. It also generates provisional risk levels according to the strictest risk principle when case risk fields are missing, and outputs the case data correction task to the abnormal data processing module or the competency association control condition generation module. The risk levels include low risk, medium risk, high risk, critical illness, and complex / difficult cases. The application scenario categories include general case summaries, assisted diagnosis, medication review, treatment plan recommendation, TCM auxiliary diagnosis, TCM treatment plan recommendation, follow-up management, and high-risk decision support scenarios.

[0105] The AI ​​system application competency profiling module for medical personnel is used to read or generate AI system application competency profiles of participating medical personnel. It calculates scores for each capability dimension and a comprehensive competency score based on AI-MDT interaction event sequences, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results. It is also used to determine the AI ​​function permission status of medical personnel based on the comprehensive competency score and major red line event records. The permission status includes basic usage status, restricted usage status, regular authorization status, and advanced authorization status. The permission status is output to the competency association control condition generation module and the AI ​​suggestion flow gating judgment module.

[0106] The competency-related control condition generation module is used to generate a set of flow control conditions that the target AI suggestion must meet before entering the MDT discussion or MDT conclusion candidates, based on the case risk level and application scenario category output by the case risk and scenario judgment module, the personnel profile and permission status output by the medical personnel AI system application competency profile module, the target AI suggestion type, the current MDT stage, the AI ​​model version status, and the integrity of the TCM characteristic data chain when the target AI suggestion belongs to the TCM auxiliary syndrome differentiation AI suggestion or the TCM treatment plan recommendation AI suggestion.

[0107] The AI ​​suggestion flow gating judgment module is used to read the AI-MDT interaction event sequence corresponding to the target AI suggestion, the medical staff's AI system application competence profile, the current permission status, and the flow control condition set output by the competence-related control condition generation module when medical staff intend to submit the target AI suggestion as an MDT discussion opinion or MDT conclusion candidate. It is also used to write the gating result, incomplete control conditions, triggered control actions, operation time, operator anonymity identifier, target AI suggestion identifier, permission status, and threshold configuration version into the AI-MDT interaction event sequence. The AI ​​suggestion flow gating judgment module determines whether the target AI suggestion meets the pre-submission gating requirements based on the necessary event type, mandatory control conditions, event completeness threshold, permission level conditions, superior confirmation conditions, and model version re-authorization conditions in the flow control condition set. If the AI ​​function permission status of medical staff meets the requirements, all mandatory control conditions are completed, and the event completeness reaches the threshold, the module outputs a release result, allowing the target AI suggestion to enter the MDT discussion opinion or MDT conclusion candidate. If the AI ​​function permission status of medical staff does not meet the requirements, any mandatory control condition is not completed, or the event completeness is below the threshold, a blocking result is output, triggering a correction prompt, review reminder, superior confirmation request, training task push, model version re-authorization, or permission restriction. The release or blocking result is used to control the opening, disabling, hiding, or entering the pending confirmation state of the AI ​​suggestion submission interface, MDT discussion opinion submission interface, or MDT conclusion candidate submission interface in the digital MDT platform.

[0108] The Traditional Chinese Medicine (TCM)-specific data chain evaluation module is used to evaluate the integrity of the data chain among the original data of the four diagnostic methods (inspection, auscultation and olfaction), syndrome elements, AI diagnosis results, human diagnosis review opinions, TCM treatment plans, and follow-up feedback when the target AI suggestion belongs to the category of TCM-assisted syndrome differentiation or TCM treatment plan recommendation. This module can be invoked when the competency association control condition generation module generates TCM-related flow control conditions, or when the AI ​​suggestion flow gating judgment module performs pre-submission gating judgment. Before consultation, this module evaluates at least the integrity of the original data of the four diagnostic methods, syndrome elements, and AI diagnosis results; during consultation, it evaluates at least the integrity of the human diagnosis review opinions; during MDT conclusion formation, it evaluates at least the association between the TCM treatment plan and the final syndrome judgment; and after consultation, the follow-up feedback status can be used as the basis for subsequent profile updates and closed-loop evaluation. When the TCM-specific data chain nodes required for the current MDT stage are missing, this module generates tasks such as supplementing TCM four diagnostic methods data, supplementing syndrome elements, manually reviewing syndrome differentiation, or associating TCM treatment plans. Before the correction is completed, it outputs control results that restrict the corresponding AI syndrome differentiation suggestions from entering the MDT discussion opinions or MDT conclusion candidates.

[0109] The AI ​​suggestion discrepancy evaluation module is used to generate a structured record of the reasons for discrepancies when there are significant differences between the target AI suggestion, the medical staff's review opinions, the proposed MDT discussion opinions, or the final MDT conclusion. This module can compare the differences between the target AI suggestion and the medical staff's proposed opinions before the medical staff submits their AI suggestions, or it can compare the differences between the target AI suggestion, the medical staff's review opinions, and the final MDT conclusion after the final MDT conclusion is formed. The reasons for discrepancies include insufficient AI basis, insufficient data, individual mismatch, TCM syndrome correction, MDT expert opinion coverage, risk control, patient factors, and other reasons confirmed manually. The structured record of the reasons for discrepancies is used to evaluate the medical staff's ability to review and judge AI suggestions, their ability to identify discrepancies, and their closed-loop processing ability, and serves as a data source for updating the medical staff's AI system application competency profile.

[0110] The AI ​​suggestion closed-loop processing evaluation module is used to construct an AI suggestion closed-loop object based on the target AI suggestion, medical staff review opinions, reasons for discrepancies, MDT final conclusions, and follow-up feedback, and to determine the AI ​​suggestion processing result. The processing result includes adoption, partial adoption, correction, rejection, omission, or no processing, and is used to calculate the credibility of the AI ​​suggestion processing result based on the completeness of the human review opinions, the completeness of the record of reasons for discrepancies, the degree of support for the MDT final conclusions, and the degree of support for the follow-up feedback. When follow-up feedback has not yet been generated or is not applicable to the target AI suggestion, the AI ​​suggestion closed-loop processing evaluation module can mark the follow-up feedback indicators as inapplicable in this evaluation period and normalize the weights of the remaining indicators involved in the calculation, or mark the AI ​​suggestion closed-loop object as pending follow-up update. The output closed-loop processing result is used to update medical staff's AI suggestion clinical translation ability, AI suggestion review ability, continuous learning and improvement ability, and subsequent training tasks.

[0111] The competency assessment result feedback control module is used to update the AI ​​function permission status of medical staff based on the gating results of the AI ​​suggestion flow gating judgment module, the AI ​​system application competency profile of medical staff, the learning status of the AI ​​model version, the reasons for discrepancies, the closed-loop processing results, the training completion status, and historical risk events. It also feeds back the gating results, review results, reasons for discrepancies, model version learning results, training completion results, and follow-up feedback results to the AI ​​system application competency profile module of medical staff to update the subsequent AI function usage permissions and gating conditions for medical staff. The permission status includes: only allowing viewing AI suggestions, allowing filling in AI review opinions, allowing submitting MDT discussion opinions, allowing submitting MDT conclusion candidates, requiring confirmation from a senior physician, restricting high-risk AI functions, re-authorizing the model version, and requiring permission to be restored after training completion.

[0112] The abnormal data processing module is used to generate abnormal status markers, alternative processing results, or correction tasks according to preset abnormality handling rules when situations arise such as missing AI suggestions, missing AI confidence fields, missing data from the four diagnostic methods of traditional Chinese medicine, missing follow-up data, unstructured MDT conclusions, missing case risk fields, unknown AI model versions, missing historical competency profiles, or missing permission audit data. It also distinguishes between system-side data anomalies and medical staff operational errors, avoiding the incorrect inclusion of situations such as the AI ​​system not returning data, follow-up data not yet generated, or external system fields not synchronizing in negative evaluations of medical staff. The abnormal status markers at least include the anomaly type, the system from which the anomaly originated, the time of anomaly discovery, the steps affecting the process, the alternative processing method, the status of the correction task, and whether it affects the gate access result. When the AI ​​confidence field is missing, the system can use AI risk level, AI evidence completeness, or AI scope of application as alternative parameters; when TCM diagnostic data is missing, the system triggers a supplementary entry prompt and temporarily disallows the corresponding AI diagnosis results from directly entering the MDT conclusion candidate options; when the AI ​​model version is unknown, the system restricts the submission of high-risk AI functions; when historical competency profiles are missing, the system generates an initial profile and sets basic AI function permissions.

[0113] During runtime, the multi-source AI-MDT data acquisition and governance module outputs standardized basic data packages; the threshold weight and flow rule configuration module outputs currently valid configuration parameters; the AI-MDT interaction event sequence generation module converts operation logs and business data into traceable event sequences; the case risk and scenario determination module generates case risk levels and application scenario categories; the medical staff AI system application competency profile module generates medical staff profiles, comprehensive competency scores, and permission status; the TCM characteristic data chain evaluation module, the AI ​​suggestion difference reason evaluation module, the AI ​​suggestion closed-loop processing evaluation module, and the AI ​​model version adaptability evaluation and re-authorization module respectively output the integrity of the TCM characteristic data chain, the reasons for the differences, the closed-loop processing results, and the model version status.

[0114] The competency-related control condition generation module generates a set of target AI suggestion flow control conditions based on the above data; the AI ​​suggestion flow gating judgment module outputs release, blocking, correction, review, superior confirmation, or re-authorization results based on the AI-MDT interaction event sequence, medical staff AI function permission status, and flow control condition set; the competency evaluation result feedback control module updates the medical staff profile and subsequent permission status based on the gating results, reasons for differences, closed-loop processing results, and model version learning results; the abnormal data processing module generates abnormal status markers, alternative processing results, or correction tasks when data is missing, fields are abnormal, or versions are unknown, to ensure that the system can still perform competency evaluation and AI suggestion gating control in the complex data environment of a real hospital.

[0115] In some embodiments, the multi-source AI-MDT data acquisition and governance module includes:

[0116] The data includes at least target MDT case data, target AI suggestion data, participating medical staff data, and operation log data. The data may further include MDT process data, TCM four diagnostic methods data, training and assessment data, AI model version data, access control audit data, and follow-up feedback data.

[0117] The target MDT case data includes at least the patient anonymity identifier, case anonymity identifier, MDT consultation identifier, current MDT stage, current MDT consultation topic, and case data completeness status. The target AI suggestion data includes at least the AI ​​suggestion identifier, AI system identifier, AI suggestion type, AI model version, AI suggestion generation time, AI suggestion status, corresponding case anonymity identifier, and corresponding MDT consultation identifier. The participating medical personnel data includes at least the medical personnel anonymity identifier, job role, current MDT participation role, professional title, specialty, current AI function permission status, and authorized AI function scope.

[0118] In some embodiments, in the threshold weight and flow rule configuration module:

[0119] The threshold configuration table is used to provide thresholds for case risk, event completeness, TCM characteristic data chain completeness, tag similarity, AI suggestion processing result credibility, model version change learning trigger, model version re-authorization trigger, capability dimension low score threshold, and permission adjustment threshold.

[0120] The weight configuration table is used to provide weights for case risk factors, capability dimensions, event completeness, TCM-specific data chain hierarchy, AI suggestion processing results credibility, model version change impact, and permission adjustment.

[0121] The flow control rule configuration table is used to provide necessary event types, completion thresholds, control actions when not completed, whether to force, whether to allow correction, and confirmation levels for different case risk levels, AI suggestion types, MDT stages, medical staff AI function permission status, and model version status.

[0122] In some embodiments, the AI-MDT interaction event sequence generation module includes:

[0123] The AI-MDT interaction event sequence uses target medical personnel, target cases, target MDT consultations, and target AI suggestions as associated objects to record events such as AI suggestion generation, AI suggestion viewing, AI evidence tracing, manual review opinion filling, TCM four diagnostic data supplementation, AI syndrome differentiation result review, record of reasons for differences, superior confirmation, model version learning, training completion, permission change, and follow-up feedback events.

[0124] Each AI-MDT interaction event includes at least the event identifier, event time, event type, event source system, event status, case anonymity identifier, MDT consultation identifier, and medical staff anonymity identifier. When the event is related to a specific AI suggestion, it also includes an AI suggestion identifier; when the event is related to model version, training, permissions, or follow-up, it also includes an AI model version identifier, training task identifier, permission status identifier, or follow-up record identifier.

[0125] In some embodiments, the competency profiling module of the medical staff AI system applies the following:

[0126] If a medical staff member already has a historical profile, the module reads their historical profile, historical ability dimension score, historical AI function permission status, historical risk events, training completion status, and AI model version learning status. If no historical profile exists, an initial profile is generated based on the job role, professional title level, authorized AI function scope, training records, and simulated case assessment results.

[0127] The profile is used to reflect the status of medical staff in terms of viewing AI suggestions, tracing AI evidence, reviewing AI suggestions, translating AI suggestions into clinical practice, collaborating with digital MDTs, integrating traditional Chinese medicine characteristics, ethical safety and risk governance, adapting to AI model versions, and continuous learning and improvement.

[0128] In some embodiments, the competency-related control condition generation module includes:

[0129] The set of workflow control conditions includes at least mandatory control conditions, correctable control conditions, superior confirmation conditions, model version re-authorization conditions, and permission level conditions. Each workflow control condition includes at least a condition identifier, condition source, applicable case risk level, applicable AI suggestion type, applicable MDT stage, applicable medical staff AI function permission status, necessary event type, completion threshold, control action when not completed, whether correction is allowed, whether superior confirmation is required, and corresponding record fields.

[0130] When multiple control conditions apply to the same target AI recommendation, this module integrates the control conditions according to the strictest principle. If there is a conflict between the release and restriction conditions, the restriction condition is used as the final control condition; if different conditions correspond to different confirmation levels, the higher confirmation level is used as the final confirmation requirement.

[0131] See Figure 2 The AI ​​competence gating evaluation method for medical staff in the digital MDT model of this embodiment is used to dynamically evaluate the AI ​​system application competence of medical staff in the digital MDT process, and to automatically gating the AI ​​suggestions before they enter the MDT discussion opinions or MDT conclusion candidates based on the evaluation results; it includes the following steps:

[0132] S100. Obtain and standardize basic data.

[0133] This step involves acquiring basic data related to the target MDT cases, target AI suggestions, and participating medical personnel from the digital MDT platform, AI-assisted diagnosis and treatment system, electronic medical record system, examination and testing system, TCM four diagnostic methods information collection system, training and assessment system, access control system, model management system, and follow-up management system. The basic data is then anonymized, mapped to fields, formatted, and its quality verified to form the standardized basic data package required for subsequent steps: S200 constructing the AI-MDT interaction event sequence, S300 generating a competency profile of medical personnel applying the AI ​​system, S400 generating AI suggestion flow control conditions, and S500 executing pre-submission gating judgment.

[0134] The standardized basic data package includes at least the target MDT case data, target AI suggestion data, data of participating medical personnel, configuration data, de-identification labeling data, and data quality status data.

[0135] S110. Obtain target MDT case data

[0136] The system obtains the basic case fields of the target MDT case, which include at least the patient anonymity identifier, case anonymity identifier, MDT consultation identifier, current MDT stage, current MDT consultation topic, and case data integrity status.

[0137] The basic fields of the case may further include one or any combination of the following: disease category, severity of illness, whether it is an acute or critical illness, whether it is a difficult or complex case, whether it involves high-risk medication, whether there are serious complications, whether there are contraindications, whether there are differences in diagnosis and treatment between traditional Chinese medicine and Western medicine, consultation application time, consultation status, abnormal examination and test results, past adverse reaction results, and whether multidisciplinary joint decision-making is required.

[0138] Among them, the patient anonymity identifier is generated by the system after desensitizing the patient's identity information, and is used to associate case data in different systems without exposing the real identity; the case anonymity identifier is used to associate electronic medical records, examination and testing, AI suggestions and consultation records in the same case; the MDT consultation identifier is used to associate application, discussion, review, confirmation, conclusion and follow-up data in the same MDT consultation process; the current MDT stage is used to determine whether the target AI suggestion is in the pre-consultation, during-consultation, conclusion formation or post-consultation closed loop stage.

[0139] Disease category, severity of illness, acute and critical illness markers, difficult and complex case markers, high-risk medication markers, serious comorbidity markers, contraindication markers, differences between traditional Chinese medicine and Western medicine diagnosis and treatment markers, and case data completeness status are used for subsequent case risk scoring, generation of AI-suggested flow control conditions, and pre-submission gating judgment.

[0140] When some fields of a target case are missing, the system generates a case data missing marker and writes the missing fields into the data quality status data; for key fields that affect the judgment of the case risk level, the system can handle them in accordance with the principle of strict risk in subsequent S300 or S1000.

[0141] S120. Obtain target AI suggestion data.

[0142] The system acquires target AI suggestion data, which includes at least the AI ​​suggestion identifier, AI system identifier, AI suggestion type, AI model version, AI suggestion generation time, AI suggestion status, corresponding case anonymity identifier, and corresponding MDT consultation identifier.

[0143] The target AI suggestion data may further include AI output content summary, AI confidence level, AI evidence basis status, AI risk warning level, AI scope of application indicator, AI contraindication warning indicator, AI evidence source, whether the AI ​​suggestion requires evidence tracing, whether the AI ​​suggestion involves high-risk decision-making, and whether the AI ​​suggestion belongs to one or any combination of TCM auxiliary diagnosis suggestion or TCM treatment plan recommendation suggestion.

[0144] The types of AI suggestions can include one or any combination of the following: AI suggestions for medical record summaries, AI suggestions for assisted diagnosis, AI suggestions for image recognition, AI suggestions for abnormal test results, AI suggestions for treatment plan recommendations, AI suggestions for medication review, AI suggestions for risk prediction, AI suggestions for auxiliary diagnosis in traditional Chinese medicine, AI suggestions for recommendations of traditional Chinese medicine treatment plans, and AI suggestions for follow-up management.

[0145] The AI ​​model version is used to determine if there have been any model version updates, changes in output fields, changes in applicable scope, changes in risk warning rules, changes in model logic, or changes in the TCM syndrome differentiation labeling system. The AI ​​evidence status is used to determine whether medical personnel need to conduct AI evidence tracing. The AI ​​risk warning level is used to generate subsequent gating conditions. The AI ​​suggestion status can include one of the following: generated, pending viewing, viewed, pending review, reviewed, pending superior confirmation, submitted, blocked, pending correction, pending model version reauthorization, or revoked.

[0146] When AI suggestions are not generated, AI suggestion identifiers are missing, AI confidence fields are missing, or AI model versions are missing, the system generates corresponding abnormal status markers and passes the abnormal status to the subsequent S1000 abnormal data processing flow.

[0147] S130. Obtaining data from participating medical personnel

[0148] The system acquires data on participating medical personnel, which includes at least the medical personnel's anonymous identifier, job role, current MDT participation role, professional title level, specialty, current AI function permission status, and authorized AI function scope.

[0149] The data of participating medical personnel may further include one or any combination of the following: historical AI system application competency profile, historical ability dimension score, historical risk events, training completion status, model version learning status, simulated case assessment results, superior confirmation permission status, whether they are MDT facilitators, whether they are designated experts, and the most recent permission change record.

[0150] The job roles can include one or any combination of the following: clinician, traditional Chinese medicine practitioner, Western medicine specialist, pharmacist, nurse, technician, MDT secretary, senior physician, MDT facilitator, and department manager.

[0151] The current MDT participant role is used to determine whether the medical staff member is a contributor, reviewer, supervisor confirmer, MDT facilitator, or designated expert in the target MDT consultation. The current AI function permission status is used for subsequent S360 permission status determination, S400 workflow control condition generation, and S500 pre-submission gate control judgment. The authorized AI function scope is used to determine whether the medical staff member has the permission to view AI suggestions, fill in AI review opinions, submit MDT discussion opinions, submit MDT conclusion candidates, use high-risk AI functions, and obtain supervisor confirmation or model version re-authorization.

[0152] Historical risk events may include one or any combination of the following: high-risk AI suggestions not reviewed, AI suggestions submitted without completing evidence tracing, attempts to submit high-risk AI suggestions without authorization, use of related AI functions after the AI ​​model version has been updated but has not completed learning, unauthorized access, abnormal use of sensitive data, or re-initiating an AI suggestion submission request that does not meet the flow control conditions after being blocked by gating.

[0153] When a medical staff member's historical profile, current permission status, or model version learning status is missing, the system generates a corresponding abnormal status marker and generates an initial profile, basic usage status, training task, or model version learning task in subsequent S300 or S1000 processes.

[0154] S140, Read Threshold, Weight and Rule Configuration Table

[0155] The system reads the threshold configuration table, weight configuration table, and flow control rule configuration table of the current valid version. These configuration tables provide executable parameters for subsequent case risk scoring, AI-MDT interaction event completeness calculation, TCM-specific data chain completeness calculation, AI suggestion difference handling, AI suggestion processing result credibility calculation, model version change impact scoring, medical staff AI system application competence profile calculation, and pre-submission gating judgment for AI suggestions.

[0156] The threshold configuration table, weight configuration table, and flow control rule configuration table can be stored in the hospital data platform, digital MDT platform, medical AI system management platform, or the configuration database of this system. Within the same evaluation period, the system uniformly calls the currently valid version of the configuration table and records the corresponding configuration version number in the risk score, profile calculation, control condition generation, and gating judgment results.

[0157] S141. Read the threshold configuration table

[0158] The threshold configuration table includes at least the following fields: threshold identifier, threshold name, applicable AI suggestion type, applicable case risk level, applicable MDT stage, applicable role type, threshold value, threshold unit, activation time, expiration time, configuration version number, and priority field.

[0159] The threshold may include one or any combination of the following: case risk threshold, event completeness threshold, TCM characteristic data chain completeness threshold, tag similarity threshold, AI suggestion processing result credibility threshold, model version change learning trigger threshold, model version re-authorization trigger threshold, capability dimension low score threshold, and permission adjustment threshold.

[0160] In some embodiments, the threshold configuration table can be set as follows:

[0161] Applicable scenario: Case risk level assessment; Threshold name: Upper limit threshold for low-risk cases; Reference value: 1; Function: It was determined to be low risk at that time;

[0162] Applicable scenario: Case risk level assessment; Threshold name: Upper limit threshold for medium-risk cases; Reference value: 3; Function: It was determined to be of medium risk at the time;

[0163] Applicable scenario: Case risk level assessment; Threshold name: Upper limit threshold for high-risk cases; Reference value: 5; Function: It was determined to be high risk at that time;

[0164] Applicable scenario: Gating before AI suggestion submission; Threshold name: Low-risk event completeness threshold; Reference value: 0.60; Function: Event completeness requirement before low-risk AI suggestion is released.

[0165] Applicable scenario: Gating before AI suggestion submission; Threshold name: Completeness threshold for medium-risk events; Reference value: 0.75; Function: Completeness requirement for events before allowing medium-risk AI suggestions to proceed.

[0166] Applicable scenario: Gating before AI suggestion submission; Threshold name: High-risk event completeness threshold; Reference value: 0.90; Function: Event completeness requirement before high-risk AI suggestions are allowed.

[0167] Applicable scenario: Gating before AI suggestion submission; Threshold name: Completeness threshold of critical and severe illness events; Reference value: 1.00; Function: Completeness requirements of events before the release of critical and severe or difficult and complex cases.

[0168] Applicable scenario: TCM AI suggestion gating, threshold name: TCM characteristic data chain pre-consultation integrity threshold, reference value: 0.70, function: integrity requirements of the four diagnostic methods data, syndrome elements and AI diagnosis results before consultation;

[0169] Applicable scenario: AI-based gating for Traditional Chinese Medicine (TCM) suggestions; Threshold name: Completeness threshold in TCM characteristic data chain consultation; Reference value: 0.80; Function: Completeness requirements for manual diagnosis and review during consultation.

[0170] Applicable scenario: TCM AI suggestion gating; Threshold name: TCM characteristic data chain conclusion formation threshold; Reference value: 0.85; Function: The requirement for correlation between TCM treatment plan and syndrome judgment in the conclusion formation stage.

[0171] Applicable scenario: AI suggestion for difference recognition; Threshold name: Label similarity threshold; Reference value: 0.60; Function: Trigger the recording of difference reasons when the value is below this value.

[0172] Applicable scenario: AI suggestion difference recognition; Threshold name: AI suggestion processing result credibility threshold; Reference value: 0.70; Function: Generate a review task when the value is below this value.

[0173] Applicable scenario: Model version adaptability evaluation; Threshold name: Model version change learning trigger threshold; Reference value: 0.30; Function: After reaching the threshold, field explanation confirmation or version learning is required.

[0174] Applicable scenario: Model version adaptability evaluation; Threshold name: Model version re-authorization trigger threshold; Reference value: 0.60; Function: Upon reaching this threshold, high-risk AI functions will be restricted and simulated case assessments will be required.

[0175] Applicable scenario: competency profile evaluation; threshold name: low score threshold for ability dimension; reference value: 60; function: add corresponding gating conditions when the score is below this value.

[0176] Applicable scenario: Permission status classification, threshold name: basic usage status upper limit threshold, reference value: 60, function: when the comprehensive competence score is lower than this value, it is marked as basic usage status;

[0177] Applicable scenario: Permission status classification, threshold name: upper limit threshold for restricted use status, reference value: 75, function: when the comprehensive competence score is 60 to 74, it is marked as restricted use status;

[0178] Applicable scenario: Permission status classification, threshold name: upper limit threshold of normal authorization status, reference value: 90, function: when the comprehensive competence score is 75 to 89, it is marked as normal authorization status.

[0179] The above reference values ​​are used to illustrate one possible implementation of the present invention. Medical institutions can adjust the threshold values ​​according to the AI ​​system type, disease characteristics, MDT management system, job authority, and risk management requirements, but the threshold for system calls within the same evaluation period should be derived from the threshold configuration table of the currently valid version.

[0180] S142. Read the weight configuration table

[0181] The weight configuration table includes at least the following fields: weight identifier, weight name, applicable scenario, applicable AI suggestion type, applicable case risk level, weight value, activation time, expiration time, configuration version number, and priority.

[0182] The weights may include one or any combination of the following: case risk factor weights, capability dimension weights, event completeness weights, TCM characteristic data chain hierarchy weights, AI suggestion processing result credibility weights, model version change impact weights, and permission adjustment weights.

[0183] In some embodiments, the weights of case risk factors can be set as follows:

[0184] Critical and severe illness, reference weight: 3;

[0185] Complex and difficult cases, reference weight: 2;

[0186] Invasive treatments are considered, with a reference weight of 2.

[0187] For medications involving high risks, the reference weight is 2.

[0188] The presence of serious comorbidities or multiple coexisting diseases is considered with a weighting of 2.

[0189] For patients with a history of serious adverse reactions or contraindications, the reference weight is 2.

[0190] The AI ​​system outputs a high-risk warning, with a reference weight of 2.

[0191] Case data is severely lacking; reference weight: 1.

[0192] Requires multidisciplinary collaborative decision-making; weighting: 1.

[0193] There are significant differences in diagnosis and treatment between traditional Chinese medicine and Western medicine. Reference weight: 1.

[0194] In some embodiments, the weights of the competency dimension for medical personnel in the AI ​​system can be set as follows:

[0195] AI-suggested review capability, reference weight: 0.20;

[0196] AI evidence tracing capability, reference weight: 0.15;

[0197] AI-suggested clinical translation capabilities, with a reference weight of 0.15;

[0198] Ethical safety and risk governance capabilities, reference weight: 0.15;

[0199] The ability to integrate traditional Chinese medicine characteristics, with a reference weight of 0.10;

[0200] AI model version adaptability, reference weight: 0.10;

[0201] Digital MDT collaborative capabilities, reference weight: 0.05;

[0202] AI cognition and operational capabilities, reference weight: 0.05;

[0203] Continuous learning and improvement ability, reference weight: 0.05.

[0204] In some embodiments, the event completeness weight can be set as follows:

[0205] Necessary event types: AI-suggested events to view, weight of ordinary AI suggestions: 1, weight of high-risk AI suggestions: 1, weight of traditional Chinese medicine AI suggestions: 1;

[0206] Necessary event types: AI evidence tracing events; weight of ordinary AI suggestions: 1; weight of high-risk AI suggestions: 2; weight of traditional Chinese medicine AI suggestions: 1.

[0207] Necessary event type: Manual review of opinions; weight of ordinary AI suggestion: 1; weight of high-risk AI suggestion: 2; weight of TCM-related AI suggestion: 2.

[0208] Necessary event type: Data supplementation event for the four diagnostic methods of traditional Chinese medicine; weight of ordinary AI suggestion: 0; weight of high-risk AI suggestion: 0; weight of traditional Chinese medicine AI suggestion: 2.

[0209] Necessary event type: AI diagnosis result review event, ordinary AI suggestion weight: 0, high-risk AI suggestion weight: 0, traditional Chinese medicine AI suggestion weight: 2;

[0210] Required event types: event recording the reason for the difference; weight of ordinary AI suggestion: 1; weight of high-risk AI suggestion: 1; weight of traditional Chinese medicine AI suggestion: 1.

[0211] Necessary event type: Superior confirmation event, ordinary AI suggestion weight: 0, high-risk AI suggestion weight: 2, traditional Chinese medicine AI suggestion weight: 1;

[0212] Required event types: Model version learning event, ordinary AI suggestion weight: 0, high-risk AI suggestion weight: 1, traditional Chinese medicine AI suggestion weight: 1.

[0213] In some embodiments, the hierarchical weights of the TCM-specific data chain can be set as follows:

[0214] The original data layer for the four diagnostic methods, with a reference weight of 0.25;

[0215] Syndrome element layer, reference weight: 0.20;

[0216] AI-generated analysis results layer, reference weight: 0.15;

[0217] Manual verification and review layer, reference weight: 0.20;

[0218] Traditional Chinese Medicine treatment plan layer, reference weight: 0.15;

[0219] Follow-up feedback layer, reference weight: 0.05.

[0220] The follow-up feedback layer is mainly used for closed-loop evaluation and profile update after consultation. Before, during or during consultation, or in the MDT conclusion formation stage, the system can avoid using the lack of follow-up feedback as the sole reason to prevent AI suggestions from entering the MDT discussion opinions or MDT conclusion candidates.

[0221] In some embodiments, the credibility weight of the AI ​​suggestion processing result can be set as follows:

[0222] Indicator: Completeness of manual review comments Reference weight 0.30;

[0223] Indicator: Completeness of records of reasons for differences Reference weight 0.25;

[0224] Indicator: Degree of support for the final conclusions of the MDT (Multidisciplinary Team) Reference weight 0.25;

[0225] Indicator: Level of support from follow-up feedback Reference weight 0.20.

[0226] When follow-up feedback has not yet been generated or is not applicable to the target AI recommendations, the system can... Marked as not applicable for this evaluation period, and normalize the weights of the remaining indicators involved in the calculation.

[0227] In some embodiments, the impact weights of model version changes can be set as follows:

[0228] Factors affecting model version changes: Changes in output fields Reference weight: 0.15;

[0229] Factors affecting model version changes: Changes in applicable scope Reference weight: 0.20;

[0230] Factors affecting model version changes: Changes in core model logic Reference weight: 0.25;

[0231] Factors affecting model version changes: Changes in risk warning rules Reference weight: 0.20;

[0232] Factors affecting model version changes: Changes in TCM diagnostic rules or syndrome labeling system Reference weight: 0.20.

[0233] The aforementioned weights can be configured based on the type of AI system deployed by the medical institution, the characteristics of the disease, the risk level suggested by the AI, and the MDT management system. When a certain weight corresponds to an indicator that is not applicable in the current evaluation period, the system can normalize the remaining indicators involved in the calculation.

[0234] S143, Read the flow control rule configuration table

[0235] The configuration table of the flow control rules includes at least the rule identifier, rule name, applicable case risk level, applicable AI suggestion type, applicable MDT stage, applicable medical staff AI function permission status, necessary event type, completion threshold, control action when not completed, whether it is mandatory, whether correction is allowed, whether superior confirmation is required, confirmation level, and configuration version number.

[0236] In some embodiments, the flow control rule configuration table may include the following rules:

[0237] Rule Name: Low-Risk AI Suggestion Viewing Rule; Applicable Conditions: Low-risk cases, general AI suggestions; Necessary Event Type: AI suggestion viewing event; Control Action When Not Completed: Prompt to view AI suggestions, restrict submission if not viewed;

[0238] Rule Name: Medium-Risk AI Suggestion Review Rule; Applicable Conditions: Medium-risk cases or medium-risk AI suggestions; Required Event Types: AI suggestion viewing event, risk warning confirmation event, manual review opinion filling event; Control Action When Not Completed: Generate a review task, and restrict access to candidate options when not completed;

[0239] Rule Name: High-Risk AI Recommendation Evidence Tracing Rule; Applicable Conditions: High-risk cases, treatment plan recommendations, medication review, invasive treatment recommendations; Necessary Event Types: AI evidence tracing event, manual review opinion completion event; Control Action When Not Completed: Triggers evidence tracing and manual review tasks, prevents submission when not completed;

[0240] Rule Name: High-Risk AI Suggestion for Superior Confirmation Rule; Applicable Conditions: High-risk cases, critical illnesses, major treatment plan adjustments; Necessary Event Type: Superior confirmation event or MDT facilitator confirmation event; Control Action When Not Completed: Triggers superior confirmation; if not confirmed, prevents entry into the MDT conclusion candidate.

[0241] Rule Name: Basic Usage Status Restriction Rule; Applicable Conditions: Medical personnel are in basic usage status; Necessary Event Type: Independent submission of high-risk AI suggestions is not allowed; Control Action When Incomplete: Disable the high-risk AI suggestion submission interface;

[0242] Rule Name: Restricted Use Status Confirmation Rule; Applicable Conditions: Medical personnel are in a restricted use status and submit high-risk AI suggestions; Necessary Event Type: Superior Confirmation Event; Control Action When Not Completed: Restrict submission when not confirmed;

[0243] Rule Name: TCM AI Data Chain Rule; Applicable Conditions: AI suggestions for TCM auxiliary diagnosis or TCM treatment plan recommendations; Necessary Event Types: Four Diagnostic Methods Data Supplementation Event, AI Diagnosis Result Review Event, Manual Diagnosis Review Opinion Event; Control Action When Not Completed: Generate Four Diagnostic Methods Supplementation or Diagnosis Review Task, and prevent entry into the candidate options when not completed;

[0244] Rule Name: Model Version Re-authorization Rule; Applicable Conditions: Model version change reaches the re-authorization trigger threshold; Required Event Types: Version learning event, field explanation confirmation event, or simulated case assessment event; Control Action When Not Completed: Mark as pending model version re-authorization and restrict high-risk AI functions;

[0245] Rule Name: Rule for Recording Reasons for Differences; Applicable Conditions: The difference between AI suggestions and review opinions or MDT conclusions reaches a threshold; Required Event Type: Event for Recording Reasons for Differences; Control Action When Not Completed: Requires filling in the reason for the difference; if not filled in, submission is restricted or the result is marked as pending correction.

[0246] Rule Name: Unknown Model Version Rule; Applicable Conditions: AI model version is unknown or version description is missing; Required Event Type: Model version confirmation event or superior confirmation event; Control Action When Not Completed: Restrict the submission of high-risk AI suggestions.

[0247] In some embodiments, the "mandatory" field in the flow control rule configuration table is used to distinguish between mandatory control conditions and correctable control conditions; the "correction allowed" field is used to determine whether the system allows medical personnel to resubmit after supplementing, reviewing, learning, or confirmation by a superior; and the "confirmation level" field is used to identify confirmation by a general senior doctor, confirmation by an MDT host, confirmation by a designated expert, or confirmation by an administrator.

[0248] When the same target AI suggestion hits multiple flow control rules at the same time, the system integrates the control conditions according to the strict principle: if multiple rules require correction, review, superior confirmation, model version re-authorization, or restriction on submission, the system will include the corresponding requirements into the flow control condition set at the same time; if there is a conflict between allowing and restricting different rules, the restriction rule will be used as the final rule; if different rules correspond to different confirmation levels, the higher confirmation level will be used as the final confirmation requirement.

[0249] S144. Determine the current valid configuration version.

[0250] The system determines the currently valid configuration version based on the activation time, expiration time, applicable AI suggestion type, applicable case risk level, applicable MDT stage, applicable role type, applicable permission status, and configuration version number.

[0251] When multiple versions of the same configuration item simultaneously meet the applicable conditions, the system determines the currently valid configuration according to the following priority:

[0252] First, prioritize configurations with more specific applicability.

[0253] Second, prioritize configurations that have been enabled more recently;

[0254] Third, prioritize configurations with higher risk levels or stricter control requirements;

[0255] Fourth, prioritize the configurations that medical institutions pre-set with higher priority.

[0256] When generating case risk scores, competency profile scores, flow control condition sets, gating results, closed-loop evaluation results, and permission status update results, the system records the configuration version number called for subsequent auditing, review, and parameter adjustment.

[0257] S150, Perform data masking, standardization, and data quality verification.

[0258] The system anonymizes patient identities, medical staff identities, and case numbers, and converts data from different systems into a unified field format. The anonymized mapping relationships are stored separately by the hospital's authorized system; this system only processes anonymous identifiers and business fields.

[0259] The system standardizes field names, data types, enumeration values, time formats, units of measurement, and status fields from different source systems. For example, it uniformly maps patient IDs from different systems to patient anonymity identifiers, case IDs from different systems to case anonymity identifiers, consultation process numbers from different systems to MDT consultation identifiers, and AI suggestion numbers from different AI systems to AI suggestion identifiers.

[0260] The system can also perform data quality checks on fields. These checks include at least verifying whether necessary fields are missing, whether field types are correct, whether enumerated values ​​fall within a preset range, whether time fields are abnormal, whether the same data is repeated, whether the source system is trustworthy, whether field update times are valid, and whether there are conflicts between key fields.

[0261] When the system detects missing fields, format errors, abnormal enumeration values, abnormal timestamps, inconsistent source systems, or conflicts in key fields, it generates a data quality status marker. This data quality status marker includes at least the abnormal field, abnormal type, source system, discovery time, whether it affects risk scoring, whether it affects event sequence construction, whether it affects profile calculation, whether it affects gating judgment, and a suggested handling method.

[0262] This step outputs a standardized basic data package, which is then provided to subsequent steps: S200 to build an AI-MDT interactive event sequence, S300 to generate a competency profile of medical personnel for AI system applications, S400 to generate AI suggestion flow control conditions, and S500 to execute pre-submission gating judgments for AI suggestions.

[0263] This step provides a unified, traceable, and computable data foundation for subsequent risk scoring, event sequence construction, profile calculation, and gating judgment, avoiding inconsistencies in fields, missing data, chaotic status fields, or inconsistent time formats across different hospital information systems, which could lead to unstable execution of AI-suggested gating judgments.

[0264] S200, Constructing an AI-MDT Interaction Event Sequence

[0265] This step is used to convert the operation logs and business data scattered in the digital MDT platform, AI-assisted diagnosis and treatment system, electronic medical record system, TCM four diagnostic information collection system, training and assessment system, access control system, model management system and follow-up management system into an AI-MDT interaction event sequence that can be invoked in subsequent competency profile calculation, flow control condition generation and pre-submission gating judgment.

[0266] The AI-MDT interaction event sequence uses target medical staff, target cases, target MDT consultations, and target AI suggestions as associated objects to record the actual operational behaviors of medical staff in the stages of AI suggestion generation, viewing, evidence tracing, manual review, supplementary recording of TCM four diagnostic methods, review of AI syndrome differentiation results, recording of reasons for differences, confirmation by superiors, model version learning, and follow-up feedback.

[0267] S210, Collect multi-source interactive data

[0268] The system collects operation logs and business data from the digital MDT platform, AI-assisted diagnosis and treatment system, electronic medical record system, TCM four diagnostic information collection system, training and assessment system, access control system, model management system, and follow-up management system.

[0269] The operation log may include one or any combination of the following: AI suggestion viewing log, AI evidence basis viewing log, manual review opinion submission log, MDT discussion opinion submission log, difference reason filling log, senior doctor confirmation log, permission change log, training completion log, and model version learning log.

[0270] The business data may include one or any combination of case data, AI suggestion content, AI suggestion type, AI model version, MDT consultation stage, TCM four diagnostic methods data, MDT final conclusion, training task data, permission status data, and follow-up feedback data.

[0271] When collecting data from multiple sources, the system can convert timestamps from different source systems into a unified time format and determine the event time according to the priority of hospital server time, source system time, and log reception time. When the event time of a source system is missing, the system can use the log reception time as a substitute event time and mark the event as a substitute time event.

[0272] S220, Generate AI-MDT Interaction Events

[0273] The system converts multi-source operation logs and business data into AI-MDT interactive events.

[0274] In some embodiments, the conversion rules are as follows:

[0275] The AI-assisted diagnosis and treatment system generates target AI suggestion records, which are then converted into AI-MDT interaction events: AI suggestion generation events.

[0276] When medical staff open or view AI-suggested content, the converted AI-MDT interaction event is: AI suggestion viewing event;

[0277] Medical staff review AI evidence, guidelines, model explanations, or risk warnings, resulting in AI-MDT interaction events: AI evidence tracing events.

[0278] Medical staff submit manual review comments; the converted AI-MDT interaction event is the manual review comment filling event.

[0279] Medical staff supplement data related to inspection, auscultation, inquiry, and palpation, which are then converted into AI-MDT interactive events: Traditional Chinese Medicine Four Diagnostic Methods Data Supplementation Events;

[0280] Medical staff confirming, correcting, or rejecting AI diagnostic results; the converted AI-MDT interaction event is called the AI ​​diagnostic result review event.

[0281] Medical staff fill in the reasons for the differences between AI suggestions, review opinions, MDT discussion opinions, or final conclusions. The converted AI-MDT interaction event is the event recording the reasons for the differences.

[0282] Confirmed by a senior physician, MDT facilitator, or designated expert, the converted AI-MDT interaction event is: Senior Confirmation Event.

[0283] After medical staff complete model version learning, field interpretation confirmation, or simulated case assessment, the converted AI-MDT interaction event is: model version learning event;

[0284] The follow-up system returns follow-up outcomes, follow-up indicators, or follow-up evaluations, which are then converted into AI-MDT interactive events: follow-up feedback events.

[0285] The permission system records permission granting, restriction, restoration, or re-authorization, and the converted AI-MDT interaction events are called permission change events.

[0286] Each AI-MDT interaction event includes at least the event identifier, event time, event type, event source system, event status, case anonymity identifier, MDT consultation identifier, and medical staff anonymity identifier.

[0287] When the event is related to a specific AI suggestion, the interaction event also includes an AI suggestion identifier; when the event is related to an AI model version, the interaction event also includes an AI model version identifier; when the event is related to training, permissions, or follow-up, the interaction event also includes a training task identifier, a permission status identifier, or a follow-up record identifier.

[0288] Event status can include one of the following: Completed, Incomplete, Pending Correction, Pending Review, Pending Supervisor Confirmation, Pending Model Version Re-authorization, Pending Follow-up Update, Abnormal, and Withdrawn. Operation results can include one of the following: View Completed, Evidence Tracing Completed, Review Comments Submitted, Reasons for Difference Recorded, Confirmation Passed, Confirmation Failed, Correction Completed, Learning Completed, Assessment Passed or Assessment Failed.

[0289] For multiple log entries from the same medical staff, the same case, the same MDT consultation, the same AI suggestion, and the same event type, with a time difference less than the preset merging window, the system can merge them into a single interactive event to avoid duplicate log entries causing repeated calculations of the event completeness score. The preset merging window can be configured by the medical institution, such as 30 seconds, 1 minute, or 5 minutes.

[0290] S230, Associate based on the same personnel, the same case, the same MDT consultation, and the same AI recommendation.

[0291] The system associates each AI-MDT interaction event with the following criteria: medical staff anonymity identifier, case anonymity identifier, MDT consultation identifier, AI suggestion identifier, and event time.

[0292] If there are multiple AI suggestions in a single MDT case, the system constructs a corresponding sequence of interactive events for each AI suggestion in order to evaluate the medical staff's operational behavior and perform independent gating judgments for different AI suggestions.

[0293] When an AI suggestion identifier exists in the event log, the system prioritizes association based on the AI ​​suggestion identifier. When no AI suggestion identifier exists in the event log but page source, functional module, MDT stage, text tag, or time context exists, the system can perform auxiliary association based on event time, source system, operation page, AI suggestion type, tag matching result, and manual confirmation result.

[0294] If the system cannot determine which target AI suggestion an event belongs to, it marks the event as an event to be associated. Events to be associated are not directly included in the event completeness score of the target AI suggestion, unless the association is completed subsequently through manual confirmation, label matching, or system correction rules.

[0295] When the same event involves multiple AI suggestions, the system can associate them with multiple target AI suggestions according to the AI ​​suggestion identifiers recorded in the event, or make a primary association based on the target AI suggestion selected by the medical staff, and mark other related AI suggestions as associated reference objects.

[0296] S240. Forming a time-ordered sequence of AI-MDT interaction events.

[0297] The system sorts the associated AI-MDT interaction events according to the event time, forming a time-ordered AI-MDT interaction event sequence corresponding to the target AI suggestion.

[0298] In some embodiments, the AI-MDT interaction event sequence may include a sequence identifier, a target medical staff anonymity identifier, a case anonymity identifier, an MDT consultation identifier, an AI suggestion identifier, an AI suggestion type, the current MDT stage, an event list, the number of events, the last update time, and the sequence status.

[0299] The event list is arranged according to event time, and records the event identifier, event type, event time, event status, source system, and operation result for each event.

[0300] When events occur at the same time, the system can prioritize them according to their type. For example, AI suggestion generation events take precedence over AI suggestion viewing events, AI suggestion viewing events take precedence over AI evidence tracing events, AI evidence tracing events take precedence over manual review opinion filling events, and manual review opinion filling events take precedence over superior confirmation events. Model version learning events and permission change events can be inserted into their corresponding positions based on their actual occurrence time.

[0301] The AI-MDT interaction event sequence output in this step is used by S300 to calculate the competency profile of medical staff in the AI ​​system application, by S400 to generate the target AI suggestion flow control conditions, and by S500 to determine whether the necessary interaction events before the target AI suggestion is submitted have been completed.

[0302] This step is used to transform operation logs and business data scattered across different systems into traceable, sortable, correlated, and computable event sequences. This enables the evaluation of medical staff's AI system application competence to no longer rely on subjective questionnaires or single exams, but to be dynamically evaluated based on real AI-MDT operational behaviors. It also ensures that the pre-submission gating judgment of AI suggestions has a repeatable data foundation.

[0303] S300, generating AI system competency profiles for medical personnel

[0304] The system generates a profile of medical staff's AI system application competence based on the target case's risk level, the type of target AI suggestion, the historical profiles of participating medical personnel, training and assessment records, model version learning status, and the AI-MDT interaction event sequence. The case risk level is not the medical staff's competence itself, but rather a scenario correction parameter used to evaluate their AI system application behavior and generate subsequent gating conditions. This parameter is designed to prevent medical staff from having their AI usage behavior in low-risk and high-risk cases evaluated equally.

[0305] S310. Calculate the case risk score.

[0306] The system uses a weighted cumulative method to calculate the case risk score of the target MDT cases:

[0307]

[0308] in, The case risk score represents the target MDT cases; This tool is used to uniformly convert risk factors scattered across different systems into case risk scores, providing case background parameters for evaluating the competence of medical staff in applying AI systems in different risk scenarios, and avoiding the same evaluation of medical staff's AI usage behavior in low-risk and high-risk cases.

[0309] Indicates the first For binary risk factors such as whether the illness is acute or critical, whether it is complex or difficult to diagnose, whether it involves high-risk medications, whether there are serious comorbidities, whether there are contraindications, and whether there are AI high-risk warnings, a value of 1 is assigned if the condition exists and 0 if it does not. For degree-based risk factors such as the proportion of missing case data, the degree of abnormality in examinations and tests, and the degree of difference between traditional Chinese medicine and Western medicine diagnosis and treatment, the values ​​are determined as follows: It can be normalized to a value between 0 and 1 based on the field missing percentage, anomaly level, or number of difference labels. Information can be obtained from electronic medical record systems, digital MDT platforms, AI-assisted diagnosis and treatment systems, examination and testing systems, or supplementary information entered by medical staff. For example, whether it is an acute or critical illness can be obtained from the disease level field, emergency markers, critical illness markers, or the reason for the MDT application; whether it involves high-risk medications can be obtained from the medical order system, pharmaceutical review system, or AI-based medication review system; whether there are differences between traditional Chinese medicine and Western medicine treatments can be obtained from the MDT consultation topic, the structured difference tags between traditional Chinese medicine and Western medicine treatment opinions; whether there is a serious lack of case data can be obtained based on the missing proportion of necessary case fields.

[0310] This represents the risk weight corresponding to the (i)th risk factor; This indicates the number of risk factors included in the calculation. To preset risk weights, medical institutions can configure them based on disease characteristics, AI system type, MDT management requirements, and AI suggestion type. After configuration, the system uniformly calls the currently effective version of the risk weight configuration for calculation within the same evaluation cycle and records the configuration version number. In some embodiments, the weight is 3 for critical and severe cases, 2 for difficult and complex cases, 2 for cases involving invasive treatments, 2 for cases involving high-risk medications, 2 for cases with serious comorbidities or multiple coexisting diseases, 2 for cases with a history of serious adverse reactions or contraindications, 2 for high-risk alerts output by the AI ​​system, 1 for cases with severely missing case data, 1 for cases requiring multidisciplinary decision-making, and 1 for cases with significant differences in diagnosis and treatment between traditional Chinese medicine and Western medicine.

[0311] This step is used to convert the risk background of the target case into numerical parameters that can be called upon for subsequent competency assessment and gating control, so that the AI ​​usage behavior of the same medical staff in different case risk scenarios can be evaluated and gating according to different control standards.

[0312] S320, Determine the risk level of the case

[0313] The system determines the risk level of the target case based on the case risk score:

[0314] When a case's risk score is less than or equal to 1, the system classifies it as low risk;

[0315] When a case's risk score is greater than 1 and less than or equal to 3, the system classifies it as medium risk.

[0316] When a case's risk score is greater than 3 and less than or equal to 5, the system identifies it as high-risk.

[0317] When the case risk score is greater than 5, the system identifies it as a critical or complex case.

[0318] When a field required for a case risk score is missing but does not affect the risk level assessment, the system can calculate the case risk score based on the existing fields and record the missing field as a case data pending correction.

[0319] When key risk fields are missing, causing the system to be unable to rule out high-risk, critical, or complex cases, the system will temporarily classify the target case as medium-risk or high-risk according to the principle of strict risk assessment, and generate a case data correction task.

[0320] For AI suggestions related to treatment plans, high-risk medication, invasive treatments, TCM-assisted diagnosis, or TCM treatment plans, if key case fields are missing, the system will preferentially classify the target case as high-risk and add requirements for manual review, evidence tracing, superior confirmation, or case data supplementation when generating flow control conditions in S400. After the case data is supplemented, the system will recalculate the case risk score and case risk level, and update the flow control condition set for the corresponding target AI suggestion.

[0321] This step is used to convert the case risk score into a risk level that can be called by subsequent S400, avoiding the incorrect use of low-risk gating rules due to missing case fields.

[0322] S330, Read or generate an initial profile of medical personnel's competence in AI system applications.

[0323] The system reads or generates an initial profile of the AI ​​system application competence of participating medical personnel:

[0324] If a historical profile exists, the system will read the medical staff’s historical profile, historical ability dimension score, historical AI function permission status, historical risk events, training completion status and model version learning status from the previous evaluation cycle or the current valid evaluation cycle.

[0325] If no historical profile exists, the system generates an initial profile based on the medical staff's job role, professional title, specialty, authorized AI function scope, completed training records, initial simulated case assessment results, and the medical institution's preset initial rules, and sets the initial AI function permissions.

[0326] The initial profile may include at least the medical staff's anonymous identifier, job role, professional title, specialty, initial comprehensive competence score, initial ability dimension score, initial permission status, scope of AI functions available, whether superior confirmation is required, whether basic training is required, and the profile generation time.

[0327] In some embodiments, if medical personnel have completed basic AI training and passed the initial simulated case assessment, the system can set their initial permission status to restricted use or regular authorization status; if medical personnel have not completed basic AI training, have not passed the initial simulated case assessment, or whose job role does not have the authority to independently submit AI suggestions, the system sets their initial permission status to basic use status, allowing them only to view AI suggestions, conduct learning-based reviews, or fill in non-submission-related review comments.

[0328] If medical staff have already been authorized to perform their duties but lack AI system usage records, the system can generate a default initial score based on the job role and training completion status. If job authorization, training records, and simulation assessment results are all missing, the system will generate a basic usage status according to the principle of strict risk assessment, and generate training or simulation case assessment tasks.

[0329] This step is used to generate executable initial permission status and initial competency profiles for medical staff who do not have historical profiles, so that newly hired staff, staff newly connected to departments, or medical staff using the AI ​​system for the first time can also enter the subsequent S340 competency dimension scoring, S350 comprehensive scoring, and S400 gating control condition generation process.

[0330] S340, Calculate scores for each ability dimension

[0331] The system divides the AI ​​system application competency profile of medical staff into multiple capability dimensions, and obtains scores for each capability dimension from AI-MDT interaction event sequences, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results.

[0332] In some embodiments, the sources and methods for calculating the capability dimension are as follows:

[0333] Capability Dimension: AI suggestion review capability; Data Source: Human review opinions, review time, review reasons, and superior confirmation results; Calculation Method: Number of complete review opinions divided by the number of AI suggestions that should be reviewed, and adjusted in conjunction with the timeliness of review;

[0334] Capability Dimension: AI evidence tracing capability; Data Source: AI evidence viewing events, evidence tracing completion events; Calculation Method: Number of completed evidence tracing events divided by the number of AI suggestions for tracing.

[0335] Capability Dimension: Clinical translation capability of AI suggestions; Data Sources: AI suggestions, review opinions, MDT conclusions, follow-up feedback; Calculation Method: Number of suggestions that have been reasonably adopted or corrected divided by the number of AI suggestions processed;

[0336] Capability Dimension: Ethical security and risk governance capabilities; Data Sources: Access control audits, unauthorized access, high-risk unreviewed activities, and abnormal use of sensitive data; Calculation Method: A base score of 100 points is used, with deductions based on the type, frequency, and severity of risk events.

[0337] Capability Dimension: Integration capability with TCM characteristics; Data Sources: Supplementation of the four diagnostic methods, syndrome correction, AI-based syndrome differentiation and verification, and association with TCM treatment plans; Calculation Method: The number of necessary nodes in the TCM characteristic data chain divided by the number of nodes to be completed;

[0338] Capability dimension: AI model version adaptability; Data source: version learning, field interpretation and confirmation, simulated case assessment; Calculation method: number of completed version learning or assessments divided by the number of version learning or assessments to be completed;

[0339] Capability Dimension: Digital MDT Collaboration Capability; Data Sources: MDT discussion opinions submitted, superior confirmation, interdisciplinary collaboration events; Calculation Method: Number of collaborative tasks completed on time divided by the number of collaborative tasks that should be completed;

[0340] Capability Dimension: AI cognition and operational ability; Data Source: AI function usage, viewing, confirmation, and basic operation training; Calculation Method: Number of compliant uses divided by total number of uses, adjusted based on training results;

[0341] Capability dimension: Continuous learning and improvement capabilities; Data sources: Training completion, changes in similar risk events, completion of debriefing tasks; Calculation method: Number of completed training and debriefing tasks divided by the number of tasks to be completed.

[0342] In the table, "divide by" indicates that the ratio is calculated using the previous quantity as the numerator and the next quantity as the denominator. When the corresponding denominator is 0, the system can mark that capability dimension as not applicable in the current evaluation period, or use the score from the previous evaluation period, the initial default score, or exclude it from the comprehensive score of the current period according to the medical institution's preset rules. For AI-suggested review capabilities, the system first calculates the complete review rate, and then generates a timeliness correction coefficient based on the review timeliness rate to avoid situations where only a formal review is completed but not completed within the time required by the MDT process being evaluated as equivalent. For ethical safety and risk governance capabilities, the system uses 100 points as the base score, and deducts points based on the type of risk event, the number of occurrences, and the severity coefficient, with no single item score lower than 0 points.

[0343] S341. Calculate the AI ​​suggestion review capability score

[0344] The system calculates the AI-suggested review capability score using a combination of completeness rate as the primary factor and timeliness rate as a correction factor. This means it first determines whether medical staff have completed the required reviews, and then adjusts the score based on whether the reviews were completed within a preset timeframe.

[0345]

[0346] in:

[0347] ; ;

[0348] in, This indicates the AI's ability to provide review and assessment scores. This indicates the number of AI suggestions that should be manually reviewed according to the gating rules within the evaluation period; This indicates the number of AI suggestions for which complete human review comments have been submitted within the evaluation period; This indicates the number of AI suggestions that were reviewed within the preset time limit; Indicates the completeness rate of the review; Indicates the timeliness of review; This represents a correction factor for the timeliness of the review, ranging from 0.8 to 1.0. It is applied when all complete review comments are completed within the preset timeframe. If all complete review comments are not completed within the preset time limit, That is, 20% is deducted from the completeness rate.

[0349] The complete manual review opinion refers to a review record that includes at least one or any combination of the following fields: review conclusion, review reason, whether the AI ​​suggestion was adopted, whether the AI ​​suggestion needs to be corrected, evidence review record when necessary, risk warning confirmation result, and review timestamp. In high-risk AI suggestion scenarios, the complete manual review opinion may also include the superior's confirmation status or contraindication verification result.

[0350] when When the AI-suggested review capability dimension is marked as not applicable in this evaluation cycle, or the score of the previous evaluation cycle, the initial default score, or the score not included in the comprehensive score of this cycle will be adopted according to the preset rules.

[0351] when and At that time, the system will Take 0, and Set the value to 0; no further calculations will be performed at this point. .

[0352] The preset time limit can be configured by medical institutions according to the type of AI suggestion, the risk level of the case, and the stage of the MDT process; when the same AI suggestion is subject to multiple preset time limits at the same time, the system uses the earliest time limit as the basis for judging whether the review is completed in a timely manner.

[0353] In some embodiments, the criteria for timely review are as follows:

[0354] For typical AI suggestion scenarios, the standard for timely review is to complete the review within 24 hours after the AI ​​suggestion is generated.

[0355] In the MDT (Multi-Level Design) pre-meeting material preparation phase, the timely review standard is to complete the review before the start of the MDT meeting.

[0356] For high-risk AI treatment recommendations, the standard for timely review is to complete the review before submitting the MDT (Multidisciplinary Team) discussion opinion.

[0357] In cases of critical illness, severe illness, or complex and difficult cases, the standard for timely review is to complete the review before confirmation by the superior.

[0358] In scenarios where TCM-assisted diagnosis is supported by AI suggestions, the standard for timely review is to complete the diagnosis review before the AI ​​diagnosis results enter the candidate options.

[0359] Assuming that the number of AI suggestions to be reviewed within a certain evaluation period is... Number of complete review comments The number of reviews completed in a timely manner ,but:

[0360] ;

[0361] ;

[0362] The AI's suggested review capability score is 76.

[0363] S342, Scoring of Ethical Security and Risk Governance Capabilities

[0364] Ethical safety and risk governance capabilities can be calculated on a 100-point scale:

[0365] ;

[0366] in: This indicates a score for ethical safety and risk governance capabilities; Indicates the number of risk event types; Indicates the first The number of times a risk event of this type occurred during the evaluation period; Indicates the first The actual deduction of points for a single risk event; Indicates the first The basic deduction score for risk-related events; Indicates the first Severity coefficient of risk events; This indicates that the minimum score is not lower than 0. The severity coefficient can be set as follows: General 1.0, Slightly Severe 1.5, Severe 2.0, Major 3.0.

[0367] In some embodiments, the basic deduction values ​​for risk events are as follows:

[0368] High-risk AI suggestions not reviewed (5 points); AI suggestions submitted without AI evidence tracing (3 points); attempting to submit high-risk AI suggestions without proper authorization (8 points); using related AI functions after the AI ​​model version has been updated but not yet completed learning (5 points); unauthorized access to unauthorized cases or unauthorized AI suggestions (10 points); abnormal use or export of sensitive data (15 points); initiating another AI suggestion submission request that does not meet the flow control conditions after being blocked by gating (10 points).

[0369] If a medical staff member commits the following acts during the evaluation period: 1 instance of failing to review a high-risk AI suggestion (base deduction of 5 points, severity coefficient 2.0); 2 instances of submitting AI suggestions without completing evidence tracing (base deduction of 3 points each time, severity coefficient 1.0); 1 instance of using related functions without completing model version learning (base deduction of 5 points, severity coefficient 1.5). Then:

[0370]

[0371]

[0372] The ethical safety and risk governance capabilities score was 76.5.

[0373] It should be noted that if the comprehensive competency scoring formula includes an additional deduction item for high-risk events, this additional deduction item should preferably only be used for major red-line events such as serious overreach of authority, abnormal export of sensitive data, and circumvention of gate control, so as to avoid duplicate deduction for the same ordinary risk event as the ethical security and risk governance capability scoring in this step.

[0374] S350, Calculate the comprehensive competence score

[0375] The system generates a comprehensive competency score for medical staff in applying the AI ​​system based on scores across various competency dimensions. To avoid the same risk event being deducted repeatedly in both the individual competency dimension score and the comprehensive score, the system prioritizes scoring the corresponding event separately in each competency dimension before weighting and summing the scores for each competency dimension. For serious red-line events such as severe unauthorized access, abnormal export of sensitive data, tampering with or deletion of AI-MDT interaction event records, and circumventing gate controls, the system can set additional deduction items or directly trigger permission restrictions.

[0376] S351, Weighted Calculation of Comprehensive Competency Score

[0377] The system calculates the comprehensive competence score of medical staff in applying the AI ​​system according to the following formula:

[0378]

[0379] in, Indicates medical staff During the evaluation period The AI ​​system within the system applies a comprehensive competency score, which is used to transform the actual AI usage behavior of medical staff into competency profiles that can be used for access control. This transforms the traditional post-event evaluation results into dynamic control parameters in the AI ​​suggestion flow process, thereby enabling the viewing, review, submission, superior confirmation, model version re-authorization, and opening of high-risk AI functions of AI suggestions to be dynamically adjusted based on the actual operational behavior of medical staff.

[0380] Indicates medical staff During the evaluation period Inner Scores for each ability dimension The data is obtained from medical staff's interactions with AI-MDT during the evaluation period, including the sequence of AI-MDT interactions, access control records, training and assessment records, model version learning records, TCM-specific data chain correction records, and the closed-loop processing results of AI suggestions. For example, AI evidence tracing capability can be determined based on the proportion of evidence actually traced by medical staff in AI suggestions requiring evidence tracing; AI suggestion review capability can be determined based on the completeness rate of manual review opinions, the timeliness rate of review, and the reasonableness of review results; TCM-specific integration capability can be determined based on the completion rate of supplementary data from the four diagnostic methods, the review rate of AI syndrome differentiation results, the completeness rate of syndrome correction basis, and the correlation with TCM treatment plans; ethical safety and risk governance capability can be determined after deducting factors such as the type, frequency, and severity coefficient of risk events.

[0381] Indicates the evaluation period The set of capability dimensions involved in the comprehensive score calculation;

[0382] Indicates the first Normalized weights for each dimension of computing power involved;

[0383] Indicates the evaluation period Additional deductions arising from major red-line events;

[0384] This indicates that the overall score must be no lower than 0.

[0385] This indicates that the overall score is no higher than 100 points.

[0386] When all capability dimensions participate in the scoring for this evaluation cycle .

[0387] When some capability dimensions are excluded from the comprehensive score because the denominator is 0 or they are not applicable in the current evaluation period, the system normalizes the weights of the remaining capability dimensions involved in the calculation.

[0388]

[0389] in, Indicates the first Preset weights for each capability dimension; This represents the sum of the preset weights of all the capability dimensions involved in the calculation within this evaluation period.

[0390] In some embodiments, the capability dimensions and their preset weights are as follows: AI suggestion review capability 0.20, AI evidence traceability capability 0.15, AI suggestion clinical translation capability 0.15, ethical safety and risk governance capability 0.15, integration capability with traditional Chinese medicine characteristics 0.10, AI model version adaptation capability 0.10, digital MDT collaboration capability 0.05, AI cognition and operation capability 0.05, and continuous learning and improvement capability 0.05. These weights can be configured by medical institutions based on the AI ​​system type, MDT management system, job roles, case risk type, and evaluation cycle. Within the same evaluation cycle, the system calls the currently valid version of the weight configuration table for calculation and records the weight configuration version number.

[0391] S352, Calculation of Additional Deductions for Major Red Line Events

[0392] Additional deductions for major red-line events can be calculated using the following formula:

[0393]

[0394] in, Indicates the evaluation period Additional deductions arising from major red-line events; Indicates the number of major red-line event types; Indicates the first Major red line events during the evaluation cycle The number of times it occurs within a period of time; Indicates the first Additional deductions for major red-line events.

[0395] In some embodiments, major red-line events include, but are not limited to: unauthorized access to unauthorized medical records or unauthorized AI suggestions that exposes sensitive information, abnormal export or transmission of sensitive data, tampering with or deleting AI-MDT interaction event records, circumventing the AI ​​suggestion gating judgment process, submitting AI suggestion requests that do not meet the flow control conditions after being blocked by gating for a preset number of times, continuously calling high-risk AI functions without obtaining the corresponding permissions, and other major AI use security events identified by medical institutions.

[0396] In some embodiments, the additional point deductions for major red-line events are as follows:

[0397] Unauthorized access to unauthorized medical records or unauthorized AI suggestions that exposes sensitive information will result in an additional deduction of 10 points.

[0398] For abnormal export or transmission of sensitive data, an additional deduction of 15 points will be made.

[0399] Tampering with, deleting, or circumventing AI-MDT interaction event records will result in an additional deduction of 20 points.

[0400] If an AI suggestion submission request that does not meet the flow control conditions is initiated again after being blocked by the gate control, and the number of such requests reaches the preset number, an additional deduction of 10 points will be made.

[0401] Continuously using high-risk AI functions without obtaining the appropriate permissions will result in an additional deduction of 15 points.

[0402] Other major AI use safety incidents identified by medical institutions will result in an additional deduction of 10 to 30 points.

[0403] It should be noted that if a risk event has already been deducted as a common risk event in the S342 ethical safety and risk governance capability score, then in the calculation... The same event will not be deducted repeatedly; only when the event reaches the critical event level preset by the medical institution will it be included. Calculate or trigger mandatory permission restrictions.

[0404] For particularly serious red-line events, the system can adjust the AI ​​function permission status of medical staff to a restricted high-risk AI function status, a model version re-authorization status, or a status requiring confirmation from a senior doctor, without waiting for the comprehensive competency score calculation result, and write the processing result into the AI-MDT interaction event sequence and permission audit record.

[0405] S353, Outputting Comprehensive Competency Score and Access Control Basis

[0406] The system scores based on overall competence. Additional deductions for major red-line events The system generates a competency profile of medical staff for AI system applications by analyzing scores across various competency dimensions and current permission status. This competency profile is used for generating subsequent AI suggestion flow control conditions, gate control judgment before AI suggestion submission, superior confirmation triggering, model version re-authorization, and adjustment of permissions for high-risk AI functions.

[0407] When the overall competence score is low, or when a specific competence dimension falls below the preset threshold of a medical institution, the system can add corresponding gating control conditions. For example, when the AI ​​evidence traceability capability is below the threshold, the system adds requirements for viewing and confirming AI evidence; when the AI ​​suggestion review capability is below the threshold, the system adds requirements for verifying the completeness of manual review opinions or for confirmation by higher authorities; when the integration capability of traditional Chinese medicine characteristics is below the threshold, the system adds requirements for supplementing data from the four diagnostic methods of traditional Chinese medicine and for reviewing AI syndrome differentiation; when the AI ​​model version adaptability is below the threshold, the system adds requirements for model version learning, field interpretation confirmation, or simulated case assessment.

[0408] The S350's AI system application performance across multiple capability dimensions is summarized into a comprehensive competency profile that can be used for subsequent gating control. This profile unifies multi-dimensional data such as AI suggestion review capability, evidence traceability capability, integration capability with traditional Chinese medicine characteristics, model version adaptability, and ethical safety and risk governance capability into access control parameters. This allows the AI ​​system application competency evaluation results of medical personnel to directly affect subsequent access control for viewing, reviewing, submitting, receiving confirmation from superiors, re-authorizing models, and using high-risk AI functions. At the same time, it avoids the same risk event being repeatedly deducted in individual capability dimensions and comprehensive scores.

[0409] S360, Determine the AI ​​function permission status of medical staff

[0410] When the overall competence score is below 60, the system marks it as a basic usage state.

[0411] When the overall competence score is between 60 and 74, the system marks it as a restricted usage status;

[0412] When the overall competence score is between 75 and 89, the system marks it as a regular authorization status;

[0413] When the overall competence score is 90 or above and there are no high-risk unreviewed events, the system marks it as an advanced authorization status;

[0414] This approach transforms case risk and personnel profiles into input parameters for subsequent gating control, enabling AI-suggested submission permissions to be dynamically adjusted based on case risk and actual usage behavior of medical staff.

[0415] S400, AI suggestion flow control conditions based on competency profile generation

[0416] Based on the AI ​​function permission status of medical personnel determined by S360, and combined with the case risk level, AI suggestion type, current MDT stage, AI model version status, and TCM characteristic data chain status, the system generates the flow control conditions that the target AI suggestion should meet before entering the MDT discussion opinions or MDT conclusion candidates.

[0417] The purpose of this step is to transform the competency profile of medical staff in the AI ​​system into control conditions that can be called by subsequent gating modules, so that the competency evaluation results are no longer limited to static scoring, but can directly affect AI suggestion viewing, evidence tracing, manual review, superior confirmation, model version re-authorization, and submission interface control.

[0418] S410. Generate basic control conditions based on case risk levels.

[0419] The system generates basic control conditions based on the risk level of the target case:

[0420] When the target case is a low-risk case, the system requires medical staff to at least review the AI ​​recommendations and record the AI ​​recommendation review events;

[0421] When the target case is a medium-risk case, the system requires medical staff to review the AI ​​suggestions, confirm the AI ​​confidence level or risk warning, and fill in the manual review comments;

[0422] When the target case is a high-risk case, the system requires medical staff to complete AI evidence tracing, manual review comments, and confirmation by a senior doctor;

[0423] When the target case is a critical or complex case, the system, in addition to the high-risk case control conditions, further requires confirmation from the MDT host, designated expert, or senior medical personnel with corresponding authority.

[0424] When the required fields for a case's risk level are missing, preventing the system from accurately determining the risk level, the system can temporarily classify the case as medium or high risk according to the principle of strict risk assessment, and generate a case data correction task.

[0425] S420. Adjust control conditions according to AI suggestion type.

[0426] The system adjusts the basic control conditions based on the type of target AI suggestion:

[0427] For AI suggestions that include medical record summaries or low-risk information prompts, the system can require the AI ​​to only view the suggested events and perform necessary manual confirmation.

[0428] For AI suggestions related to auxiliary diagnosis, medication review, treatment plan recommendation, and surgery or invasive treatment, the system adds requirements for AI evidence tracing, contraindication verification, manual review comments, and superior confirmation.

[0429] For AI suggestions such as high-risk medication, major treatment plan adjustments, invasive procedures, and critical care management, the system sets confirmation from superiors or MDT moderators as a mandatory control condition.

[0430] For AI suggestions that assist in TCM diagnosis or recommend TCM treatment plans, the system adds a requirement to verify the integrity of the TCM-specific data chain. The TCM-specific data chain includes at least one or any combination of the following: original data from the four diagnostic methods, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plan association status, and follow-up feedback status.

[0431] S430. Adjust control conditions based on the competency profile of medical personnel using the AI ​​system.

[0432] The system adjusts control conditions based on the medical staff's AI system application competency profile and permission status:

[0433] When medical staff are in basic usage mode, the system can only allow them to view AI suggestions, conduct learning-based reviews, or fill in non-submission-type review opinions, and does not allow them to independently submit high-risk AI suggestions as MDT discussion opinions or MDT conclusion candidates.

[0434] When medical staff are in a restricted usage state, the system allows them to view AI suggestions and fill in manual review opinions. However, for high-risk AI suggestions, treatment plan recommendation AI suggestions, TCM auxiliary diagnosis AI suggestions, or AI suggestions after the model version has changed, the system requires confirmation from a senior doctor before proceeding to the subsequent gating judgment.

[0435] When medical staff are in a regular authorized state, the system allows them to submit ordinary AI suggestions or some high-risk AI suggestions after completing the viewing of AI suggestions, evidence tracing, manual review and recording of reasons for differences under the corresponding risk level, but they still need to meet the necessary control conditions determined by S410 and S420.

[0436] When medical staff are in a high-level authorized state, the system allows them to undertake tasks such as reviewing, submitting, or confirming higher-level AI suggestions, provided that the necessary gating conditions are met; however, for critical and severe cases, difficult and complex cases, major treatment plan recommendations, or major changes in model versions, the system can still require confirmation from the MDT host, designated experts, or personnel with higher-level permissions.

[0437] When medical staff’s AI evidence tracing ability is below a preset threshold, the system adds requirements for viewing AI evidence, confirming the source of evidence, or completing evidence tracing.

[0438] When the AI-suggested review capability of medical staff is lower than the preset threshold, the system adds checks on the completeness of human review opinions, the timeliness of review, or the requirement for confirmation from superiors.

[0439] When the target AI suggestion belongs to the category of AI suggestion for TCM-assisted syndrome differentiation or AI suggestion for TCM treatment plan recommendation, and the medical staff's ability to integrate TCM characteristics is lower than the preset threshold, the system will add requirements for supplementing the four diagnostic methods data, confirming syndrome elements, manually reviewing syndrome differentiation, or associating with TCM treatment plans.

[0440] When the AI ​​model version adaptability of medical staff is lower than the preset threshold, the system adds requirements for model version learning, field interpretation and confirmation, or simulated case assessment.

[0441] S440. Adjust control conditions according to AI model version status.

[0442] The system adjusts the flow control conditions suggested by the target AI based on the AI ​​model version status:

[0443] When the AI ​​model version does not change substantially, the system generates routine control conditions based on the case risk level, AI suggestion type, and medical staff's AI function permission status.

[0444] When the AI ​​model version undergoes changes in output fields, scope of application, risk warning rules, model logic, or TCM syndrome differentiation labeling system, the system adds requirements for version learning, field explanation confirmation, simulated case assessment, or model version re-authorization.

[0445] When the impact of AI model version changes on the score reaches a preset threshold, the system can restrict medical staff from independently submitting relevant AI suggestions in high-risk cases and require them to complete version learning or simulated case assessment before restoring their corresponding permissions.

[0446] When the AI ​​model version status is unknown, or the system has not obtained the model version change description, the system will mark the relevant AI suggestions as requiring review. For high-risk AI suggestions, treatment plan recommendation AI suggestions, or TCM auxiliary diagnosis AI suggestions, the system will add requirements for superior confirmation, model version confirmation, or restrict submission.

[0447] When the target AI suggestion belongs to the category of AI suggestion for TCM auxiliary diagnosis or AI suggestion for TCM treatment plan recommendation, the system will include the completeness of the original data of the four diagnostic methods, the completeness of the syndrome elements, the completeness of the AI ​​diagnosis results, the completeness of the manual diagnosis review opinions, the association status of the TCM treatment plan, and the follow-up feedback status in the usage control conditions.

[0448] S450. Adjust control conditions based on the status of the data chain characteristic of traditional Chinese medicine.

[0449] When the target AI suggestion belongs to the category of AI suggestion for TCM auxiliary diagnosis or AI suggestion for TCM treatment plan recommendation, the system adjusts the control conditions according to the status of the TCM characteristic data chain; the system incorporates the integrity of the original data of the four diagnostic methods, the integrity of the syndrome elements, the integrity of the AI ​​diagnosis results, the integrity of the manual diagnosis review opinions, the association status of the TCM treatment plan, and the follow-up feedback status into the use control conditions.

[0450] Before the consultation, the system requires that the original data of the four diagnostic methods, syndrome elements and AI diagnosis results meet the preset completeness requirements; during the consultation, the system requires that the human diagnosis review opinions be complete; in the MDT conclusion formation stage, the system requires that the TCM treatment plan be related to the final syndrome judgment; after the consultation, the system can use the follow-up feedback status as the basis for subsequent profile updates and closed-loop evaluation.

[0451] When the integrity of the TCM characteristic data chain does not reach the corresponding threshold of the current MDT stage, the system generates tasks such as supplementing the four diagnostic methods data, supplementing syndrome elements, manually reviewing syndrome differentiation, or associating TCM treatment plans. Before the correction is completed, the corresponding AI syndrome differentiation suggestions are prevented from entering the MDT discussion opinions or MDT conclusion candidates.

[0452] S460, Generate a set of flow control conditions suggested by the target AI.

[0453] The system integrates the control conditions generated in S410 to S450 to form a set of flow control conditions corresponding to the target AI suggestion; the set of flow control conditions includes at least mandatory control conditions, correctable control conditions, superior confirmation conditions, model version re-authorization conditions, and permission level conditions.

[0454] In some embodiments, each flow control condition includes at least a condition identifier, condition source, applicable case risk level, applicable AI suggestion type, applicable MDT stage, applicable medical staff AI function permission status, necessary event type, completion threshold, control action when not completed, whether correction is allowed, whether superior confirmation is required, and corresponding record fields.

[0455] Among them, the source of the condition is used to identify that the condition originates from the case risk level, AI suggestion type, medical staff competency profile, AI model version status, or TCM characteristic data chain status; the necessary event type can include AI suggestion viewing event, AI evidence tracing event, manual review opinion filling event, TCM four diagnostic data supplementation event, AI syndrome differentiation result review event, difference reason recording event, superior confirmation event, model version learning event, or simulated case assessment event; the control action when it is not completed can include prompting, correction, restricting submission, triggering superior confirmation, triggering model version re-authorization, or preventing entry into MDT discussion opinions or MDT conclusion candidates.

[0456] When multiple control conditions apply to the same target AI suggestion, the system adopts a strict principle to integrate the control conditions. If multiple control conditions require correction, review, superior confirmation, version re-authorization, or restriction on submission, the system will include the corresponding requirements into the set of flow control conditions simultaneously. If there is a conflict between release and restriction between different control conditions, the restriction condition will be used as the final control condition. If different control conditions correspond to different confirmation levels, the higher confirmation level will be used as the final confirmation requirement.

[0457] S400 is used to unify competency profiles, case risks, AI suggestion types, model version status, and TCM-specific data chain status into flow control conditions before AI suggestion submission. This enables medical staff to directly participate in AI suggestion flow control based on their AI system application competency evaluation results. It achieves dynamic gating based on risk, role, AI suggestion type, model version status, and TCM-specific data chain status, allowing S500 to subsequently perform release, blocking, correction, review, superior confirmation, or re-authorization judgments based on a clear set of control conditions.

[0458] S500, Gating before submission of AI suggestions based on competency assessment results

[0459] This step is used to make a system-level judgment on the AI ​​function permission status of medical staff, the completion status of necessary interaction events, and the completeness of events based on the set of flow control conditions generated by S400 before the target AI suggestion enters the MDT discussion opinions or MDT conclusion candidates, and outputs control actions such as release, block, correction, review, superior confirmation, or reauthorization accordingly.

[0460] S510, Receive AI suggestion submission request

[0461] When medical staff intend to submit a target AI suggestion as an MDT discussion opinion or an MDT conclusion candidate, the system receives a submission request. This submission request can originate from the individual consultation opinion submission interface, MDT discussion opinion submission interface, MDT conclusion candidate submission interface, or high-risk AI suggestion adoption interface within the digital MDT platform.

[0462] S520, Read the interactive event sequence and flow control condition set

[0463] The system reads the AI-MDT interaction event sequence corresponding to the target AI suggestion, the current AI system application competence profile and permission status of the target medical staff, and reads the set of target AI suggestion flow control conditions generated by S400.

[0464] The set of flow control conditions is used to determine the necessary event types, mandatory control conditions, event integrity thresholds, superior confirmation conditions, model version re-authorization conditions, permission level conditions, and control actions when conditions are not met for this gating judgment.

[0465] S530. Determine whether the mandatory control conditions have been met.

[0466] The system determines whether a corresponding completion event exists in the AI-MDT interaction event sequence based on the necessary event types recorded in the target AI suggestion flow control condition set. The necessary event types may include one or any combination of the following: AI suggestion viewing event, AI evidence tracing event, manual review opinion filling event, superior confirmation event, model version learning event, TCM four diagnostic data supplementation event, AI diagnosis result review event, and difference reason recording event.

[0467] For necessary events marked as mandatory control conditions, if there is no corresponding completion event in the event sequence, the system directly determines that the mandatory control condition has not been completed.

[0468] S540, Calculate the completeness score of interactive events

[0469]

[0470] in, AI suggestions for the target The completeness score of interactive events This is used to quantify the necessary actions taken by medical staff before AI-suggested MDT discussion opinions or MDT conclusion candidates are included. It transforms the actual operational behavior of medical staff into gating judgment criteria, avoids relying solely on manual judgment, and ensures that competency gating evaluation has a repeatable rule basis.

[0471] This indicates whether the (k)th necessary event has been completed; it is set to 1 if completed and 0 if not completed. The value is obtained from the AI-MDT interaction event sequence: if the system finds the corresponding event in the event sequence, it takes 1; if the corresponding event is not found, it takes 0.

[0472] Indicates the first The weight of each necessary event; Pre-configured based on the current case risk level, AI suggestion type, and MDT stage. For example, for high-risk treatment recommendations, the weight of AI evidence tracing events, manual review opinion completion events, and senior physician confirmation events can be higher than that of ordinary AI suggestion viewing events; for TCM-assisted diagnosis suggestions, TCM four diagnostic data supplementation events and AI diagnosis result review events can be assigned higher weights.

[0473] This indicates the number of necessary events that the AI ​​recommendation needs to complete under the current case risk level, current MDT stage, current AI recommendation type, and current medical staff competency profile. In some embodiments, the event completeness threshold is 0.60 for low-risk cases, 0.75 for medium-risk cases, 0.90 for high-risk cases, and 1.00 for critical, severe, or complex cases.

[0474] When the target AI suggests that there are no necessary events in the set of flow control conditions that require the calculation of completeness, the system can generate gating results based solely on the completion status of permission level conditions and mandatory control conditions; when there are necessary events but the total weight is 0, the system will mark the event completeness score as uncalculation and trigger manual review or superior confirmation in accordance with the principle of strict risk management.

[0475] S550, Generate Gated Results

[0476]

[0477] in, This represents the gating result of the target AI suggestion(s); This indicates that the target AI's suggestions are allowed to be included in the MDT discussion or as candidates for the MDT conclusion; This indicates that access is not allowed and triggers a correction prompt, review reminder, superior confirmation request, training task push, model version re-authorization, or permission restriction.

[0478] Whether the AI ​​function permission status of the medical staff meets the requirements is determined by the permission level conditions generated by S400. If the target medical staff is in the basic use status or the restricted use status, and the target AI suggestion is a high-risk AI suggestion, a treatment plan recommendation AI suggestion or a TCM auxiliary diagnosis AI suggestion, the system can determine that it does not meet the independent submission permission and requires confirmation from the superior before proceeding to the next judgment.

[0479] S560, Output Gating Control Action

[0480] If the gating result is "allow", the digital MDT platform will open the target AI suggestion submission interface;

[0481] If the gating result is restricted submission, the digital MDT platform will hide or disable the target AI suggestion submission interface.

[0482] If the gating result is correction, review, superior confirmation, training, or reauthorization, the system generates the corresponding task and marks the target AI suggestion as pending correction, pending review, pending superior confirmation, pending training, or pending model version reauthorization.

[0483] S570, Record gate control results

[0484] The system writes the gating results, incomplete control conditions, triggered control actions, operation time, operator anonymity identifier, target AI suggestion identifier, permission status, and corresponding threshold configuration version into the event sequence for subsequent auditing and profile updates.

[0485] The S500 can perform system-level release or interception before AI suggestions enter the MDT discussion opinions or MDT conclusion candidates, and transform the medical staff's AI system application competence evaluation results into actual interface control actions to prevent AI suggestions from directly entering the MDT conclusion candidates without necessary viewing, tracing, review, confirmation or re-authorization.

[0486] S600, Evaluation of Competency in Implementing AI Applications with Traditional Chinese Medicine Characteristics

[0487] For AI suggestions related to TCM-assisted syndrome differentiation or TCM treatment plan recommendations, this step is invoked when the system generates flow control conditions or performs pre-submission gating judgments to obtain a TCM-specific data chain integrity score and corresponding correction tasks. This step can serve as a specific evaluation process when S400 generates TCM-related flow control conditions, or it can be invoked when S500 performs gating judgments; the step number does not require it to be executed after S500.

[0488] S610, Identifying AI suggestions related to Traditional Chinese Medicine

[0489] The system determines whether the target AI suggestion belongs to the category of AI suggestions for TCM-assisted diagnosis or TCM treatment plan recommendations. If the target AI suggestion does not belong to the above-mentioned TCM-related AI suggestions, the system may not invoke the TCM-specific data chain integrity verification process, or may only use TCM data as non-mandatory reference information.

[0490] S620, Building a Data Chain with Traditional Chinese Medicine Characteristics

[0491] The system constructs a data chain consisting of original data from the four diagnostic methods, syndrome elements, AI-based syndrome differentiation results, human syndrome differentiation review opinions, TCM treatment plans, and follow-up feedback.

[0492] The raw data from the four diagnostic methods can include observation, auscultation and olfaction, inquiry, and palpation. Observation data can include tongue image, tongue body, tongue coating, complexion, body shape, and demeanor; auscultation and olfaction data can include sounds, cough sounds, odors, breathing sounds, and speech patterns; inquiry data can include chills and fever, sweating, diet, sleep, bowel movements, pain, past medical history, present medical history, and emotional state; palpation data can include pulse location, pulse rate, pulse strength, pulse characteristics, and palpation information.

[0493] S630. Determine necessary nodes based on the MDT phase.

[0494] Before the consultation, the system requires that the original data of the four diagnostic methods, syndrome elements and AI diagnosis results meet the preset integrity requirements.

[0495] During the consultation phase, the system requires at least a complete manual review of the patient's diagnostic and verification opinions;

[0496] During the MDT conclusion formation stage, the system requires at least that the TCM treatment plan be related to the final syndrome judgment;

[0497] In the post-consultation phase, the system can use the follow-up feedback status as a basis for subsequent profile updates and closed-loop evaluation;

[0498] Follow-up feedback status is mainly used for closed-loop evaluation after consultation and competency profile update; before consultation, during consultation and MDT conclusion formation stage, the system does not use the lack of follow-up feedback as the sole reason to prevent AI suggestions from entering the MDT discussion opinions or MDT conclusion candidates.

[0499] S640, Calculate the integrity score of the data chain featuring traditional Chinese medicine.

[0500]

[0501] in, The score represents the integrity of the TCM-specific data chain for the target case;

[0502] This indicates the integrity of the original data layer for the four diagnostic methods; The calculation is based on whether the fields for inspection, auscultation, inquiry, and palpation are complete. If a certain type of data is complete, the corresponding value is 1; if it is completely missing, the value is 0; if some fields are filled, the value is between 0 and 1 based on the proportion of filled fields to the required fields of that type.

[0503] Indicates the completeness of the syndrome element layer; Calculated based on the completeness of syndrome fields such as location of disease, nature of disease, cold or heat, deficiency or excess, state of qi, blood and body fluids, and pathogenesis elements;

[0504] This indicates the completeness of the AI-generated analytical results layer; Calculations are based on whether the AI ​​diagnostic results include syndrome name, confidence level, supporting fields, pathogenesis explanation, and risk warnings.

[0505] This indicates the completeness of the manual verification and review layer; The calculation is based on whether the manual review opinion includes confirmation, correction, rejection, or requests for supplementary information, and whether a reason is provided.

[0506] This indicates the completeness of the TCM treatment plan layer; The calculation is based on whether the TCM treatment plan is correlated with the final syndrome judgment, treatment principles, and MDT conclusion candidates;

[0507] This indicates the completeness of the follow-up feedback layer. The calculation is based on whether the follow-up feedback includes fields such as symptom score, syndrome score, tongue appearance changes, pulse appearance changes, quality of life evaluation, and adverse reactions;

[0508] to This indicates the weight of the corresponding level.

[0509] The integrity of the original data layer for the four diagnostic methods is as follows:

[0510]

[0511] in, This indicates the completeness of the diagnostic data. This indicates the completeness of the auscultation and olfaction data; This indicates the completeness of the consultation data; This indicates the completeness of the palpation data. A value of 1 is assigned to each complete data category, 0 to incomplete data, and for partially complete data, a value between 0 and 1 can be assigned based on the proportion of fields already filled in.

[0512] This is used to transform the integrity of the TCM-specific data chain into a repeatable score, evaluating whether medical staff have the ability to complete the data, verify the diagnosis, and associate the treatment plan with the TCM AI diagnosis suggestions. This avoids AI diagnosis results that lack the four diagnostic methods, syndrome evidence, or manual diagnosis verification from directly entering the MDT discussion opinions or MDT conclusion candidates.

[0513] When a certain level is not applicable in the current MDT phase, the system can mark that level as not applicable in this phase and normalize the weights of the remaining levels involved in the calculation; when a certain level is a necessary node in the current phase but the data is missing, the system will set the integrity of the corresponding level to 0 and generate a corresponding correction task.

[0514] S650, Determine if the threshold has been reached.

[0515] In some embodiments, the data chain integrity threshold for the AI-based diagnostic stage before consultation is 0.70, the threshold for the manual review stage during consultation is 0.80, the threshold for the conclusion formation stage is 0.85, and the threshold for the closed-loop stage after consultation is 0.75.

[0516] If the completeness score is lower than the threshold corresponding to the current MDT stage, the system determines that the data chain is incomplete and triggers tasks such as supplementing data from the four diagnostic methods of TCM, supplementing syndrome elements, manual syndrome differentiation review, or associating TCM treatment plans. Before the correction is completed, the system can prevent the corresponding AI syndrome differentiation suggestion from entering the MDT discussion opinions or MDT conclusion candidates, and use the result as a data source for evaluating the ability of medical staff to integrate TCM characteristics.

[0517] S700, Evaluation of AI Recommendation Difference Handling Capability

[0518] This step can be performed in two phases:

[0519] Firstly, before medical staff intend to submit AI suggestions as MDT discussion opinions or MDT conclusion candidates, the system can compare the differences between the target AI suggestions and the opinions that medical staff intend to submit, and require the reasons for the differences to be filled in when the differences reach a threshold.

[0520] Secondly, after the final conclusion of the MDT is formed, the system can compare the differences between the target AI suggestions, the medical staff's review opinions and the final conclusion of the MDT, and use the reasons for the differences as the basis for closed-loop analysis and competency profile updates.

[0521] S710, Generate Tag Set

[0522] The system converts AI suggestions, medical staff review opinions, and MDT final conclusions into a structured set of tags. The tags include at least one of the following: disease name, disease stage, diagnosis category, treatment plan, medication recommendations, risk warnings, TCM syndrome, treatment principles, follow-up indicators, and contraindications.

[0523] S720, Extract Unstructured Text Tags

[0524] For unstructured AI suggestions or MDT conclusion text, the system extracts tags through medical terminology dictionary matching, structured form mapping, manual annotation, or natural language processing, and associates them with the original text location or field source.

[0525] For TCM-related labels, the system can use existing TCM terminology databases, disease and syndrome classification codes, syndrome terminology databases, or manual confirmation results from medical institutions for mapping.

[0526] S730, Calculate the similarity between AI suggestions and MDT conclusions

[0527]

[0528] in, This indicates the degree of label similarity between the AI ​​suggestions and the final conclusions of the MDT (Multidisciplinary Team). Determine if there is a substantial difference between the AI ​​suggestions and the final conclusion of the MDT (Multidisciplinary Team). Its function is to trigger the structured recording of the reasons for the difference, so that the reasons why the AI ​​suggestions were adopted, corrected, rejected, or not processed can be structured, preserved, and traced.

[0529] This represents the set of AI-suggested tags; Generate based on the diagnostic labels, treatment labels, medication labels, risk labels, TCM syndrome labels, and follow-up labels in the AI-suggested output;

[0530] This represents the set of labels representing the final conclusion of the MDT. Generate based on structured fields in the final conclusion of the MDT, consultation conclusion forms, manually annotated results, or medical tags after natural language processing;

[0531] This represents the number of tags in the intersection of two tag sets.

[0532] This represents the number of tags in the union of two tag sets.

[0533] When the union of two tag sets is empty, the system can mark the similarity as uncalcifiable and trigger manual annotation, structured supplementation, or exclude it from the current difference calculation.

[0534] S740. Determine if there is a significant difference.

[0535] when When the value is below a preset threshold, the system determines that there is a significant difference between the AI ​​suggestion and the final conclusion of the MDT. Preferably, the preset threshold is 0.60.

[0536] The system can also calculate the tripartite consistency between AI suggestions, medical staff review opinions, and the final conclusions of the MDT (Multidisciplinary Team):

[0537]

[0538] in, Indicates the degree of consistency among the three parties; This indicates the degree of similarity between the AI's suggestions and the medical staff's review opinions;

[0539] This indicates the degree of similarity between the medical staff's review opinions and the final conclusion of the MDT (Multidisciplinary Team).

[0540] This indicates the degree of similarity between the AI's recommendations and the final conclusions of the MDT.

[0541] When the consensus among the three parties is lower than a preset threshold, or when a high-risk label from the AI ​​suggestion does not appear in the MDT conclusion candidates, or when the MDT conclusion adds a treatment option label that conflicts with the AI ​​suggestion, the system triggers a structured recording process for the reasons for the discrepancy.

[0542] S750, Structured Record of Triggering Differences

[0543] The system requires medical staff to select or fill in the reasons for discrepancies and save them in conjunction with AI suggestions, review opinions, MDT conclusions, and case fields. Reasons for discrepancies may include insufficient AI support, insufficient data, individual mismatch, TCM syndrome correction, coverage of MDT expert opinions, risk control, patient factors, and other reasons confirmed manually.

[0544] The S700 is used to identify substantial differences between AI recommendations and MDT conclusions, enabling the reasons why AI recommendations are adopted, corrected, rejected, or left untreated to be structurally stored and used as a data source for updating the competency profile of medical staff in applying the AI ​​system.

[0545] S800, Evaluation of AI Suggestion Closed-Loop Processing Capability

[0546] S810, Constructing AI-Suggested Closed-Loop Objects

[0547] The system combines the target AI suggestions, medical staff review opinions, MDT final conclusions, reasons for discrepancies, and follow-up feedback into a closed-loop AI suggestion object.

[0548] S820, Determine the AI ​​suggestion processing result

[0549] Based on the relationship between AI suggestions and the final conclusions of the MDT, the system determines the processing result as adoption, partial adoption, correction, rejection, omission, or no processing.

[0550] When the AI's suggestion is highly consistent with the final conclusion of the MDT (Multidisciplinary Team), and medical staff have completed the necessary review process, the result is adoption;

[0551] When AI suggests that some content be included in the final conclusion of the MDT (Multidisciplinary Team), and there are some corrections or additions, the result is partial adoption;

[0552] When an AI suggestion is not directly incorporated into the MDT (Multidisciplinary Team) conclusion, but is modified into other suggestions after review by medical staff and adopted by the MDT conclusion, the processing result is a correction.

[0553] When an AI suggestion is not adopted due to insufficient evidence, insufficient data, incompatibility with the patient's individual condition, incompatibility with TCM syndrome, contraindications, or lack of support from MDT expert opinions, the result is rejection.

[0554] When the AI ​​system has generated a suggestion but medical staff have not reviewed or verified it, and the MDT conclusion does not indicate whether the suggestion was considered, the processing result is omission or no processing.

[0555] S830, Calculating the credibility of AI suggestion processing results

[0556]

[0557] in, This indicates the credibility of the AI's suggested processing results. This is used to evaluate whether medical staff have sufficient evidence for review and closed-loop support for the processing results of AI suggestions. The review, reasons for differences, MDT conclusions and follow-up feedback are integrated into the credibility of traceable processing results, reducing the direct use of AI suggestion processing results that have not been fully reviewed or lack follow-up support as the basis for competency evaluation.

[0558] This indicates the completeness of the manual review comments; Calculated based on whether the manual review opinion includes fields such as review conclusion, review basis, reasons for adoption or rejection, explanation of individual patient factors, and evidence tracing records;

[0559] Indicate the completeness of the record of reasons for the difference; Based on whether the reasons for the discrepancies have been selected from the discrepancy reason database, whether supplementary explanations have been filled in, whether the corresponding evidence fields have been associated, and whether the superior's confirmation calculation has been completed;

[0560] This indicates the degree to which the final conclusion of the MDT supports the processing results; The calculation is based on whether the final conclusion of the MDT supports the AI-suggested processing results.

[0561] This indicates the degree to which follow-up feedback supports the treatment outcome; The calculation is based on whether the follow-up feedback supports the processing result.

[0562] , , and Completeness or support level can be represented by a value between 0 and 1. to The corresponding weights are set and normalized within the scope of the indicators involved in the calculation.

[0563] When manual review opinions, records of reasons for discrepancies, final conclusions of the MDT (Multidisciplinary Team) investigation, and follow-up feedback have all been generated and are all applicable to the target AI recommendations, the system can proceed according to... Calculate the credibility of the AI ​​suggestion processing results for the target.

[0564] When follow-up feedback has not yet been generated, the follow-up feedback is not applicable to the target AI suggestion, or other indicators are not applicable within the current evaluation period, the system will... Excluded from the set of credibility calculation indicators for the current evaluation period, and calculate the credibility of the interim processing results for the remaining indicator weights according to the following normalization formula:

[0565]

[0566] in, AI suggestions for the target The set of indicators used in credibility calculations during the current evaluation period; Indicates the first The values ​​of each participating indicator; Indicates the first The weights of each participating indicator.

[0567] When follow-up feedback is generated, the system will update the target AI-suggested closed-loop object from the pending follow-up update state to the complete closed-loop evaluation state, and... Incorporate it into the credibility calculation.

[0568] S840, Generate Review Task

[0569] When the credibility of the processing result is lower than the threshold, the system marks the AI ​​suggestion processing result as an event that needs to be reviewed and incorporates it into subsequent quality control and profile updates.

[0570] S900, implementation of AI model version adaptive re-authorization and special adjustment of permission status

[0571] This step is used to specifically adjust the AI ​​system application competency profile and permission status of medical staff when the AI ​​model version changes, the model version status is unknown, or the medical staff has not completed model version learning. S900 does not replace the comprehensive competency score generated by S350 and the basic permission status determined by S360, but rather performs re-authorization verification, temporary restriction, or restoration processing on the basic permission status in scenarios where the model version changes.

[0572] S910, Recognizing AI Model Version Changes

[0573] The system identifies whether the AI ​​model has undergone changes in output fields, scope of application, model logic, risk warning rules, or TCM syndrome differentiation labeling system. This model version change information can be provided by the AI ​​system provider, hospital information department, model management platform, or medical institution AI system operation and maintenance platform.

[0574] If the system fails to obtain a description of the model version change, it will mark the model version status as unknown and mark the relevant AI suggestions as requiring review at least. For high-risk AI suggestions, treatment plan recommendation AI suggestions, or TCM auxiliary diagnosis AI suggestions, the system will restrict them from being directly submitted to the MDT discussion opinions or MDT conclusion candidates.

[0575] S920, Impact of Computational Model Version Changes on Scoring

[0576]

[0577] in, This indicates that changes in the AI ​​model version affect the score. This is used to quantify the impact of AI model version changes on medical staff's access rights, link model version updates with medical staff's version adaptability assessment and re-authorization control, and prevent medical staff from continuing to use high-risk AI functions without understanding the changes in the output logic, risk rules or scope of application of the new version of the AI ​​system.

[0578] Indicates whether the output field has changed; This is obtained by comparing the current AI model version with the previous version to see if the output field list, field meaning, field unit, and field explanation have changed.

[0579] Indicates whether the scope of application has changed; This was obtained by comparing whether the applicable diseases, target populations, departments, and data types of the current AI model version have changed.

[0580] This indicates whether the core logic of the model has changed; The determination is based on the version description, model change record, algorithm description, and output rule change description provided by the model supplier;

[0581] Indicate whether the risk warning rules have changed; This is obtained by comparing whether the risk level output rules, risk warning thresholds, and risk labeling system have changed.

[0582] This indicates whether there have been changes in the TCM diagnostic rules or syndrome labeling system; This was obtained by comparing whether the TCM diagnostic rules, syndrome labeling system, syndrome mapping table, and TCM treatment principle recommendation logic had changed.

[0583] to This indicates the corresponding weight.

[0584] , , , , The value is 0 if there is no change, and 1 if there is a change. For cases where only a partial change occurs, the value between 0 and 1 can be determined based on the model version change description, the scope of field impact, the degree of change in the scope of application, or the medical institution configuration table.

[0585] S930, Determine if reauthorization is triggered

[0586] The system affects the scoring based on model version changes. Compared with the preset first threshold Second threshold The comparison results determine whether to trigger model version learning, field interpretation confirmation, simulated case assessment, or re-authorization control. Among these, the first threshold... A learning threshold for model version changes is used to distinguish between low-impact version changes and medium-impact version changes that require medical personnel to complete version learning or field interpretation confirmation; a second threshold... A re-licensing threshold for model versions is set to distinguish between medium-impact version changes and high-impact version changes that require restrictions on high-risk AI functions and completion of simulated case assessments.

[0587] in:

[0588]

[0589] when When the system determines that the model version change is a low-impact change, it only pushes the version update description and writes the version update description viewing event into the AI-MDT interaction event sequence;

[0590] when When the system determines that the model version change is a medium-impact change, it requires medical staff to complete field interpretation confirmation or version learning, and writes the field interpretation confirmation event or version learning event into the AI-MDT interaction event sequence.

[0591] when When the system determines that the model version change is a high-impact change, it restricts the medical staff's independent adoption or submission permissions in high-risk cases and requires them to complete a simulated case assessment before restoring their corresponding permissions.

[0592] In some embodiments, when When using a normalized value between 0 and 1, the first threshold The second threshold can be 0.30. A value of 0.60 can be used. The above threshold can also be configured by medical institutions in the threshold configuration table based on the AI ​​system type, AI suggestion type, case risk level, model update frequency, and hospital management requirements. The threshold configuration table must at least record the threshold name, applicable AI suggestion type, applicable case risk level, threshold value, activation time, expiration time, and configuration version number.

[0593] When the target AI suggestion belongs to the category of TCM-assisted syndrome differentiation AI suggestion or TCM treatment plan recommendation AI suggestion, and the version change involves TCM syndrome differentiation rules, syndrome labeling system, syndrome mapping table, or TCM treatment principle recommendation logic, the system can increase the requirements for version learning, simulated case assessment, or manual syndrome differentiation review corresponding to the target AI suggestion. In some embodiments, the system can directly determine such changes as changes of at least moderate impact, or upgrade them to high impact changes based on the configuration of the medical institution.

[0594] S940, Restrict permissions for personnel who have not completed reauthorization.

[0595] Before medical staff complete the reauthorization, the system restricts them from independently adopting or submitting relevant AI suggestions in high-risk cases, and uses the results of this restriction as a data source for evaluating the adaptability of medical staff to the AI ​​model version.

[0596] The restrictions may include disabling the high-risk AI suggestion submission interface, marking the target AI suggestion as pending confirmation from higher authorities, marking the target AI suggestion as pending model version re-authorization, or requiring medical personnel to complete version learning, field interpretation confirmation, or simulated case assessment before entering the S500 gating judgment.

[0597] S950, Calculation Version Association Permission Adjustment Rating

[0598]

[0599] in, Indicates medical staff Version-related permission adjustment rating, It is used to make specific corrections to the basic permission status determined by S360 in the case of model version change. It is not used to replace the comprehensive competence score in S350, nor is it used to repeatedly deduct the same event that has already been processed in the S342 ethical security and risk governance capability score or the S352 major red line event additional deduction item.

[0600] This indicates the overall competence score of medical staff; Obtained based on the comprehensive competency score calculated from S350;

[0601] Indicates the learning completion status of the AI ​​model version; Obtained from training and assessment systems, model version learning records, or simulated case assessment results.

[0602] This indicates the historical usage and performance of the norms; The proportion of medical staff who complete AI evidence tracing, manual review, record of reasons for discrepancies, confirmation by superiors, correction tasks, and follow-up closure within the most recent evaluation period is determined;

[0603] Indicates the intensity of historical risk events; According to model version

[0604] The frequency and severity of events such as using related AI functions after the update has not been completed, attempting to submit high-risk AI suggestions when the model version status is unknown, and continuing to initiate high-risk AI suggestion submission requests after failing the simulated case assessment are determined.

[0605] to This indicates the corresponding weight.

[0606] All parameters should be uniformly scored from 0 to 100 in the same calculation, or uniformly normalized from 0 to 1, and should not be mixed. to For the corresponding weights.

[0607] S960, Update Permission Status

[0608] The system adjusts the scoring, model version learning completion status, and mandatory restrictions based on version-related permissions, and makes specific updates to the current permission status of medical staff.

[0609] Once medical staff have completed version learning, field explanation confirmation, or simulated case assessment, and there are no mandatory restrictions, the system can restore their corresponding AI function permissions.

[0610] When medical staff have not completed model version reauthorization, the model version status is unknown, high-risk AI functions are still used incorrectly after version learning, or there are abnormal events in permission audit, the system can directly adjust the medical staff's AI function permission status to restricted high-risk AI function status, model version reauthorization status, or status requiring confirmation from a senior doctor, without being subject to version-related permission adjustment scoring restrictions.

[0611] The permission status update result is written into the permission audit record and the AI-MDT interaction event sequence, and fed back to the medical staff AI system application competency profile for subsequent S400 control condition generation and S500 gating judgment.

[0612] S1000, Execute abnormal data processing and competency profile feedback update

[0613] This step is designed to improve the system's feasibility in complex real-world hospital data environments. It ensures that even in situations where AI suggestions are missing, AI confidence fields are missing, TCM diagnostic data is missing, follow-up data is delayed, MDT conclusions are unstructured, case risk levels are missing, AI model versions are unknown, historical competency profiles are missing, or access control audit data is missing, the system can still continue to perform competency evaluation, flow control condition generation, pre-submission gate control judgment, closed-loop evaluation, and access control feedback updates according to preset anomaly handling rules.

[0614] When identifying abnormal data, the system generates anomaly status markers for corresponding cases, target AI suggestions, medical staff profiles, or event sequences. These anomaly status markers include at least the anomaly type, the system from which the anomaly originated, the time of anomaly discovery, the steps affecting the process, alternative handling methods, the status of the correction task, and whether it affects gating results. The anomaly status markers are written into the AI-MDT interaction event sequence or permission audit records for subsequent auditing, correction, profile updates, and gating condition adjustments.

[0615] S1010, Lack of AI suggestion processing

[0616] When the AI ​​system fails to generate a target AI suggestion, or when the AI ​​suggestion generation fails, returns an empty value, or times out without returning a value, the system marks the target AI suggestion as unavailable.

[0617] When AI is unavailable, the system will not include the target case in the evaluations directly related to the target AI suggestion in the AI ​​suggestion adoption rate, AI suggestion review completion rate, AI suggestion difference handling capability, and AI suggestion closed-loop processing capability. At the same time, the system records AI unavailability events to avoid attributing the failure of the AI ​​system to generate suggestions to medical staff for not viewing, reviewing, or adopting AI suggestions.

[0618] When the target case still needs to enter the MDT (Multidisciplinary Team) process, the system allows medical staff to submit manual opinions according to the regular MDT process, but does not perform the pre-submission AI suggestion gating judgment for that target AI suggestion. If the medical institution requests supplementary AI analysis, the system can generate AI suggestions to re-call the task, a manual review task, or an information technology department verification task.

[0619] This step is used to distinguish between two different situations: "the AI ​​system did not generate suggestions" and "medical staff did not process AI suggestions," in order to avoid erroneously lowering medical staff's AI system application competence score.

[0620] S1020, Handling AI Confidence Deficiency

[0621] When an AI suggestion has been generated but the AI ​​confidence field is missing, the system marks the target AI suggestion as having missing AI confidence and selects alternative parameters according to preset alternative rules.

[0622] In some embodiments, the system may sequentially use AI risk level, AI evidence completeness, AI application scope prompts, AI prohibition prompts, or model version risk prompts as alternative parameters. If none of the above alternative parameters exist, the system may mark the target AI suggestion as requiring manual review or superior confirmation, according to the principle of strict risk assessment.

[0623] The AI ​​confidence deficiency state does not automatically prevent the target AI from being included in the MDT discussion or MDT conclusion candidates. However, the system can add manual review, evidence tracing, or superior confirmation requirements when generating flow control conditions in S400, and use this state as one of the risk control conditions when making gating judgments in S500.

[0624] This step ensures that even if the AI ​​confidence field is missing, the system can still generate control conditions based on other available fields, thus preventing the entire gating process from failing due to the absence of a single field.

[0625] S1030, Handling missing data in the four diagnostic methods of Traditional Chinese Medicine

[0626] When the target AI suggestion belongs to the category of TCM auxiliary diagnosis AI suggestion or TCM treatment plan recommendation AI suggestion, and the original data of the four diagnostic methods are missing or incomplete, the system generates a TCM four diagnostic data missing marker and triggers a four diagnostic data supplementation task.

[0627] For the four diagnostic methods data that are necessary nodes in the current MDT stage, if the corresponding fields are missing, the system will set the corresponding level of integrity to 0 or a value between 0 and 1 according to the proportion of fields filled in the S600 Traditional Chinese Medicine Characteristic Data Chain Integrity Score; for the four diagnostic methods data that are not applicable to the current MDT stage, the system can mark them as not applicable to this stage and normalize the weights of the remaining participating levels.

[0628] Before the supplementary data entry is completed, the system may not directly calculate the completion status of AI diagnostic review, and may prevent the corresponding AI diagnostic suggestions from entering the MDT discussion opinions or MDT conclusion candidates based on the flow control conditions generated by S400. After the supplementary data entry is completed, the system will write the TCM four diagnostic methods data supplementary data entry event into the AI-MDT interaction event sequence and recalculate the TCM characteristic data chain integrity score.

[0629] This step is used to prevent AI diagnosis results lacking the four diagnostic methods from directly entering the MDT conclusion candidates, while ensuring that the system can continue to perform the evaluation of the integration capabilities of traditional Chinese medicine characteristics after the supplementary data is entered.

[0630] S1040, Unstructured processing of MDT conclusions

[0631] When the final conclusion of the MDT, the discussion opinions of the MDT, or the review opinions of medical staff are unstructured text and a structured tag set cannot be directly generated, the system will mark the corresponding conclusion as pending structuring and proceed to the process of supplementing the structured consultation form, manual annotation, medical terminology dictionary matching, or natural language processing conversion.

[0632] Before the structured processing is completed, the system may temporarily not calculate the label similarity, three-party consistency and the credibility of the AI ​​suggestion processing results in S700, or mark the closed-loop object of the AI ​​suggestion as pending structured update.

[0633] Once the structured processing is complete, the system generates a set of structured labels and re-executes the AI ​​suggestion difference handling capability evaluation and the AI ​​suggestion closed-loop handling capability evaluation. If the structured processing is not completed within the preset time, the system can mark the case as requiring manual review and will not use it as a negative evaluation basis for the medical staff's AI suggestion difference handling capability within this evaluation cycle.

[0634] This step is used to address the problem that the textual and unstructured nature of MDT conclusions prevents the execution of difference calculations and closed-loop evaluations.

[0635] S1050, Handling missing or inapplicable follow-up feedback

[0636] When follow-up feedback has not yet been generated, follow-up feedback fields are missing, or the target AI suggestion is not applicable to follow-up evaluation, the system will mark the corresponding AI suggestion closed-loop object as pending follow-up update status or follow-up inapplicable status.

[0637] For the pending follow-up update status, the system can first generate a phased closed-loop evaluation result based on the manual review opinions, the record of reasons for differences, and the final conclusion of the MDT, and then recalculate the credibility of the processing result and the closed-loop processing capability score after the follow-up feedback is generated.

[0638] If follow-up is not applicable, the system can exclude follow-up feedback indicators from the set of credibility calculation indicators for the current evaluation period and normalize the weights of the remaining indicators involved in the calculation.

[0639] This step is to prevent the S800 closed-loop evaluation from failing due to delayed follow-up data, and also to avoid scoring 0 points for follow-up feedback errors that have not yet occurred.

[0640] S1060, Case handling risk level field missing.

[0641] When the required fields for the risk level of a case are missing, causing the system to be unable to accurately determine the risk level of the target case, the system marks the target case as having a risk level that needs to be corrected and generates a case data correction task.

[0642] Before the correction is completed, the system can temporarily classify the target case as medium-risk or high-risk according to the principle of strict risk control, and generate AI-suggested transfer control conditions accordingly. After the correction is completed, the system recalculates the case risk score and case risk level, and updates the corresponding set of AI-suggested transfer control conditions.

[0643] This step is used to prevent the system from incorrectly using low-risk gating rules due to missing case risk fields.

[0644] S1070, Addressing the lack of historical competency profiles

[0645] When a medical staff member's historical competency profile is missing, the system generates an initial profile and sets basic AI function permissions.

[0646] The initial profile can be generated based on the medical staff's job role, professional title, specialty, completed AI training records, simulated case assessment results, authorized AI function scope, and the medical institution's preset initial rules. If the above information is incomplete, the system can set the medical staff to basic usage status, allowing them only to view AI suggestions, conduct learning-based reviews, or fill in non-submission-based review comments.

[0647] As subsequent gating results, review results, reasons for discrepancies, closed-loop results, training completion status, and model version learning status are generated, the system gradually updates the AI ​​system application competency profile of the medical staff.

[0648] This step is designed to enable newly hired staff, staff newly assigned to departments, or medical personnel using AI systems for the first time to be included in the operation of the system of this invention.

[0649] S1080, AI model version unknown.

[0650] When AI model version information is missing, the version number cannot be read, the version change description is missing, or the model version status cannot be confirmed, the system will mark the target AI suggestion as having an unknown model version.

[0651] When the model version is unknown, the system restricts the submission of high-risk AI functions and can mark AI suggestions such as treatment plan recommendations, high-risk drug review, TCM auxiliary diagnosis, or TCM treatment plan recommendations as requiring confirmation from higher authorities or model version confirmation.

[0652] The system can also generate model version confirmation tasks, information technology department verification tasks, or AI system supplier version description supplementation tasks. After the version confirmation is completed, the system will write the model version confirmation event into the AI-MDT interaction event sequence and re-execute the S900 model version adaptive re-authorization and permission status special adjustment.

[0653] This step is to prevent medical staff from continuing to use high-risk AI functions independently when the AI ​​model version, scope of application, or risk warning rules are unclear.

[0654] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0655] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A gating evaluation system for the AI ​​competence of medical staff under a digital MDT model, characterized in that: include: The multi-source AI-MDT data acquisition and governance module is used to acquire data related to target MDT cases, target AI suggestions, and participating medical personnel, and to perform data anonymization, field standardization, and data quality verification to generate a standardized basic data package. The AI-MDT interaction event sequence generation module is used to convert the operation logs and business data in the standardized basic data package into AI-MDT interaction events, and associate them according to the target medical staff, target cases, target MDT consultations and target AI suggestions to form a time-ordered AI-MDT interaction event sequence corresponding to the target AI suggestions; The case risk and scenario determination module is used to generate the risk level and AI application scenario category of the target MDT case based on the target MDT case data, AI system risk prompts, current MDT stage and consultation topic; The AI ​​system application competency profile module for medical personnel is used to generate or update the AI ​​system application competency profile of participating medical personnel based on one or any combination of the AI-MDT interaction event sequence, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results, and to determine the AI ​​function permission status of medical personnel. The competency-related control condition generation module is used to generate a set of flow control conditions that the target AI suggestion must meet before entering the MDT discussion opinions or MDT conclusion candidates, based on the risk level of the target MDT case, the category of AI application scenario, the type of target AI suggestion, the competency profile of medical staff in applying the AI ​​system, the AI ​​function permission status of medical staff, the AI ​​model version status, and the integrity of the TCM characteristic data chain when the target AI suggestion belongs to the TCM auxiliary diagnosis type AI suggestion or the TCM treatment plan recommendation type AI suggestion. The AI ​​suggestion flow gating judgment module is used to read the AI-MDT interaction event sequence, the medical staff's AI system application competence profile, the medical staff's AI function permission status, and the flow control condition set when medical staff intend to submit the target AI suggestion as an MDT discussion opinion or MDT conclusion candidate. Based on the flow control condition set, it determines whether the target AI suggestion meets the pre-submission gating requirements and outputs the release result, blocking result, correction prompt, review reminder, superior confirmation request, training task, model version re-authorization request, or permission restriction result. The competency evaluation result feedback control module is used to update the medical staff's AI system application competency profile, the medical staff's AI function permission status, and subsequent flow control conditions based on one or any combination of the gating results, review results, reasons for discrepancies, closed-loop processing results, training completion status, model version learning results, and follow-up feedback results from the AI ​​suggestion flow gating judgment module.

2. The AI ​​competency gating evaluation system for medical personnel under the digital MDT model according to claim 1, characterized in that, It also includes one or any combination of the following modules: threshold weight and circulation rule configuration module, traditional Chinese medicine characteristic data chain evaluation module, AI suggestion difference reason evaluation module, AI suggestion closed-loop processing evaluation module, AI model version adaptability evaluation and re-authorization module, and abnormal data processing module; The threshold weight and flow rule configuration module is used to read the threshold configuration table, weight configuration table and flow control rule configuration table of the current effective version, and provide the current effective configuration version to one or any combination of the case risk and scenario judgment module, medical staff AI system application competency profile module, competency association control condition generation module, AI suggestion flow gating judgment module, traditional Chinese medicine characteristic data chain evaluation module, AI suggestion closed-loop processing evaluation module and AI model version adaptability evaluation and re-authorization module; The TCM-featured data chain evaluation module is used to evaluate the integrity of the data chain between the original data of the four diagnostic methods, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plans and follow-up feedback when the target AI suggestion belongs to the TCM-assisted syndrome differentiation AI suggestion or the TCM treatment plan recommendation AI suggestion. The AI ​​suggestion difference evaluation module is used to generate a structured record of the reasons for the difference when there are differences between the target AI suggestion, the medical staff's review opinion, the opinion to be submitted to the MDT discussion, or the final conclusion of the MDT. The AI ​​suggestion closed-loop processing and evaluation module is used to construct the AI ​​suggestion closed-loop object based on the target AI suggestion, medical staff review opinions, reasons for differences, MDT final conclusions and follow-up feedback, and to determine the AI ​​suggestion processing result; The AI ​​model version adaptability evaluation and re-authorization module is used to generate version learning tasks, field explanation and confirmation tasks, simulated case assessment tasks, or permission re-authorization requirements when the AI ​​model version undergoes changes in output fields, scope of application, model logic, risk warning rules, or TCM syndrome differentiation labeling system. The abnormal data processing module is used to generate abnormal status markers, alternative processing results, or correction tasks according to preset abnormal processing rules when there are missing AI suggestions, missing AI confidence fields, missing data from the four diagnostic methods of traditional Chinese medicine, missing follow-up data, unstructured MDT conclusions, missing case risk fields, unknown AI model versions, missing historical competency profiles, or missing permission audit data.

3. A gating evaluation method for AI competence of medical staff under a digital MDT model, characterized in that, Includes the following steps: S100. Obtain and standardize basic data: Acquire data on target MDT cases, target AI suggestions, and relevant medical staff, perform anonymization, field standardization, and data quality verification, and generate a standardized basic data package; S200, Constructing the AI-MDT Interaction Event Sequence: Operation logs and business data are converted into AI-MDT interaction events and then sorted and associated according to target medical personnel, target cases, target MDT consultations, and target AI suggestions. S300: Generate a competency profile of medical personnel for AI system applications. Based on the case risk level, AI suggestion type, historical profile, training and assessment records, model version learning status, and the AI-MDT interaction event sequence, generate or update the AI ​​system application competency profile of medical staff, and determine the AI ​​function permission status of medical staff. S400, AI suggestion flow control conditions based on competency profile generation: Based on the case risk level, AI application scenario category, AI suggestion type, AI system application competence profile of medical staff, AI function permission status of medical staff, and AI model version status, a set of flow control conditions that the target AI suggestion must meet before entering the MDT discussion or MDT conclusion candidate is generated; when the target AI suggestion belongs to the TCM auxiliary diagnosis type AI suggestion or the TCM treatment plan recommendation type AI suggestion, the evaluation results of the TCM characteristic data chain are called or the integrity of the generated TCM characteristic data chain is read, and it is used as the basis for generating TCM-related flow control conditions; S500, Gating before submitting AI recommendations: When medical staff intend to submit a target AI suggestion, the system reads the AI-MDT interaction event sequence, the medical staff's AI system application competency profile, the medical staff's AI function permission status, and the set of flow control conditions. It then judges the medical staff's AI function permission status, the completion status of necessary events, mandatory control conditions, event completeness, superior confirmation requirements, and model version re-authorization requirements, and outputs a release result or a blocking result. If blocked, it triggers one or any combination of control actions such as correction prompts, review reminders, superior confirmation requests, training tasks, model version re-authorization, or permission restrictions. S600, Evaluation of Competency in Implementing AI Applications with Traditional Chinese Medicine Characteristics: When the target AI suggestion belongs to the category of TCM auxiliary diagnosis AI suggestion or TCM treatment plan recommendation AI suggestion, the integrity of the data chain between the four diagnostic methods data, syndrome elements, AI diagnosis results, manual diagnosis review opinions, TCM treatment plan and follow-up feedback is evaluated; S600 is executed as needed before, during or when S400 generates TCM-related circulation control conditions, or when S500 makes gating judgment. S700, Evaluation of AI suggestion discrepancy handling capability: When there are discrepancies between the target AI suggestions, review opinions, proposed MDT discussion opinions, or the final conclusion of the MDT, generate a structured record of the reasons for the discrepancies. S800's ability to execute AI suggestions in a closed-loop processing capability evaluation: Based on the target AI suggestions, review opinions, reasons for discrepancies, final conclusions of the MDT, and follow-up feedback, construct a closed-loop object for AI suggestions and determine the processing results of AI suggestions; S900, Implement AI Model Version Adaptive Re-authorization and Special Adjustment of Permission Status: When the AI ​​model version changes in output fields, scope of application, model logic, risk warning rules, or TCM syndrome differentiation labeling system, version learning tasks, field explanation confirmation tasks, simulated case assessment tasks, or permission re-authorization requirements are generated, and the corresponding AI function permission status is adjusted according to the completion status.

4. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S300 also includes: The risk level of the target MDT case is determined based on the target MDT case data, AI system risk alerts, the current MDT stage, and the consultation topic. Read the historical profiles, historical ability dimension scores, historical AI function permission status, historical risk events, training completion status, and model version learning status of the participating medical personnel; When no historical profile exists, an initial profile is generated based on the medical staff's job role, professional title, authorized AI function scope, training records and / or simulated case assessment results, and initial AI function permissions are set. Based on the AI-MDT interaction event sequence, permission audit records, training and assessment records, model version learning records, TCM characteristic data chain correction records, and AI suggestion closed-loop processing results, update one or any combination of the following in the medical staff's AI system application competency profile: competency dimension score, comprehensive competency score, AI function permission status, scope of usable AI functions, historical risk events, training completion status, and model version learning status.

5. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S400 also includes: Basic control conditions are generated based on the risk level of the target MDT cases; Adjust the basic control conditions according to the target AI suggestion type, the current MDT stage, the AI ​​model version status, and the AI ​​function permission status of medical staff; When at least one competency dimension in the competency profile of medical staff in the AI ​​system is lower than the preset threshold, additional requirements will be added for viewing, reviewing, tracing evidence, recording the reasons for the discrepancy, confirming with superiors, training, or re-authorizing the model version corresponding to that competency dimension. When the target AI suggestion belongs to the category of AI suggestions for TCM-assisted syndrome differentiation or AI suggestions for TCM treatment plan recommendations, additional requirements are added based on the integrity of the TCM-specific data chain, including supplementary recording of the four diagnostic methods, verification of AI syndrome differentiation results, completion of manual syndrome differentiation verification opinions, or association with TCM treatment plans. The integrity of the TCM-specific data chain is determined by at least a portion of the data from the original data of the four diagnostic methods, syndrome elements, AI syndrome differentiation results, manual syndrome differentiation verification opinions, TCM treatment plans, and follow-up feedback in the standardized basic data package. When the target AI suggestion does not belong to the category of AI suggestions for TCM-assisted syndrome differentiation or AI suggestions for TCM treatment plan recommendations, the integrity of the TCM-specific data chain is not a mandatory transfer control condition. When multiple control conditions apply to the same target AI recommendation, the control conditions are integrated according to the strictest principle; if there is a conflict between the release and restriction between different control conditions, the restriction condition is used as the final control condition; if different control conditions correspond to different confirmation levels, the higher confirmation level is used as the final confirmation requirement.

6. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S500 also includes: Read the set of flow control conditions corresponding to the target AI suggestion. The set of flow control conditions includes at least mandatory control conditions, correctable control conditions, superior confirmation conditions, model version re-authorization conditions, and permission level conditions. Based on the necessary event types in the set of flow control conditions, retrieve whether the corresponding interaction event has been completed in the AI-MDT interaction event sequence; When a necessary event type is marked as a mandatory control condition and the corresponding interactive event is not completed, it is determined that the target AI suggestion does not meet the pre-submission gating requirements. When the AI ​​function permission status of medical staff does not meet the permission level conditions corresponding to the target AI suggestion, it is determined that the target AI suggestion does not meet the pre-submission gating requirements. When the event completeness does not reach the event completeness threshold corresponding to the target AI suggestion, the target AI suggestion is deemed not to meet the pre-submission gating requirements. For target AI suggestions that do not meet the pre-submission gating requirements, output the blocking result, and trigger one or any combination of control actions based on the incomplete control conditions, such as correction prompts, review reminders, superior confirmation requests, training task pushes, model version re-authorization, or permission restrictions; For target AI suggestions that meet the pre-submission gating requirements, the release result is output, and the gating result, operation time, operator anonymity identifier, target AI suggestion identifier, medical staff AI function permission status and configuration version are written into the AI-MDT interaction event sequence.

7. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S600 function can be invoked before, during, or when the S400 function generates TCM-related circulation control conditions, or when the S500 function performs a pre-submission gating judgment; the S600 function also includes: When the target AI suggestion belongs to the category of AI suggestion for TCM auxiliary diagnosis or AI suggestion for TCM treatment plan recommendation, the TCM characteristic data chain nodes that should be completed should be determined according to the current MDT stage. Before the consultation, at least the completeness of the original data of the four diagnostic methods, syndrome elements and AI diagnosis results should be assessed. During the consultation phase, at least the completeness of the manual diagnostic review opinion should be assessed; During the MDT conclusion formation stage, at least the correlation between the TCM treatment plan and the final syndrome diagnosis should be determined. In the post-consultation phase, the follow-up feedback will serve as the basis for subsequent profile updates and closed-loop evaluation; When the TCM-specific data chain nodes required for the current MDT stage are missing, tasks such as supplementing TCM four diagnostic methods data, supplementing syndrome elements, manual syndrome differentiation review, or associating TCM treatment plans are generated. Before the correction is completed, the corresponding TCM-related AI suggestions are restricted from entering the MDT discussion opinions or MDT conclusion candidates.

8. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S700 also includes: Before medical staff submit the target AI suggestion as an opinion in the MDT discussion or as a candidate for the MDT conclusion, compare the differences between the target AI suggestion and the opinion that the medical staff intends to submit; and after the final conclusion of the MDT is formed, compare the differences between the target AI suggestion, the medical staff's review opinion and the final conclusion of the MDT. When the difference reaches a preset threshold, a task is generated to generate a structured record of the reason for the difference. The reasons for the discrepancies include one or any combination of the following: insufficient AI basis, insufficient data, individual mismatch, TCM syndrome correction, coverage of MDT expert opinions, risk control, patient factors, and other reasons confirmed manually. The results of recording the reasons for the differences are written into the AI-MDT interaction event sequence and used as the basis for updating the medical staff's ability to review AI suggestions, identify differences, or process closed loops.

9. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to claim 3, characterized in that, The S800 also includes: The AI ​​suggestion closed-loop object is constructed based on the target AI suggestions, the review opinions of medical staff, the reasons for the differences, the final conclusion of the MDT, and the follow-up feedback; The processing result of the AI ​​suggestion is determined, including adoption, partial adoption, correction, rejection, omission, or no processing; The credibility of AI-recommended processing results is determined based on the completeness of human review opinions, the completeness of records of reasons for differences, the degree of support for the final conclusion of the MDT, and the degree of support for follow-up feedback. When follow-up feedback has not yet been generated or is not applicable to the target AI suggestion, the corresponding AI suggestion closed-loop object is marked as pending follow-up update status, and the remaining review opinions, reasons for differences and final conclusions of MDT are used for phased evaluation; The results of AI suggestion processing and the credibility of AI suggestion processing are fed back to the AI ​​system application competency profile of medical staff, so as to update the clinical translation ability, AI suggestion review ability, closed-loop processing ability, or continuous learning and improvement ability of AI suggestions.

10. The gating evaluation method for AI competence of medical personnel under the digital MDT model according to any one of claims 3-9, characterized in that, This also includes S1000, execution of exception data processing, and competency profile feedback updates: When AI suggestions are missing, AI confidence field is missing, TCM four diagnostic methods data is missing, follow-up data is missing, MDT conclusions are unstructured, case risk field is missing, AI model version is unknown, historical competency profile is missing, or permission audit data is missing, the system generates an abnormal status marker, alternative processing results, or correction tasks; among them, when the AI ​​confidence field is missing, the system uses AI risk level, AI evidence completeness, or AI scope of application prompts as alternative parameters; When data from the four diagnostic methods of TCM is missing, the system generates a task to supplement the data and restricts the corresponding AI suggestions for TCM auxiliary diagnosis or TCM treatment plan recommendation from directly entering the MDT conclusion candidate before the supplementation is completed. When the AI ​​model version is unknown, the system marks the AI ​​model version status as unknown and restricts the submission of high-risk AI functions; When historical competency profiles are missing, the system generates an initial profile and sets basic AI function permissions. The S1000 is also used to distinguish between abnormal data on the system side and missing operations by medical staff, so as to avoid directly including data missing caused by the AI ​​system not returning, follow-up data not being generated, or external system fields not being synchronized into the negative evaluation of medical staff. In addition, the system updates the AI ​​system application competency profile of medical staff, the AI ​​function permission status of medical staff, and subsequent flow control conditions based on the gating results of S500, the difference reason recording results of S700, the AI ​​suggestion closed-loop processing results of S800, and the model version learning results or re-authorization results of S900.

Images