A medical model evaluation method and device, electronic equipment and storage medium

Abstract

The application provides a medical model evaluation method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a general evaluation set; converting tasks in the general evaluation set through a data adaptation layer based on an interface protocol of a target model, and inputting the tasks into the target model to obtain first output data; performing pre-security verification on the first output data; evaluating the target model that passes the pre-security verification based on the first output data to obtain capability distribution data of the target model; determining a special evaluation set from a preset task library based on the capability distribution data; converting tasks in the special evaluation set through the data adaptation layer, and inputting the tasks into the target model to obtain second output data; and evaluating the target model based on the first output data and the second output data. The application can improve the accuracy and comprehensiveness of medical model evaluation.

Description

Technical Field

[0001] This application relates to the field of model evaluation technology, and more specifically, to a medical model evaluation method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the deep integration of artificial intelligence (AI) technology into the medical field, various medical AI models (such as diagnostic assistance models, medical image analysis models, and electronic medical record understanding models) are constantly emerging. To ensure the safety, effectiveness, and reliability of these models in actual clinical applications, a comprehensive and professional evaluation has become an indispensable and crucial step. This evaluation of medical AI models aims to systematically examine their ability to process medical data, follow clinical logic, and mitigate medical risks, serving as an important basis for model development, selection, and market access.

[0003] However, research has revealed significant technical limitations in existing medical AI model evaluation schemes. First, the fragmentation of evaluation frameworks is a prominent issue. Currently, the industry lacks unified evaluation standards and platforms. For models of different modalities (text, image, speech) and specialties, separate and incompatible evaluation tools, datasets, and scoring systems are often required. This not only complicates the evaluation process and increases costs but also makes fair horizontal comparisons of the capabilities of different models difficult. Second, the clinical depth and safety considerations in the evaluations are severely insufficient.

[0004] Therefore, a solution is needed. Summary of the Invention

[0005] In view of this, embodiments of this application provide a medical model evaluation method, apparatus, electronic device, and storage medium to address the problems of fragmented medical AI model evaluation frameworks, insufficient consideration of clinical safety, and static and inefficient evaluation processes in the prior art.

[0006] In a first aspect, embodiments of this application provide a method for evaluating a medical model, the method comprising: Obtain a universal assessment set covering multiple clinical specialties; Based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through the data adaptation layer, and the transformed tasks are input into the target model to obtain the first output data of the target model; Perform a pre-security check on the first output data; Based on the first output data, a preliminary evaluation is performed on the target model that has passed the pre-security verification to obtain the capability distribution data of the target model in each of the clinical specialties. Based on the capability distribution data, a set of specialized assessments suitable for assessing the target model is determined from a pre-set database of in-depth assessment tasks for specific specialties; wherein, in-depth assessment tasks of different difficulty or coverage are assigned to clinical specialties with different capabilities of the target model. The data adaptation layer standardizes and transforms the tasks in the special assessment set, and the transformed tasks are input into the target model to obtain the second output data of the target model. Based on the first output data and the second output data, the target model is evaluated, and standardized evaluation results are generated.

[0007] In one feasible implementation, performing a pre-security check on the first output data includes: The system invokes a preset medical knowledge base to scan the first output data; wherein the medical knowledge base contains disabled processing items for the target population. The method further includes: If the first output data is found to match any of the disabled processing items in the medical knowledge base, the target model is marked as failing the verification; otherwise, it is marked as passing the verification.

[0008] In one feasible implementation, the target model to be evaluated includes at least one of the following types: text-based large language model LLM, visual language model VLM, medical optical character recognition (OCR) model, and medical automatic speech recognition (ASR) model.

[0009] In one feasible implementation, based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through a data adaptation layer, including: Identify the interface protocol type of the target model; Select the target conversion logic that is compatible with the interface protocol type from a variety of preset data conversion logics; Based on the target transformation logic, the task data in the general evaluation set is converted into standardized requests that conform to the target model input format.

[0010] In one feasible implementation, the target model is evaluated, including: The target model is evaluated based on at least one preset scoring strategy; the scoring strategy includes: an exact matching strategy based on keywords, fields or numerical values, and / or an LLM-as-Judge strategy based on semantic and logical evaluation of a third-party large model.

[0011] In one feasible implementation, the tasks in the general assessment set and / or the specialty depth assessment task library include at least one of the following types: Text question answering tasks, medical image understanding tasks, medical speech transcription and analysis tasks, and information extraction tasks that extract specific fields from structured or unstructured medical text.

[0012] In one feasible implementation, the method further includes: In response to a user's custom data upload request, receive the user-provided custom medical assessment dataset; Perform format compliance verification on the custom medical assessment dataset; After successful verification, the custom medical assessment dataset will be incorporated into the general assessment set or the specialty in-depth assessment task library.

[0013] Secondly, embodiments of this application also provide a medical model evaluation device, the device comprising: The acquisition module is used to acquire a universal assessment set covering multiple clinical specialties; The first conversion module is used to standardize and convert the tasks in the general evaluation set based on the interface protocol of the target model to be evaluated through the data adaptation layer, and input the converted tasks into the target model to obtain the first output data of the target model. The verification module is used to perform pre-security verification on the first output data; The first evaluation module is used to perform a preliminary evaluation of the target model that has passed the pre-security verification based on the first output data, and to obtain the capability distribution data of the target model in each of the clinical specialties. The task determination module is used to determine, based on the capability distribution data, a set of specialized assessment tasks suitable for assessing the target model from a preset specialized in-depth assessment task library; wherein, in-depth assessment tasks of different difficulty or different coverage are assigned to clinical specialties with different capabilities of the target model. The second conversion module is used to standardize and convert the tasks in the special evaluation set through the data adaptation layer, and input the converted tasks into the target model to obtain the second output data of the target model. The second evaluation module is used to evaluate the target model based on the first output data and the second output data, and generate standardized evaluation results.

[0014] In one feasible implementation, the verification module is configured to perform a pre-security verification on the first output data, for the following purposes: The system invokes a preset medical knowledge base to scan the first output data; wherein the medical knowledge base contains disabled processing items for the target population. The device further includes: The marking module is used to mark the target model as failing the verification if the first output data is found to match any of the disabled processing items in the medical knowledge base; otherwise, it is marked as passing the verification.

[0015] In one feasible implementation, the target model to be evaluated includes at least one of the following types: text-based large language model LLM, visual language model VLM, medical optical character recognition (OCR) model, and medical automatic speech recognition (ASR) model.

[0016] In one feasible implementation, the first transformation module is used to standardize and transform the tasks in the general evaluation set based on the interface protocol of the target model to be evaluated through a data adaptation layer, for the following purposes: Identify the interface protocol type of the target model; Select the target conversion logic that is compatible with the interface protocol type from a variety of preset data conversion logics; Based on the target transformation logic, the task data in the general evaluation set is converted into standardized requests that conform to the target model input format.

[0017] In one feasible implementation, the first evaluation module is used to evaluate the target model, for the following purposes: The target model is evaluated based on at least one preset scoring strategy; the scoring strategy includes: an exact matching strategy based on keywords, fields or numerical values, and / or an LLM-as-Judge strategy based on semantic and logical evaluation of a third-party large model.

[0018] In one feasible implementation, the tasks in the general assessment set and / or the specialty depth assessment task library include at least one of the following types: Text question answering tasks, medical image understanding tasks, medical speech transcription and analysis tasks, and information extraction tasks that extract specific fields from structured or unstructured medical text.

[0019] In one feasible implementation, the device further includes: The response module is used to respond to users' custom data upload requests and receive custom medical assessment datasets provided by users. The format verification module is used to verify the format compliance of the custom medical assessment dataset. The dataset composition module is used to incorporate the custom medical assessment dataset as a component of the general assessment set or the specialty in-depth assessment task library after successful verification.

[0020] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the medical model evaluation method as described in any one of the first aspects.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the medical model evaluation method as described in any one of the first aspects.

[0022] This application provides a medical model evaluation method, apparatus, electronic device, and storage medium. First, it uses a universal evaluation set covering multiple clinical specialties to conduct a preliminary capability assessment of the target model. A data adaptation layer standardizes the evaluation task, enabling models with different interface protocols to quickly connect and complete testing, solving the problems of high access costs and slow startup caused by fragmented evaluation frameworks in existing technologies. By performing pre-emptive security checks on the first output data of the target model, outputs with serious clinical safety risks can be identified and filtered early in the evaluation process, compensating for the lack of clinical safety considerations in universal evaluation methods.

[0023] Secondly, based on the capability distribution data obtained from the preliminary assessment, specialized assessment sets are dynamically determined from a pre-defined database of in-depth specialty assessment tasks. According to the differences in the initial performance of the target model across various clinical specialties, in-depth assessment tasks of corresponding difficulty and coverage are assigned to different specialties, achieving optimized allocation of assessment resources. Compared with the "one-size-fits-all" assessment method using a fixed test set in existing technologies, this method can conduct more targeted in-depth validation of the target model's strengths and weaknesses with the same or less resource consumption.

[0024] Finally, by executing customized specialized assessment sets and integrating the first output data with the second output data obtained from the specialized assessments, standardized assessment results are generated. This process reflects both the model's breadth of capabilities across multiple specialties and its depth of performance in specific domains, providing a unified and reliable basis for cross-sectional comparisons of the clinical capabilities of different models.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of a medical model evaluation method provided in an embodiment of this application is shown.

[0028] Figure 2 A flowchart of another medical model evaluation method provided in an embodiment of this application is shown.

[0029] Figure 3 A schematic diagram of the structure of a medical model evaluation device provided in an embodiment of this application is shown.

[0030] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0032] With the deep integration of artificial intelligence (AI) technology into the medical field, various medical AI models (such as diagnostic assistance models, medical image analysis models, and electronic medical record understanding models) are constantly emerging. To ensure the safety, effectiveness, and reliability of these models in actual clinical applications, a comprehensive and professional evaluation has become an indispensable and crucial step. This evaluation of medical AI models aims to systematically examine their ability to process medical data, follow clinical logic, and mitigate medical risks, serving as an important basis for model development, selection, and market access.

[0033] However, research has revealed significant technical limitations in existing medical AI model evaluation schemes. First, the fragmentation of evaluation frameworks is a prominent issue. Currently, the industry lacks unified evaluation standards and platforms. For models of different modalities (text, image, speech) and specialties, separate and incompatible evaluation tools, datasets, and scoring systems are often required. This not only complicates the evaluation process and increases costs but also makes fair horizontal comparisons of the capabilities of different models difficult. Second, the clinical depth and safety considerations in the evaluations are severely insufficient.

[0034] Based on this, embodiments of this application provide a medical model evaluation method, apparatus, electronic device, and storage medium, which are described below through embodiments.

[0035] To facilitate understanding of this embodiment, a medical model evaluation method disclosed in this application will first be described in detail. For example... Figure 1 As shown, the method includes the following steps: Step 101: Obtain a universal assessment set covering multiple clinical specialties.

[0036] This step begins with preparing a basic, interdisciplinary set of test questions. This set should cover multiple major clinical medical specialties, including cardiology, respiratory medicine, neurology, obstetrics and gynecology, and pediatrics. The purpose of this general assessment set is not to test the model's depth in a specific specialty, but rather to quickly assess the model's breadth of clinical knowledge and basic abilities. The content typically selects typical diagnostic and treatment scenarios for common and frequently occurring diseases in each specialty, such as the initial diagnosis and medication selection principles for hypertension, the identification of community-acquired pneumonia, emergency treatment procedures for stroke, and the etiological analysis of common childhood fevers.

[0037] The construction of a universal assessment set can be derived from multiple sources. One approach is to integrate existing, publicly available medical assessment benchmarks, such as academically validated datasets, and select representative cross-specialty questions from them. Another approach is to construct a standardized basic question bank based on authoritative sources such as clinical practice guidelines, with the participation of clinical experts. These questions can be in the form of multiple-choice questions, short-answer questions, or question-and-answer pairs based on structured medical record fragments. Importantly, the composition of this set must ensure that it effectively reflects whether a model possesses the potential to become a qualified "general practitioner assistant," providing a reliable preliminary basis for subsequent, more refined specialty competency assessments.

[0038] Step 102: Based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through the data adaptation layer, and the transformed tasks are input into the target model to obtain the first output data of the target model.

[0039] After obtaining a general evaluation set, it's often not feasible to directly send the questions to the target model being evaluated, as different models vary significantly in interface type, data format, and communication protocol. For example, some models only provide an HTTP interface conforming to the OpenAI API specification, expecting to receive JSON data with a specific structure; others may require calls through a dedicated software development kit (SDK), whose input parameter organization is completely different from that of the HTTP interface. Furthermore, for models targeting specific tasks such as medical image recognition or speech transcription, the input may be directly image files or audio streams, a format vastly different from that of text models.

[0040] To address the evaluation obstacles caused by the lack of standardized interfaces, this step introduces a data adaptation layer. The primary function of this layer is to act as a "translator" and "interface operator" between the evaluation process and various heterogeneous models. Its workflow is as follows: First, based on the publicly available or pre-configured interface protocol specifications of the target model, the specific specifications of the input data expected by the target model are identified. This includes, but is not limited to, data encapsulation format, field names, encoding methods, and transmission protocols. Then, according to these identified specifications, the data adaptation layer performs a series of transformations on the original questions (which may be text descriptions, file paths, or structured data objects) in the general evaluation set. For example, it might need to reorganize a text multiple-choice question into a JSON object containing fields such as "prompt" and "temperature," according to the target model's requirements; or it might need to convert the local path of a medical image file into a Base64 encoded string that can be transmitted via the HTTP protocol.

[0041] After the standardization transformation is completed, the adaptation layer will accurately send the transformed standardized request data to the target model according to the method agreed upon by the target model interface (such as initiating an HTTP POST request or calling a specific function), and receive the processing result returned by the target model, i.e., the first output data. This process ensures that regardless of the technical implementation of the target model, it can be incorporated into the subsequent evaluation process in a unified and automated manner, greatly reducing the workload and technical threshold of writing evaluation adaptation code separately for each new model. It is a key technical link to achieve large-scale, standardized model evaluation.

[0042] For example, the target model to be evaluated includes at least one of the following types: text-based large language model LLM, visual language model VLM, medical optical character recognition (OCR) model, and medical automatic speech recognition (ASR) model.

[0043] The target models to be evaluated can cover a variety of types used in practical medical scenarios, including but not limited to Large Language Models (LLMs) that understand and generate text-based data, Vision-Language Models (VLMs) that can process both image and text information simultaneously, Optical Character Recognition (OCR) models specifically designed to extract text information from medical documents or image reports, and Automatic Speech Recognition (ASR) models that convert recorded doctor-patient consultations into structured text. These models offer different interface types due to their varying data modalities and task objectives.

[0044] In order to achieve unified evaluation and access for the above diverse models, Figure 2 This example demonstrates the detailed process of standardization transformation performed by the data adaptation layer, such as... Figure 2 As shown, based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through a data adaptation layer, including the following steps: Step 201: Identify the interface protocol type of the target model.

[0045] First, the interface protocol type of the target model is identified. This identification process can be automatically completed based on the application programming interface (API) documentation publicly available from the model service provider, pre-registered model configuration information in the platform, or through trial connection and handshake interactions. The purpose of identification is to determine what form of calls the model accepts, such as a standard RESTful API, gRPC service, specific Python functions, or other custom communication methods.

[0046] Step 202: Select the target conversion logic that is compatible with the interface protocol type from a variety of preset data conversion logics.

[0047] After identifying the interface protocol type, the data adaptation layer selects a target conversion logic from a variety of pre-built data conversion logics maintained internally, which matches the currently identified interface protocol type. These pre-built logics are data processing modules that have been pre-designed and validated for different interface paradigms. For example, for a Large Language Model (LLM) that follows the OpenAI Application Programming Interface (API) specification, its conversion logic might be designed to wrap the input content into a specific message array; while for a Visual Language Model (VLM) that receives a binary stream of an image file, its conversion logic might focus on constructing a multi-part form request that includes the correct Hypertext Transfer Protocol (HTTP) header and body.

[0048] Step 203: Based on the target transformation logic, convert the task data in the general evaluation set into a standardized request that conforms to the target model input format.

[0049] Finally, the selected target transformation logic processes the specific task data extracted from the general evaluation set. This process transforms the original task description (such as a text prompt, an image file path, or an audio clip) into an unambiguous, standardized request that can be directly sent to the model server endpoint, according to the precise format, field naming, and encoding specifications required by the target model interface. For example, for a medical imaging task requiring Visual Language Model (VLM) analysis, the transformation logic might read, compress, and encode the image pointed to by the image file path along with other text parameters, ultimately generating a Hypertext Transfer Protocol (HTTP) request body that conforms to the Visual Language Model (VLM) Application Programming Interface (API) specification. Through this series of steps, the data adaptation layer effectively shields the heterogeneity of downstream models, providing a consistent and standardized model calling interface for the upper-layer evaluation process.

[0050] Step 103: Perform a pre-security check on the first output data.

[0051] After obtaining the initial output data generated by the target model for the general evaluation set, the process does not immediately proceed to scoring or in-depth analysis. Instead, a crucial "safety checkpoint" is first implemented. The core purpose of this pre-implementation safety verification step is to conduct a highly deterministic safety check on the output of the target model (the initial output data) based on clear clinical rules, aiming to identify and intercept answers that may contain serious medical risks at an early stage.

[0052] This validation does not rely on subjective semantic judgment from another complex model, but rather on a pre-built, structured medical knowledge rule base. This rule base encodes hard constraints extracted from authoritative clinical guidelines, drug instructions, and medical safety consensus. For example, the rule base explicitly includes deterministic rules such as "when the patient description includes 'pregnancy' or 'pregnant woman' status, the first output data (answer) of the target model must not contain clearly contraindicated drugs such as 'angiotensin-converting enzyme inhibitors (ACEIs),' 'angiotensin II receptor blockers (ARBs),' or 'statins'"; it also includes rules for identifying critical values, such as "when the model output involves blood potassium levels, if the value is greater than or equal to 6.5 mmol / L, then 'hyperkalemia' and 'requiring emergency treatment' must be explicitly mentioned, otherwise it is considered incorrect treatment."

[0053] During the pre-emptive safety verification process, the text content of the first output data is parsed, key entities (such as drug names, test indicators, patient status descriptions, etc.) are extracted, and quickly matched and logically judged against entries in the rule base. If the first output data triggers any high-risk safety rule (e.g., recommending contraindicated drugs for a hypothetical "pregnant woman" case, or omitting emergency treatment recommendations for the critical value of "blood glucose below 2.8 mmol / L"), the first output data and the target model that produced it will be directly marked as "safety verification failed." This interception is mandatory, designed to prevent target model outputs with clear safety flaws from entering subsequent performance scoring stages, strengthening the protection of clinical safety bottom lines from the source of the evaluation process. This design ensures an independent, objective, and efficient evaluation of the primary indicator of the target model's "safety," compensating for the significant deficiency of traditional evaluation methods that only focus on the "correctness" of the answer while ignoring its "harmlessness."

[0054] For example, performing a pre-security check on the first output data includes: A preset medical knowledge base is invoked to scan the first output data; wherein, the medical knowledge base is configured with disabled processing items for the target population.

[0055] First, a pre-defined medical knowledge base is invoked to scan the first output data. This medical knowledge base is a structured set of rules, which specifically includes prohibited items for specific target populations. For example, for the target population of "pregnant and breastfeeding women," prohibited items may explicitly list drugs classified as FDA Pregnancy Category X and examination methods with clear teratogenic risks (such as certain radiological examinations); for "children under 18 years of age," prohibited items may include quinolone antibiotics, tetracycline drugs, etc.; and for "patients with hepatic or renal insufficiency," a list of drugs that require strict dosage adjustments based on creatinine clearance or that are prohibited may be specified.

[0056] The method further includes: If the first output data is found to match any of the disabled processing items in the medical knowledge base, the target model is marked as failing the verification; otherwise, it is marked as passing the verification.

[0057] This means that once the target model's initial output data detects recommendations for drugs, examinations, or treatments that are explicitly prohibited or strictly avoided by the target population, regardless of how reasonable the recommendation may seem in other medical descriptions, a high-risk determination will be triggered directly. This labeling is a mandatory, veto-based safety conclusion, independent of the target model's score on expertise. This ensures that the assessment process maintains the highest sensitivity and zero-tolerance attitude towards the most serious types of errors that could cause direct personal harm, elevating the assessment of clinical safety from a vague semantic judgment level to a hard standard level based on well-defined rules and automated execution.

[0058] Step 104: Based on the first output data, perform a preliminary evaluation on the target model that has passed the pre-security verification to obtain the capability distribution data of the target model in each of the clinical specialties.

[0059] After completing the preliminary safety verification and filtering out model outputs with high-risk defects, this step will conduct a preliminary quantitative assessment of the performance of the target model that has passed the verification. The purpose of this step is not to conduct an in-depth and detailed analysis of the target model's specialty capabilities, but rather to quickly outline the target model's capabilities in different clinical fields through a universal assessment set covering multiple specialties, providing data support for subsequent resource allocation.

[0060] Specifically, the initial evaluation process involves processing the target model's first output data on a general evaluation set based on pre-defined basic scoring rules. For example, for each question in the general evaluation set, the target model's answer is automatically scored according to its clinical specialty (such as cardiology, respiratory medicine, pediatrics, etc.) and question type, using methods such as answer keyword matching, key value extraction and comparison, or comparison with pre-defined answer options. For open-ended questions, a relatively lightweight semantic similarity calculation method may also be used to estimate the quality of the answer.

[0061] After the assessment, the scores will be categorized and summarized by clinical specialty. For example, the average score or accuracy rate of the target model on "cardiovascular medicine" related questions, the average score or accuracy rate on "pediatrics" related questions, and so on can be calculated. Finally, these scores or performance indicators summarized by specialty are organized to form the aforementioned ability distribution data. This data intuitively reflects the relative strengths and weaknesses of the target model in various major clinical specialties during this "diagnostic test." For example, the data may show that the target model scores highly on "common internal medicine diseases" questions, but scores significantly lower on "emergency critical care identification" or "pediatric medication calculation" related questions.

[0062] This capability distribution data is a key input for intelligent assessment and planning in subsequent steps. It enables the assessment process to shift from a "one-size-fits-all" comprehensive test to a "targeted" approach based on the characteristics of the target model itself, providing a scientific basis for deeply exploring the model's true capabilities and potential risks under limited resources.

[0063] Step 105: Based on the capability distribution data, determine the special assessment set suitable for assessing the target model from the preset specialty depth assessment task library; wherein, for clinical specialties with different capabilities of the target model, assign depth assessment tasks of different difficulty or different coverage.

[0064] This step is the core decision-making process for achieving intelligent and refined assessment. After obtaining the capability distribution data of the target model on the general assessment set, the assessment process no longer conducts indiscriminate and complex in-depth tests on all clinical specialties. Instead, it "tailors" the focus and intensity of subsequent in-depth assessments based on the characteristics of the target model revealed by the capability distribution data.

[0065] Specifically, the pre-built specialty in-depth assessment task library is a structured resource pool. For each clinical specialty (such as cardiology, neurology, and emergency medicine), multiple sets of in-depth assessment tasks with different gradients and focuses are pre-configured. The design of these tasks relies heavily on a structured specialty assessment indicator system (details are provided in the "Specialty Assessment Indicator System" section below). This system defines core assessment points for each specialty, covering dimensions such as safety avoidance capabilities, critical value identification, time-series management, and guideline consistency, and clarifies the automated evaluation rules and risk levels.

[0066] The decision-making logic is as follows: For clinical specialties that perform exceptionally well or show potential in the initial assessment (e.g., scores significantly above the threshold), more challenging, broader-coverage, or more focused-on rare diseases, complex cases, and advanced clinical reasoning tasks will be selected from the specialty-specific in-depth assessment task library. The aim is to further explore the "ceiling" of the target model in this field and verify its ability to handle complex and marginal clinical scenarios. Conversely, for clinical specialties that perform relatively poorly or have significant weaknesses in the initial assessment, more focused-on in-depth assessment tasks will be assigned, emphasizing basic safety rules, consolidating core diagnostic and treatment processes, and standardizing the handling of common diseases. The aim is to confirm whether their basic safety and core capabilities meet the standards and to pinpoint their weaknesses.

[0067] This differentiated task allocation strategy optimizes the allocation of assessment resources. It precisely targets valuable in-depth assessment resources (such as tasks requiring significant computing power for complex judgments or tests involving rare case data) to the areas most worthy of in-depth investigation, while avoiding unnecessary repetitive testing on already weak aspects of the model. This maximizes the depth and efficiency of the assessment within limited time and computational costs, resulting in a high-information-density assessment report that reflects both the model's strengths and weaknesses and clearly reveals its risks and shortcomings.

[0068] Detailed Explanation of the Specialty Evaluation Index System: In constructing an in-depth evaluation system for medical artificial intelligence models, this solution designs a structured evaluation indicator system to ensure objectivity and specialty relevance. This system first categorizes clinical specialties, such as cardiology, neurology, emergency medicine, obstetrics and gynecology, and pediatrics. Each specialty has several primary evaluation indicators, focusing on core competencies and safety baselines in clinical practice.

[0069] The primary indicators mainly include: safety and pitfall avoidance capabilities, critical value identification and handling priority, timing correctness of key disease treatment procedures, consistency with clinical guidelines, and integrity of the diagnostic logic chain. Each primary indicator is further refined into secondary indicators that can be automatically judged directly through rules or algorithms.

[0070] For example, under the universally applicable safety and pitfall avoidance capability item, specific checkpoints include: whether the model output includes drugs that are absolutely contraindicated for pregnant women (such as ACEI / ARB antihypertensive drugs, statins), whether it includes drugs that are recommended for contraindication in children under 18 years of age (such as quinolones, tetracycline antibiotics), and whether there are any drug interactions that could cause serious adverse reactions. Under the critical value identification item, it is explicitly required that the model must be able to identify and prioritize the handling of several life-threatening abnormal values ​​such as serum potassium ≥6.5mmol / L, blood glucose ≤2.8mmol / L, and systolic blood pressure ≥180mmHg, and provide emergency treatment recommendations.

[0071] For specific specialties, the indicators delve into key decisions specific to that field. In cardiology, for example, key indicators include: whether the recommended time window of "reperfusion within 90-120 minutes after the first ECG" is strictly followed for patients with ST-segment elevation myocardial infarction (STEMI); and whether relevant usage rules are correctly identified and avoided when selecting antithrombotic drugs (such as aspirin and clopidogrel). In neurology, core indicators focus on the accurate assessment of indications for intravenous thrombolysis within 4.5 hours and thrombectomy for large vessel occlusion within 6-24 hours in patients with acute ischemic stroke, as well as the correct sequence of medications for status epilepticus.

[0072] All indicators are categorized into "fatal," "serious," and "moderate" levels based on the severity of their potential clinical risks, and each is assigned a corresponding weight. Violations of "fatal" level indicators (such as recommending contraindicated drugs for pregnant women, omitting critical values ​​requiring immediate attention, or delaying treatment within a critical time window) typically result in a score of zero or a significant deduction of points for that assessment item, thus achieving mandatory control over the highest level of medical safety risks. This indicator system transforms the abstract concept of "clinical competence" into quantifiable and automatically enforceable rules, thereby supporting in-depth, objective, and clinically relevant assessments of the model's specialty capabilities.

[0073] Step 106: The tasks in the special evaluation set are standardized and transformed through the data adaptation layer, and the transformed tasks are input into the target model to obtain the second output data of the target model.

[0074] This step is the execution phase of the in-depth assessment. Its core operations are technically the same as the processing of the general assessment set in step 102, but there is a progressive relationship in terms of task content and purpose. In this step, the data adaptation layer once again plays its key bridging role, transforming the dynamically determined, specialty-specific assessment tasks in step 105 into standardized requests that the target model can recognize and process.

[0075] Specifically, the tasks in specialized assessment sets are more specialized and complex than those in general assessment sets. For example, they may include complex cases that require the analysis of multimodal information (such as a combination of medical images and text descriptions), or they may involve adherence to guidelines for the diagnosis and treatment of rare diseases, or accurate judgment of the timing of critical care. These tasks may themselves exist in the form of specific data structures or documents.

[0076] The data adaptation layer, based on the identified target model interface protocol, applies corresponding transformation logic to standardize and encapsulate the data for these specific tasks. This transformation process ensures that regardless of the complexity of the specific task or its original format, it can be adapted to a unified format that conforms to the target model input specifications. For example, for a comprehensive assessment task of "acute appendicitis CT images and medical history text" that requires Visual Language Model (VLM) analysis, the data adaptation layer may need to combine and encode the image files and text descriptions into a structured multipart request according to the specific API requirements of the VLM.

[0077] Subsequently, the standardized task data is accurately sent to the target model. The response of the target model after processing these more challenging in-depth assessment tasks constitutes the second output data. This set of output data is the direct basis for evaluating the model's advanced capabilities in specific specialized fields, its ability to handle complex scenarios, and potential risks. Its depth and professionalism significantly surpass the first output data, providing core material for ultimately generating a comprehensive profile of the model's capabilities.

[0078] Step 107: Based on the first output data and the second output data, evaluate the target model and generate standardized evaluation results.

[0079] This step is the final integration stage of the evaluation process. After completing the breadth-based capability survey and in-depth specialized testing of the target model in the previous steps, this step does not simply add up the two sets of data. Instead, based on a unified evaluation framework and quantitative standards, it conducts a comprehensive analysis of the target model's performance at both levels, ultimately condensing it into a structured capability report.

[0080] The analysis process first places the target model's performance on the general assessment set (reflected by the first output data) and its performance on the specialized assessment set (reflected by the second output data) under the same measurement system for correlation interpretation. For example, for a target model that performs well in a certain specialty in the preliminary assessment, the in-depth assessment results will further verify whether this advantage is stable and reveal its specific upper limit in complex or high-risk scenarios; while for specialties that show weaknesses in the preliminary assessment, the in-depth assessment results are used to accurately diagnose the nature and severity of its weaknesses, such as whether it is a lack of basic knowledge, unfamiliarity with safety rules, or a flaw in clinical reasoning logic.

[0081] The evaluation process strictly adheres to a pre-defined scoring strategy (such as combining exact matching with large-scale model-based assessment) and a specialized indicator system, performing quantitative scoring and qualitative evaluation on both parts of the output data. All evaluation conclusions are compiled and organized and formatted according to a pre-defined, fixed report template, thereby generating standardized evaluation results. These results not only include a quantitative total score and ranking reflecting the model's overall capabilities but also emphasize its specialized capability profile, clearly indicating in which clinical areas the target model has reliable application potential and in which areas there are application risks or restrictions. Simultaneously, the report summarizes the most critical safety risks and core advantages identified during the evaluation process, enabling users to quickly grasp the core conclusions.

[0082] The resulting standardized evaluation results ensure comparability between evaluation conclusions from different models and batches, providing a direct and reliable basis for horizontal model comparison, selection decisions, or subsequent targeted optimization.

[0083] In an optional implementation, the target model is evaluated, including: The target model is evaluated based on at least one preset scoring strategy; the scoring strategy includes: an exact matching strategy based on keywords, fields or numerical values, and / or an LLM-as-Judge (Large Language Model as a Judge) strategy based on a third-party large model for semantic and logical evaluation.

[0084] This implementation plan defines the core scoring methodology used in the assessment process. These two strategies are not mutually exclusive, but rather should be selected or combined based on the specific nature of the assessment task.

[0085] The precise matching strategy is primarily designed for scenarios where the answer has clear standards and the output structure is well-defined. Essentially, it involves a direct and objective technical comparison between the target model's response and a pre-defined standard answer. For example, this strategy can be used to evaluate a model's ability to extract specific test result values ​​from medical record text, directly determining whether the values ​​extracted by the target model are completely consistent with the standard answer or within the allowable error range. This strategy is highly automated in its execution, and the evaluation results are objective and unambiguous.

[0086] The LLM-as-Judge strategy is applied to complex scenarios where answers are open-ended and require contextual reasoning. When the assessment task involves case analysis, explanation of treatment decisions, or evaluation of the completeness of diagnostic logic, precise literal matching is insufficient. In such cases, introducing a third-party large language model guided by professional prompts as a "judge" can simulate the judgment process of human experts. This judge model, based on evaluation criteria incorporating clinical thought processes and medical knowledge, performs semantic understanding, analysis, and comprehensive judgment on the target model's output, assessing the reasonableness, coherence, and clinical relevance of its answers. This strategy enables a deep assessment of the target model's high-level cognitive abilities.

[0087] By providing the above-mentioned optional scoring strategies, this implementation plan enables the assessment framework to flexibly address different types of assessment needs, ranging from simple factual questions and answers to complex clinical decision support, thereby ensuring that the assessment results reflect both the model's objective accuracy and its deep-seated clinical reasoning ability.

[0088] In an optional implementation, the tasks in the general assessment set and / or the specialty depth assessment task library include at least one of the following types: Text question answering tasks, medical image understanding tasks, medical speech transcription and analysis tasks, and information extraction tasks that extract specific fields from structured or unstructured medical text.

[0089] This implementation plan clarifies the scope of assessment tasks it can handle, reflecting its broad adaptability to the multimodal and multi-task processing capabilities of medical AI models. The diversity of task types ensures that the assessment can comprehensively examine the various data formats and processing requirements that the model may encounter in real-world medical scenarios.

[0090] Text-based question answering is a fundamental and core task, involving generating diagnoses, suggestions, or explanations from medical questions (such as medical history and symptoms) based on plain text descriptions. Medical image understanding tasks require the target model to process medical images (such as X-rays, CT scans, and pathology slides) and complete descriptions, anomaly identification, or answer related questions, testing its ability to combine visual perception with medical knowledge. Medical speech-to-text transcription and analysis tasks simulate a clinical consultation environment; the target model needs to first convert the recording of doctor-patient dialogue into text, and then further extract key information or analyze clinical intent from the transcribed content. Information extraction tasks focus on automatically locating and structurally extracting predefined entities or fields (such as drug names, surgical procedures, and laboratory indicator values) from massive amounts of medical documents in various formats (such as discharge summaries and laboratory reports), which is crucial for automating medical record data processing.

[0091] By supporting the above-mentioned multiple types of tasks, the evaluation framework constructed in this implementation plan can conduct integrated evaluation of multimodal medical AI models that integrate text, image, and speech processing capabilities, and can also conduct specialized capability assessments of dedicated models that focus on specific modalities or tasks, thereby providing a unified, fair and application-specific evaluation benchmark for different types of models.

[0092] In an optional implementation, the method further includes: In response to a user's custom data upload request, the system receives a custom medical assessment dataset provided by the user; performs format compliance verification on the custom medical assessment dataset; and, after passing the verification, incorporates the custom medical assessment dataset as a component of the general assessment set or the specialized in-depth assessment task library.

[0093] This implementation scheme provides the assessment process with a high degree of flexibility and user scalability, enabling it to overcome the limitations of pre-built assessment sets and adapt to specific user assessment needs or proprietary scenarios. When users need to examine the performance of a target model in its specific disease area of ​​interest, regional treatment patterns, or internal clinical data formats, pre-built general or specialty assessment sets may not be fully covered.

[0094] The process begins by responding to a user's upload request. Users can prepare and submit their custom datasets according to preset data format specifications. Subsequently, the dataset is automatically subjected to format compliance verification. This verification process does not assess the medical correctness of the data content, but rather ensures that its data structure, field definitions, encoding methods, etc., comply with the technical specifications that this method can process and parse. For example, it checks whether necessary metadata fields are complete, whether data fragments are complete, and whether the file format is supported.

[0095] Only after passing format validation will a user's custom dataset be securely incorporated into the assessment resource system. Based on the dataset's content characteristics and user specifications, it can be integrated into a general assessment set to broaden the coverage of model capability surveys; or it can be categorized into a specific clinical specialty's in-depth assessment task library to strengthen or supplement the specialty's specialized assessment capabilities. In this way, users can utilize their proprietary, high-quality clinical data resources to conduct more targeted assessments of the target model, tailored to their real-world application environments, thereby greatly enhancing the practical value and scenario adaptability of this assessment method.

[0096] Based on the same technical concept, embodiments of this application also provide a medical model evaluation device, such as... Figure 3 As shown, the device includes: Module 301 is used to acquire a universal assessment set covering multiple clinical specialties.

[0097] The first conversion module 302 is used to standardize and convert the tasks in the general evaluation set based on the interface protocol of the target model to be evaluated through the data adaptation layer, and input the converted tasks into the target model to obtain the first output data of the target model.

[0098] The verification module 303 is used to perform a pre-security verification on the first output data.

[0099] The first evaluation module 304 is used to perform a preliminary evaluation on the target model that has passed the pre-security verification based on the first output data, and to obtain the capability distribution data of the target model in each of the clinical specialties.

[0100] The task determination module 305 is used to determine, based on the capability distribution data, a set of specialized assessment tasks suitable for assessing the target model from a preset specialized in-depth assessment task library; wherein, in-depth assessment tasks of different difficulty or different coverage are assigned to clinical specialties with different capabilities of the target model.

[0101] The second conversion module 306 is used to standardize and convert the tasks in the special evaluation set through the data adaptation layer, and input the converted tasks into the target model to obtain the second output data of the target model.

[0102] The second evaluation module 307 is used to evaluate the target model based on the first output data and the second output data, and generate standardized evaluation results.

[0103] In one feasible implementation, the verification module is configured to perform a pre-security verification on the first output data, for the following purposes: A preset medical knowledge base is invoked to scan the first output data; wherein, the medical knowledge base is configured with disabled processing items for the target population.

[0104] The device further includes: The marking module is used to mark the target model as failing the verification if the first output data is found to match any of the disabled processing items in the medical knowledge base; otherwise, it is marked as passing the verification.

[0105] In one feasible implementation, the target model to be evaluated includes at least one of the following types: text-based large language model LLM, visual language model VLM, medical optical character recognition (OCR) model, and medical automatic speech recognition (ASR) model.

[0106] In one feasible implementation, the first transformation module is used to standardize and transform the tasks in the general evaluation set based on the interface protocol of the target model to be evaluated through a data adaptation layer, for the following purposes: Identify the interface protocol type of the target model.

[0107] Select the target conversion logic that is compatible with the interface protocol type from a variety of preset data conversion logics.

[0108] Based on the target transformation logic, the task data in the general evaluation set is converted into standardized requests that conform to the target model input format.

[0109] In one feasible implementation, the first evaluation module is used to evaluate the target model, for the following purposes: The target model is evaluated based on at least one preset scoring strategy; the scoring strategy includes: an exact matching strategy based on keywords, fields or numerical values, and / or an LLM-as-Judge strategy based on semantic and logical evaluation of a third-party large model.

[0110] In one feasible implementation, the tasks in the general assessment set and / or the specialty depth assessment task library include at least one of the following types: Text question answering tasks, medical image understanding tasks, medical speech transcription and analysis tasks, and information extraction tasks that extract specific fields from structured or unstructured medical text.

[0111] In one feasible implementation, the device further includes: The response module is used to respond to users' custom data upload requests and receive custom medical assessment datasets provided by users.

[0112] The format verification module is used to verify the format compliance of the custom medical assessment dataset.

[0113] The dataset composition module is used to incorporate the custom medical assessment dataset as a component of the general assessment set or the specialty in-depth assessment task library after successful verification.

[0114] This application provides a medical model evaluation method, apparatus, electronic device, and storage medium. First, it uses a universal evaluation set covering multiple clinical specialties to conduct a preliminary capability assessment of the target model. A data adaptation layer standardizes the evaluation task, enabling models with different interface protocols to quickly connect and complete testing, solving the problems of high access costs and slow startup caused by fragmented evaluation frameworks in existing technologies. By performing pre-emptive security checks on the first output data of the target model, outputs with serious clinical safety risks can be identified and filtered early in the evaluation process, compensating for the lack of clinical safety considerations in universal evaluation methods.

[0115] Secondly, based on the capability distribution data obtained from the preliminary assessment, specialized assessment sets are dynamically determined from a pre-defined database of in-depth specialty assessment tasks. According to the differences in the initial performance of the target model across various clinical specialties, in-depth assessment tasks of corresponding difficulty and coverage are assigned to different specialties, achieving optimized allocation of assessment resources. Compared with the "one-size-fits-all" assessment method using a fixed test set in existing technologies, this method can conduct more targeted in-depth validation of the target model's strengths and weaknesses with the same or less resource consumption.

[0116] Finally, by executing customized specialized assessment sets and integrating the first output data with the second output data obtained from the specialized assessments, standardized assessment results are generated. This process reflects both the model's breadth of capabilities across multiple specialties and its depth of performance in specific domains, providing a unified and reliable basis for cross-sectional comparisons of the clinical capabilities of different models.

[0117] Figure 4 A schematic diagram of an electronic device provided in this application embodiment includes: a processor 401, a storage medium 402, and a bus 403. The storage medium 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the medical model evaluation method as described in the embodiment, the processor 401 communicates with the storage medium 402 via the bus 403, and the processor 401 executes the machine-readable instructions to perform the steps as described in the embodiment.

[0118] In this embodiment, the storage medium 402 may also execute other machine-readable instructions to perform other methods as described in the embodiment. For details on the specific execution steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0119] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform the steps as described in the embodiments.

[0120] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0122] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0124] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0125] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating a medical model, characterized in that, The method includes: Obtain a universal assessment set covering multiple clinical specialties; Based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through the data adaptation layer, and the transformed tasks are input into the target model to obtain the first output data of the target model; Perform a pre-security check on the first output data; Based on the first output data, a preliminary evaluation is performed on the target model that has passed the pre-security verification to obtain the capability distribution data of the target model in each of the clinical specialties. Based on the capability distribution data, a set of specialized assessments suitable for assessing the target model is determined from a pre-set database of in-depth assessment tasks for specific specialties; wherein, in-depth assessment tasks of different difficulty or coverage are assigned to clinical specialties with different capabilities of the target model. The data adaptation layer standardizes and transforms the tasks in the special assessment set, and the transformed tasks are input into the target model to obtain the second output data of the target model. Based on the first output data and the second output data, the target model is evaluated, and standardized evaluation results are generated.

2. The method according to claim 1, characterized in that, The pre-security check performed on the first output data includes: The system invokes a preset medical knowledge base to scan the first output data; wherein the medical knowledge base contains disabled processing items for the target population. The method further includes: If the first output data is found to match any of the disabled processing items in the medical knowledge base, the target model is marked as failing the verification; otherwise, it is marked as passing the verification.

3. The method according to claim 1, characterized in that, The target models to be evaluated include at least one of the following types: text-based large language model (LLM), visual language model (VLM), medical optical character recognition (OCR) model, and medical automatic speech recognition (ASR) model.

4. The method according to claim 3, characterized in that, Based on the interface protocol of the target model to be evaluated, the tasks in the general evaluation set are standardized and transformed through a data adaptation layer, including: Identify the interface protocol type of the target model; Select the target conversion logic that is compatible with the interface protocol type from a variety of preset data conversion logics; Based on the target transformation logic, the task data in the general evaluation set is converted into standardized requests that conform to the target model input format.

5. The method according to claim 1, characterized in that, The target model is evaluated, including: The target model is evaluated based on at least one preset scoring strategy; the scoring strategy includes: an exact matching strategy based on keywords, fields or numerical values, and / or an LLM-as-Judge strategy based on semantic and logical evaluation of a third-party large model.

6. The method according to claim 1, characterized in that, The general assessment set and / or the tasks in the specialty depth assessment task library include at least one of the following types: Text question answering tasks, medical image understanding tasks, medical speech transcription and analysis tasks, and information extraction tasks that extract specific fields from structured or unstructured medical text.

7. The method according to claim 1, characterized in that, The method further includes: In response to a user's custom data upload request, receive the user-provided custom medical assessment dataset; Perform format compliance verification on the custom medical assessment dataset; After successful verification, the custom medical assessment dataset will be incorporated into the general assessment set or the specialty in-depth assessment task library.

8. A medical model evaluation device, characterized in that, The device includes: The acquisition module is used to acquire a universal assessment set covering multiple clinical specialties; The first conversion module is used to standardize and convert the tasks in the general evaluation set based on the interface protocol of the target model to be evaluated through the data adaptation layer, and input the converted tasks into the target model to obtain the first output data of the target model. The verification module is used to perform pre-security verification on the first output data; The first evaluation module is used to perform a preliminary evaluation of the target model that has passed the pre-security verification based on the first output data, and to obtain the capability distribution data of the target model in each of the clinical specialties. The task determination module is used to determine, based on the capability distribution data, a set of specialized assessment tasks suitable for assessing the target model from a preset specialized in-depth assessment task library; wherein, in-depth assessment tasks of different difficulty or different coverage are assigned to clinical specialties with different capabilities of the target model. The second conversion module is used to standardize and convert the tasks in the special evaluation set through the data adaptation layer, and input the converted tasks into the target model to obtain the second output data of the target model. The second evaluation module is used to evaluate the target model based on the first output data and the second output data, and generate standardized evaluation results.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the medical model evaluation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the medical model evaluation method as described in any one of claims 1 to 7.

Images