Medical AI model intelligent arbitration gray release method and device and medium

Abstract

This invention proposes a method, device, and medium for intelligent arbitration canary deployment of medical AI models, aiming to solve the problems of limited arbitration information, lack of scenario adaptability, and disconnect between the verification process and clinical decision-making in existing canary deployments of medical AI models. The method distributes case requests in parallel to both new and old models for inference according to a canary deployment ratio; constructs comprehensive features based on the differences in outputs of the new and old models, the multimodal fusion features of the original input data, and clinical context information; inputs these features into the arbitration decision model to calculate the arbitration score; dynamically obtains a safety threshold based on the clinical context; and compares the arbitration score with the safety threshold to determine the final decision: adopt the new model, adopt the old model, or trigger manual review. This invention achieves accurate, safe, and efficient online verification and controllable deployment of medical AI models.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical information processing technology, and in particular to a method, apparatus and medium for intelligent arbitration grayscale release of medical AI models based on multimodal fusion and clinical context awareness. Background Technology

[0002] With the deepening application of artificial intelligence technology in the medical field, medical AI models such as pathology-assisted diagnosis and image analysis have become important tools for improving diagnostic efficiency and consistency. These models need to be continuously iterated and optimized based on new data and knowledge to maintain and improve their performance.

[0003] Currently, medical AI models are mainly deployed in the following two ways:

[0004] The first approach is a full switch after offline validation. This method validates the new model in an offline testing environment, and once validation is successful, the new model directly replaces the old model and goes live. This approach is simple and straightforward, but due to differences in data distribution and application scenarios between the offline testing environment and the real clinical production environment, unforeseen performance degradation or errors may occur after the new model goes live, leading to interruptions in clinical diagnostic services or incorrect diagnostic recommendations.

[0005] The second approach is shadow model testing. This method deploys the new model in a production environment, receiving the same real-world case input data as the old model for inference. However, the new model's output does not participate in actual clinical decision-making; it is only used for offline comparative analysis. While this approach does not pose a direct risk to clinical decision-making, the new model's output has not been validated through a real-world decision chain. Therefore, it is impossible to assess whether the differences between its output and existing clinical standards or the results of the old model are clinically acceptable, resulting in low validation efficiency and inadequacy.

[0006] Furthermore, in the field of general software and AI services, there are already canary release methods based on phased traffic switching, health monitoring, and rapid rollback mechanisms. These methods typically make decisions based on simple metrics such as response time and error rate, or simple differences between the outputs of new and old versions.

[0007] Therefore, there is an urgent need for a method, device, and medium for intelligent arbitration grayscale release of medical AI models to solve the problems existing in the current technology. Summary of the Invention

[0008] This invention provides a method, device, and medium for intelligent arbitration grayscale release of medical AI models. It addresses the problems of existing technologies, which suffer from the inability to fully utilize multimodal inputs and clinical context information due to the single source of arbitration decision information and the use of static thresholds. This results in the inability to adapt to complex and ever-changing medical scenarios and the difficulty in balancing model validation efficiency and clinical safety.

[0009] The core technology of this invention is to construct a comprehensive feature that integrates the differences between the outputs of the old and new models, the original multimodal features, and clinical context information, and to use an arbitration decision model to compare the outputs of the old and new models with dynamic safety thresholds that are tied to the risks of the clinical scenario, thereby achieving a three-level adaptive arbitration decision and closed-loop iteration for the outputs of the old and new models and manual review.

[0010] In a first aspect, the present invention provides a method for intelligent arbitration canary release of a medical AI model, the method comprising the following steps: Upon receiving clinical case reasoning requests, the inference requests are distributed in parallel to the online running old medical AI model and the new medical AI model to be verified according to a preset gray-scale traffic ratio, and the outputs of the old model and the new model are obtained respectively. Based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data corresponding to the inference request, and the clinical context information corresponding to the inference request, a comprehensive feature is constructed. By inputting comprehensive features into a pre-trained arbitration decision model, an arbitration score is calculated to represent the tendency to adopt the new model's output. The dynamic threshold mapping table is queried based on clinical context information to obtain a safety threshold that matches the current clinical scenario. The arbitration score is compared with a safety threshold, and the final arbitration decision is determined based on the comparison result: adopt the new model output, adopt the old model output, or trigger a manual review.

[0011] Furthermore, the final arbitration decision is determined based on the comparison results, including: If the arbitration score is greater than or equal to the safety threshold, the arbitration decision is determined to adopt the new model output; If the arbitration score is less than the safety threshold and lower than the preset conservative threshold, the arbitration decision is determined to adopt the old model output. If the arbitration score is less than the safety threshold but greater than or equal to the conservative threshold, a manual review process is triggered, and the corresponding case is marked as pending review.

[0012] Furthermore, based on the difference information between the old model output and the new model output, including: Extract comparable key fields from the outputs of the old model and the new model respectively, calculate the difference measure between the key fields, and generate a difference vector; Key fields include at least one of the following: classification label code, confidence score, and lesion region coordinates.

[0013] Furthermore, the multimodal fusion features are obtained in the following way: A pre-trained feature extraction network is used to extract features from data of different modalities in the original input data to obtain feature representations for each modality. The feature representations of each modality are fused and their dimensionality reduced to generate a fixed-dimensional multimodal fusion feature vector; Data of different modalities includes at least medical imaging data and text report data.

[0014] Furthermore, clinical context information is obtained by encoding the structured clinical metadata corresponding to the reasoning request; the structured clinical metadata includes at least one of the following: department code, disease label, urgency level, and patient age group.

[0015] Furthermore, based on clinical context information, a dynamic threshold mapping table is queried to obtain a safety threshold matching the current clinical scenario, including: Using the encoded clinical context information as an index, the pre-configured dynamic threshold mapping table is queried to obtain the corresponding safety threshold. Different clinical scenarios correspond to different safety thresholds, with the safety threshold for high-risk clinical scenarios being higher than that for low-risk clinical scenarios.

[0016] Furthermore, it also includes: The input data identifiers, old model outputs, new model outputs, comprehensive features, arbitration scores, safety thresholds, and arbitration decision results of each arbitration process are recorded in a structured manner to construct a grayscale verification database. Based on historical data accumulated in the grayscale verification database and expert review results after manual review, the arbitration decision model is retrained, and / or the strategy for obtaining the security threshold is updated.

[0017] Secondly, this invention provides a medical AI model intelligent arbitration grayscale release device, comprising: The traffic scheduling module is used to receive clinical case inference requests and distribute the inference requests in parallel to the old medical AI model running online and the new medical AI model to be verified according to the preset gray traffic ratio, and obtain the output of the old model and the output of the new model respectively. The feature construction module is used to construct comprehensive features based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data corresponding to the inference request, and the clinical context information corresponding to the inference request; The arbitration scoring module is used to input comprehensive features into a pre-trained arbitration decision model and calculate an arbitration score that represents the tendency to adopt the new model output. The dynamic threshold module is used to query the dynamic threshold mapping table based on clinical context information and obtain a safe threshold that matches the current clinical scenario. The decision module compares the arbitration score with a safety threshold and determines the final arbitration decision based on the comparison result: adopt the new model output, adopt the old model output, or trigger a manual review.

[0018] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described medical AI model intelligent arbitration grayscale release method.

[0019] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the above-described intelligent arbitration grayscale release method for the medical AI model.

[0020] The main contributions and innovations of this invention are as follows: 1. This invention integrates the differences in outputs between old and new models, the deep fusion features of raw multimodal data, and structured clinical context information into a comprehensive feature vector as the basis for arbitration. This enables arbitration decisions to understand the clinical meaning behind the output differences, effectively distinguish between "acceptable technical differences" and "unacceptable clinical risk differences," and significantly reduce misjudgments.

[0021] 2. This invention introduces a dynamic safety threshold mechanism that is bound to the clinical context, enabling the strictness of gray-scale release to be adjusted in real time according to the risk level of the case: in high-risk clinical scenarios, a higher threshold is automatically adopted to implement a conservative strategy, while in low-risk scenarios, a lower threshold is adopted to accelerate the validation of the new model, thereby achieving a balance between ensuring patient safety and improving validation efficiency.

[0022] 3. This invention records complete information on all arbitration cases, forming a grayscale verification database. Based on the expert review results after manual review, the arbitration decision-making model is retrained and the threshold strategy is updated, enabling the system to continuously self-optimize as the clinical environment changes.

[0023] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a system architecture diagram of the intelligent arbitration canary release method for medical AI models according to an embodiment of the present invention; Figure 2 This is a flowchart of a method for intelligent arbitration grayscale release of a medical AI model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0026] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0027] like Figure 1 As shown, the system architecture built on the medical AI inference server in this invention includes the following core components: an online-running legacy medical AI model (denoted as Model_v1), a new medical AI model to be validated (denoted as Model_v2), an intelligent arbitration layer (including an arbitration decision model, a dynamic threshold mapping table, and supporting processing modules), a traffic scheduling component, a feature extraction module, and a validation database. Model_v1 and Model_v2 are deployed in parallel, and their input / output interfaces remain compatible. The intelligent arbitration layer sits on top of the two models and is responsible for making arbitration decisions by combining multimodal features and clinical context information. The validation database is used to store complete arbitration process data, supporting closed-loop monitoring and optimization.

[0028] Example 1 This invention aims to propose a method for intelligent arbitration and canary release of medical AI models. Specifically, refer to... Figure 2 The method specifically includes the following steps: Step S101: Receive clinical case reasoning requests, distribute them in parallel to the old and new models according to the preset grayscale ratio, and obtain the output of the old model and the output of the new model respectively.

[0029] The system receives clinical case reasoning requests from clinical workstations or PACS systems. These requests include patient IDs, multimodal data, and clinical context metadata. The multimodal data includes at least medical imaging data (such as DICOM format CT, MRI, or X-ray images) and text report data (such as pathology reports, imaging reports, etc.). The clinical context metadata consists of structured tagging information, including at least department code, disease label, urgency level, and patient age group.

[0030] The traffic scheduling component determines whether a current case request should enter the gray-scale process based on a preset gray-scale traffic ratio (e.g., 5%) using a random algorithm or a consistent hashing algorithm. If it is determined to enter the gray-scale process, the case request is copied and distributed in parallel to the online stable old medical AI model Model_v1 and the new medical AI model Model_v2 to be verified; if it is determined not to enter the gray-scale process, the case request is only sent to the old medical AI model.

[0031] Model_v1 and Model_v2 each invoke their respective inference engines (such as TensorRT or ONNXRuntime), inputting preprocessed multimodal data to obtain the old model output. and the output of the new model . and All data are structured and contain at least classification labels, confidence scores, and lesion location coordinates.

[0032] Step S102: Construct comprehensive features based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data, and clinical context information.

[0033] This step extracts and integrates information from three dimensions.

[0034] First, the difference information is calculated. The system starts from... and Extract comparable key fields, including the one-hot encoded vector of the classification label. Confidence score lesion boundary coordinates Etc., generate difference vectors :

[0035] in, , These represent the one-hot encoded vectors of the classification labels output by the old and new models, respectively, with values ​​ranging from 1 to 2. , Total number of categories; , These represent the confidence scores of the old model and the new model for the final adopted label, respectively, with values ​​ranging from [value range missing]. ; The intersection-union ratio (IU) distance between lesion bounding boxes is calculated using the following formula: The range of values ​​is ;symbol" "Represents absolute value or element-wise absolute value of a vector;" "Indicates other variance measures that can be extended based on specific tasks."

[0036] Secondly, multimodal fusion features are extracted. The feature extraction module receives the raw multimodal data and uses a pre-trained feature extraction network to extract features from different modalities: for medical image data, a convolutional neural network (such as ResNet) is used to extract image features; for text report data, a pre-trained language model (such as BERT) is used to extract text features. The extracted image and text features are concatenated and then dimensionality-reduced using a fully connected layer to generate... Multimodal fusion feature vector .

[0037] Finally, clinical context information is obtained. The clinical context parsing function encodes structured label information such as department code, disease label, urgency level, and patient age group into... Dimensional clinical context vector .

[0038] Difference vector Multimodal fusion feature vector and clinical context vectors Concatenate the data in a specified order to generate a comprehensive feature vector. .

[0039] Step S103: Input the comprehensive features into the pre-trained arbitration decision model and calculate the arbitration score used to characterize the tendency to adopt the new model output.

[0040] Arbitration decision function based on comprehensive feature vector Using the input as input, forward propagation computation is performed. In this embodiment, the arbitration decision model employs a multilayer perceptron network, containing one hidden layer and one output layer. The arbitration score is calculated as follows:

[0041] in, For arbitration scores, scalars, and their range of values. A higher score indicates a greater tendency to adopt the new model's output; This represents the Sigmoid activation function. ; This indicates a modified linear unit activation function. ; These are the weight matrix and bias vector of the hidden layer; These are the weight vector and bias scalar of the output layer.

[0042] Step S104: Dynamically obtain a safety threshold that matches the current clinical scenario based on the clinical context information.

[0043] The system uses clinical context vectors As an index, query the pre-configured dynamic threshold mapping table in memory to obtain the safety threshold that matches the current clinical scenario. The range of values ​​is If no exact match is found, the nearest neighbor matching strategy will be used or the default value will be returned.

[0044] For example, when the clinical context is coded as (Department = Emergency, Disease = Stroke, Urgency = High), It can be set to 0.9 for a more stringent strategy; when the clinical context coding is (department = dermatology, disease = common nevus, urgency = low), It can be set to 0.6 to adopt a relatively lenient strategy.

[0045] Step S105: Compare the arbitration score with the safety threshold, and determine the final arbitration decision based on the comparison result: adopt the new model output, adopt the old model output, or trigger manual review.

[0046] The decision logic function receives the arbitration score. Safety threshold and preset conservative threshold (In this embodiment) ).because The range of values ​​is and The final decision is generated according to the following rules: 1. If Then the arbitration decision is determined to be the adoption of the new model output. ; 2. If and If so, the arbitration decision is to adopt the output of the old model. ; 3. If and If so, a manual review process will be triggered, and the corresponding case will be marked as pending review.

[0047] The above three branches of logic constitute a... Throughout the entire domain The complete division ensures that there is a corresponding decision output for each situation.

[0048] Step S106: Data recording and closed-loop optimization.

[0049] The system stores complete information from each arbitration process in a structured format in a grayscale verification database, including: input data identifiers, , , , , , , , And the final decision outcome.

[0050] Regularly run monitoring and analysis tasks to extract data from the validation database and calculate metrics such as new model adoption rate, manual review trigger rate, and consistency rate between arbitration decisions and manual review results. Based on the changing trends of the monitoring metrics, the proportion of gray-scale traffic can be dynamically adjusted (e.g., gradually increase the gray-scale proportion when the new model adoption rate is consistently higher than the preset threshold and the review trigger rate is lower than the preset threshold), and guide the optimization of arbitration strategies.

[0051] When enough new data is accumulated, especially data labeled with clinical gold standards after manual review, the retraining process of the arbitration decision model is triggered. Using the new training data, the model parameters are fine-tuned and updated with the cross-entropy loss function, and the online model file is replaced. Simultaneously, based on clinical feedback and incident analysis results, the values ​​in the dynamic safety threshold mapping table are reviewed and updated, and the updated configuration is hot-loaded into the arbitration service.

[0052] In this embodiment, the arbitration decision model is pre-trained based on historical ruling data or simulated data. The training process is as follows: Constructing training data: Collect or simulate a large number of clinical case reasoning scenarios, obtain the reasoning outputs of the old model and the new model for each case, and construct a comprehensive feature vector according to the method in step S102. As input features; clinical experts label each case with the gold standard label for arbitration decision-making (adopting the new model, adopting the old model, or requiring manual review), which serves as the target label.

[0053] Training objective: To maximize the consistency between the output of the arbitration decision model and the clinical gold standard, the model parameters are optimized using the cross-entropy loss function.

[0054] in, The number of training samples. The decision category is 1 (1 represents adopting the new model, 2 represents adopting the old model, and 3 represents triggering manual review). For the sample In category The actual label on the screen (with a value of 0 or 1). The predicted category for the model The probability value. The weight parameters are updated using the backpropagation algorithm and gradient descent optimizer. This continues until the model's decision accuracy on the validation set converges.

[0055] Example 2 The difference between this embodiment and Embodiment 1 lies in the type of arbitration decision model. In this embodiment, instead of the multilayer perceptron model in Embodiment 1, a gradient boosting decision tree (GBDT) model is used as the arbitrator. The input to this ensemble model is also a comprehensive feature vector. The output is a calibrated probability score used as the arbitration score. Its value range, after probability calibration, is also mapped to Ensemble learning models are more robust to changes in feature distribution and can provide better interpretability through feature importance analysis.

[0056] The remaining steps are the same as in Example 1, and will not be repeated here.

[0057] Example 3 The difference between this embodiment and Embodiment 1 lies in the method of obtaining the dynamic security threshold. In this embodiment, instead of the lookup table method in Embodiment 1, a lightweight threshold prediction model is used to obtain the dynamic security threshold. Specifically, a small regression model (such as linear regression or a shallow neural network) is constructed to incorporate clinical context vectors. As input, directly predict the safety threshold for the current scenario. This threshold prediction model can be pre-trained based on historical clinical data, making the determination of the safety threshold smoother and more continuous, avoiding the accuracy loss caused by discretization in the lookup table method.

[0058] This embodiment can be used in combination with Embodiment 1 or Embodiment 2, meaning that the arbitration decision model and threshold acquisition method can be replaced independently.

[0059] The remaining steps are the same as in Example 1, and will not be repeated here.

[0060] Example 4 The difference between this embodiment and Embodiment 1 lies in the execution method of the arbitration decision. In this embodiment, instead of a single arbitration decision model, multiple expert sub-modules focusing on different dimensions are designed for collaborative arbitration. Specifically, three expert sub-modules are set up: The difference analysis expert module receives difference vectors. Output difference risk score This characterizes the severity of the decision-making conflict between the old and new models; the context risk assessment expert module receives clinical context vectors. Output scenario risk coefficient It characterizes the risk level of the current scenario; the multimodal consistency expert module receives the multimodal fusion feature vector. The outputs of the old and new models, and the consistency score of the outputs. .

[0061] The arbitration score is calculated using the weighted aggregation rule:

[0062] in, The normalized weighted coefficients satisfy... The weighting coefficients can be adjusted based on clinical feedback. The weighted aggregation results... Perform normalization and map it to After the interval, the same dynamic safety threshold as in Example 1. Compare them to make a final decision.

[0063] When the context risk assessment expert module detects that the current case belongs to a high-risk scenario, a "one-vote veto" rule can be activated, meaning that as long as... If the value falls below the preset safety limit, regardless of the overall score, the final decision will be to trigger manual review or adopt the old model output.

[0064] The remaining steps are the same as in Example 1, and will not be repeated here.

[0065] Example 5 The difference between this embodiment and Embodiment 1 lies in the optimization method of the arbitration strategy. In this embodiment, a reinforcement learning online learning framework is introduced instead of a static arbitration decision model and a fixed threshold mapping table.

[0066] The arbitration decision-making process is modeled as a Markov decision process: the state space is defined as the current comprehensive feature vector. The action space is defined as three discrete actions (adopting the new model, adopting the old model, and triggering manual review); the reward function... Defined as a comprehensive assessment based on long-term returns:

[0067] in, The efficiency metric for model validation is quantified as the ratio of the number of correct adoptions of the new model to the total number of adoptions within the current statistical period, with a value range of [range missing]. ; This is a clinical physician satisfaction indicator, quantified by the physician's acceptance rate of the arbitration result after manual review, with a value range of... ; The indicator is the accident-free operating time, which is quantified by the normalized continuous arbitration operating time without major errors. This is a penalty item, negatively correlated with the number of clinical diagnostic accidents; These are the weighting coefficients for each dimension.

[0068] The Proximal Policy Optimization (PPO) algorithm is employed to fine-tune the parameters of the arbitration decision-making model online or dynamically adjust the threshold mapping strategy. After each arbitration decision, the system calculates an immediate reward based on subsequent manual review results or clinical feedback, and updates the strategy network parameters to enable the system to continuously adapt to changes in the clinical environment.

[0069] The remaining steps are the same as in Example 1, and will not be repeated here.

[0070] Example 6 This embodiment provides a medical AI model intelligent arbitration canary release device based on multimodal fusion and clinical context awareness, used to implement any one of the methods in Embodiments 1 to 5 above. The device includes: The traffic scheduling module is used to receive clinical case inference requests and distribute the inference requests in parallel to the old medical AI model running online and the new medical AI model to be verified according to the preset gray-scale traffic ratio, so as to obtain the output of the old model and the output of the new model respectively.

[0071] The feature construction module is used to construct comprehensive features based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data corresponding to the inference request, and the clinical context information corresponding to the inference request.

[0072] The arbitration scoring module is used to input comprehensive features into a pre-trained arbitration decision model and calculate an arbitration score that represents the tendency to adopt the new model's output.

[0073] The dynamic threshold module is used to query the dynamic threshold mapping table based on clinical context information to obtain a safe threshold that matches the current clinical scenario.

[0074] The decision module compares the arbitration score with a safety threshold and determines the final arbitration decision based on the comparison result: adopt the new model output, adopt the old model output, or trigger a manual review.

[0075] The data recording module is used to structurally record the input data identifiers, old model outputs, new model outputs, comprehensive features, arbitration scores, safety thresholds, and arbitration decision results for each arbitration process, build a grayscale verification database, and retrain the arbitration decision model or update the threshold acquisition strategy based on the accumulated data and expert review results.

[0076] Example 7 This embodiment also provides an electronic device, see reference. Figure 3 It includes a memory 402 and a processor 401, the memory 402 storing a computer program and the processor 401 being configured to run the computer program to perform the steps in any of the above method embodiments.

[0077] Specifically, the processor 401 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0078] The memory 402 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 402 may include removable or non-removable (or fixed) media. Where appropriate, the memory 402 may be internal or external to a data processing device. In a particular embodiment, the memory 402 is non-volatile memory. In a particular embodiment, the memory 402 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0079] The memory 402 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 401.

[0080] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the medical AI model intelligent arbitration canary release methods in the above embodiments.

[0081] Optionally, the electronic device may further include a transmission device 403 and an input / output device 404, wherein the transmission device 403 is connected to the processor 401 and the input / output device 404 is connected to the processor 401.

[0082] The transmission device 403 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 403 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0083] Input / output device 404 is used for inputting or outputting information. It can be a speaker, microphone, monitor, or keyboard.

[0084] Example 8 This embodiment also provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the medical AI model intelligent arbitration grayscale release method according to Embodiment 1.

[0085] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0086] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0087] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 2 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0088] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for the intelligent arbitration gray-scale release of a medical AI model, characterized in that, Includes the following steps: Upon receiving a clinical case reasoning request, the reasoning request is distributed in parallel to the online running old medical AI model and the new medical AI model to be verified according to a preset grayscale traffic ratio, and the output of the old model and the output of the new model are obtained respectively. Based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data corresponding to the inference request, and the clinical context information corresponding to the inference request, a comprehensive feature is constructed. The comprehensive features are input into the pre-trained arbitration decision model to calculate the arbitration score used to characterize the tendency to adopt the new model output; Based on the clinical context information, query the dynamic threshold mapping table to obtain a safety threshold that matches the current clinical scenario; The arbitration score is compared with the security threshold, and the final arbitration decision is determined based on the comparison result: adopt the new model output, adopt the old model output, or trigger a manual review.

2. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, The arbitration score is compared with the security threshold, and the final arbitration decision is determined based on the comparison result, including: If the arbitration score is greater than or equal to the safety threshold, the arbitration decision is determined to be to adopt the new model output; If the arbitration score is less than the safety threshold and lower than the preset conservative threshold, the arbitration decision result is determined to adopt the old model output. If the arbitration score is less than the safety threshold but greater than or equal to the conservative threshold, a manual review process is triggered, and the corresponding case is marked as pending review.

3. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, Based on the difference information between the old model output and the new model output, including: Comparable key fields are extracted from the output of the old model and the output of the new model, respectively. The difference measure between the key fields is calculated, and a difference vector is generated. The key fields include at least one of the following: classification label code, confidence score, and lesion region coordinates.

4. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, The multimodal fusion features are obtained through the following methods: The pre-trained feature extraction network is used to extract features from the data of different modalities in the original input data to obtain the feature representation of each modality. The feature representations of each modality are fused and their dimensionality reduced to generate a fixed-dimensional multimodal fusion feature vector; The data of different modalities include at least medical imaging data and text report data.

5. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, The clinical context information is obtained by encoding the structured clinical metadata corresponding to the reasoning request; the structured clinical metadata includes at least one of the following: department code, disease label, urgency level, and patient age group.

6. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, Based on the clinical context information, a dynamic threshold mapping table is queried to obtain a safety threshold matching the current clinical scenario, including: Using the encoded clinical context information as an index, the pre-configured dynamic threshold mapping table is queried to obtain the corresponding safety threshold. Different clinical scenarios correspond to different safety thresholds, with the safety threshold for high-risk clinical scenarios being higher than that for low-risk clinical scenarios.

7. The method for intelligent arbitration and gray-scale release of medical AI models as described in claim 1, characterized in that, Also includes: The input data identifiers, old model outputs, new model outputs, comprehensive features, arbitration scores, safety thresholds, and arbitration decision results of each arbitration process are recorded in a structured manner to construct a grayscale verification database. Based on the historical data accumulated in the grayscale verification database and the expert review results after manual review, the arbitration decision model is retrained, and / or the strategy for obtaining the security threshold is updated.

8. A medical AI model intelligent arbitration grayscale release device, characterized in that, include: The traffic scheduling module is used to receive clinical case inference requests, distribute the inference requests in parallel to the old medical AI model running online and the new medical AI model to be verified according to a preset gray-scale traffic ratio, and obtain the output of the old model and the output of the new model respectively. The feature construction module is used to construct comprehensive features based on the difference information between the old model output and the new model output, the multimodal fusion features of the original input data corresponding to the inference request, and the clinical context information corresponding to the inference request; The arbitration scoring module is used to input the comprehensive features into the pre-trained arbitration decision model and calculate the arbitration score used to characterize the tendency to adopt the new model output; The dynamic threshold module is used to query the dynamic threshold mapping table based on the clinical context information to obtain a safety threshold that matches the current clinical scenario. The decision module is used to compare the arbitration score with the security threshold, and determine the final arbitration decision result based on the comparison result: adopt the new model output, adopt the old model output, or trigger manual review.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the medical AI model intelligent arbitration grayscale release method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the intelligent arbitration grayscale release method for medical AI models according to any one of claims 1 to 7.

Images