Multimodal AI Model Fine-Tuning for Iterative Clinical Workflows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AI models for medical imaging are unable to iteratively request specific additional information necessary to achieve a given task, unable to communicate with various healthcare subsystems to obtain or modify medical information, and are limited to processing fixed inputs, failing to explore a more open medical data space defined by healthcare subsystems.
Innovation Solution
A multimodal generative AI model is developed to interact with healthcare subsystems, iteratively process medical content, perform actions, and accomplish multi-action tasks by generating internal state prompts, converting them into executable commands, and updating the state prompt with modified content, enabling interaction with multiple three-dimensional datasets over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional AI models process fixed inputs, then processing simplicity is maintained, but adaptability to different healthcare subsystems and iterative data exploration is lost
Solution Approach 1:
The system transitions from static fixed-input processing to dynamic iterative processing where the AI model can adaptively adjust its data exploration strategy based on task requirements and subsystem responses, enabling flexible interaction with multiple healthcare subsystems
Solution Approach 2:
The AI model is designed with universal capabilities to interact with multiple types of healthcare subsystems (EHR, PACS, laboratory systems, etc.) through a unified interface, allowing it to perform diverse functions across different data modalities and subsystems
2Productivity
If AI models are limited to fixed inputs, then processing speed is maintained, but ability to iteratively request additional information is lost
Solution Approach 1:
The system performs preliminary actions by proactively requesting and retrieving necessary medical data from multiple subsystems before task completion, reducing the need for repeated iterative queries and accelerating overall task execution
Solution Approach 2:
The AI model maintains continuous useful action by seamlessly orchestrating iterative data retrieval, processing, and analysis across multiple subsystems without idle waiting periods, ensuring productive use of time throughout the task execution
3Adaptability or versatility
If AI models cannot communicate with healthcare subsystems, then system simplicity is preserved, but capability to obtain or modify medical information is lost
Solution Approach 1:
The AI model serves as an intermediary that coordinates communication between the user task requirements and multiple healthcare subsystems, translating high-level task goals into specific data retrieval and manipulation operations across EHR, PACS, and other medical systems
Data Source
AI summary
A method, computer program product, and computing system for generating an internal state prompt with medical content and a multi-action task to perform on a healthcare system. A first output healthcare system command is generated by processing the internal state prompt using a trained multimodal generative artificial intelligence (AI) model. The first output healthcare system command is converted into a first healthcare system-executable command associated with the multi-action task for a first target healthcare subsystem. Modified medical content is generated by executing the first healthcare system-executable command on the medical content using the first target healthcare subsystem. The internal state prompt is updated with the modified medical content generated by executing the first healthcare system-executable command and the first output healthcare system command listed as a past action performed during execution of the multi-action task.