Screen Capture Mediator for AI Integration in PACS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The integration of AI-based applications with Picture Archiving and Communication Systems (PACS) and Radiology Information Systems (RIS) is hindered by the lack of standard mechanisms for displaying AI results and interacting with them, leading to inconvenient and costly deep integration processes, which limits the adoption of AI in medical imaging.
Innovation Solution
A method for exchanging data between an external AI-based data source and an integrated medical data display system, using screen capture techniques to determine and display information from the PACS/RIS system, allowing for automatic selection and display of AI data without concealing the original information, and extracting user-added annotations to improve AI algorithm performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep integration is performed through proprietary API or iFrame embedding, then compatibility between AI tool and PACS viewer is improved, but integration cost and complexity increase significantly
Solution Approach 1:
The patent introduces a window manager as an intermediary component that sits between the AI confirmation tool and the PACS viewer. This window manager captures screen content from the PACS viewer, determines appropriate display regions, and manages the positioning and sizing of the confirmation tool window, thereby eliminating the need for deep proprietary integration while maintaining compatibility
Solution Approach 2:
The system captures the screen content of the PACS viewer and creates a visual copy or representation of the displayed medical images and layout. This captured screen information is then used by the window manager to intelligently position and size the confirmation tool window, ensuring it appears in an appropriate region without requiring direct integration with the PACS software
2Ease of manufacture
If external application window is used for AI confirmation, then integration cost is reduced, but the window hides useful PACS content and requires manual repositioning
Solution Approach 1:
The confirmation tool window is designed to be dynamic rather than static. The window manager continuously monitors the PACS viewer screen content and automatically adjusts the confirmation window's position and size in real-time. This dynamic behavior ensures the window always appears in an optimal location that does not obscure important medical image content, eliminating the need for manual repositioning by the user
Solution Approach 2:
The system implements a feedback mechanism where the window manager continuously captures screen content from the PACS viewer, analyzes the current display state, and uses this information to automatically adjust the confirmation tool window's positioning. This closed-loop feedback ensures the window adapts to changing display conditions and maintains optimal visibility without user intervention
3Area of stationary object
If confirmation UI is displayed on full screen, then AI results are visible, but it obscures the PACS viewer content and requires manual movement
Solution Approach 1:
Instead of displaying the confirmation tool window in a fixed full-screen mode, the window manager applies local quality by dynamically determining the best local region on the screen for window placement. It analyzes the PACS viewer layout and positions the confirmation window in an appropriate region that maximizes visibility of both the PACS content and the AI confirmation results, ensuring important medical images remain visible while AI results are accessible
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method for exchanging data between an external data source (5) for annotations (ANO) and an integrated medical data display system (1) is described. The method comprises the steps of determining information (SCR-C) displayed on a screen (2) of the integrated medical data display system (1) by capturing the screen (2) and performing at least one of the following steps: selecting data (D-EXT, ECW) from the external data source (5) assigned to the determined information (SCR-C) and displaying the selected data (D-EXT, ECW) complementary, extracting annotations (ANO) from the integrated medical data display system (1) based on the determined information (SCR-C) and appending the extracted annotations (ANO) to the external data source (5) for annotations (ANO). Further, a data exchange device (60) is described. Furthermore, a medical data analysis system (10) is described.