Multi-Client AR Instrument Tracking via WebSocket Mediator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) technologies face challenges in providing a unified and effective multi-client deployment for surgical planning, intervention, and education, particularly in ensuring consistent experiences across different devices and platforms, managing device-specific implementations, coordinating device/platform rendering, and synchronizing data across multiple clients.
Innovation Solution
The system employs a server with a WebSocket server for multi-client broadcasting and streaming, using HTTP and WebSocket clients to manage sessions and data synchronization, and client devices with processors to receive and process real-time input data, generate metrics, and display AR graphics, ensuring synchronized data across devices through time-stamped buffers and reference coordinate frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a server with WebSocket server is used for multi-client broadcasting and streaming, then real-time data synchronization across multiple devices is improved, but system complexity and infrastructure requirements increase
Solution Approach 1:
A server acts as an intermediary between multiple client devices, receiving data from one client and broadcasting it to others via WebSocket connections. This mediator approach enables real-time synchronization without requiring direct peer-to-peer connections between all devices, reducing overall system complexity while maintaining reliability.
Solution Approach 2:
The server performs multiple functions: it manages WebSocket connections, broadcasts data to multiple clients, handles HTTP requests, and coordinates AR sessions. This multi-functional approach consolidates infrastructure requirements into a single system rather than requiring separate components for each function.
2Adaptability or versatility
If device-specific implementations are managed for different platforms, then adaptability across devices is improved, but coordination difficulty and development complexity increase
Solution Approach 1:
The system separates concerns by dividing functionality into platform-specific client implementations and a universal server backend. Each client device can be optimized for its specific platform (iOS, Android, desktop) while the server maintains consistent data protocols, reducing coordination difficulty through clear separation of responsibilities.
Solution Approach 2:
Different client devices implement platform-specific optimizations and interfaces tailored to their local characteristics (touch interfaces for mobile, keyboard/mouse for desktop, AR headset controls). The server maintains universal data formats that adapt to these local qualities without requiring complex cross-device coordination.
3Measurement precision
If time-stamped buffers and reference coordinate frames are used for synchronization, then AR graphic alignment precision is improved, but data processing complexity increases
Solution Approach 1:
The system pre-establishes reference coordinate frames and assigns time stamps to data packets before transmission. This preliminary setup creates a common temporal and spatial reference system that simplifies real-time synchronization during AR sessions, reducing the need for complex runtime calculations.
Solution Approach 2:
The server continuously monitors and adjusts data synchronization using time-stamped feedback from multiple clients. By tracking temporal relationships and coordinate frame alignments in real-time, the system maintains precise AR graphic alignment while using relatively simple buffer management and time-synchronization algorithms.
Data Source
AI summary
Methods and related systems and devices are described for performing various AR medical applications, including a method of guiding augmented reality (AR) intervention. In one aspect, a primary client device: receives model sets, an intervention plan having an intervention field, and session information about a session related to the AR intervention from a server; receives first real-time input data from a first input device; generates metrics by evaluating an execution of the intervention plan by comparing the intervention plan to the first real-time input data; displays real-time graphics, based at least in part on the metrics, spatially over the intervention field; receives real-time status data, from the server, about a replicate client device that joins the session; sends the first real-time input data, the metrics and the evaluation computed from the intervention plan, through the server, to the replicate client device.