XR Interaction Visualization for Medical Device Design

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional product development of technical devices, especially medical devices, lacks the ability to effectively visualize interactions between hardware and software before the first real system is developed, leading to delays and challenges in data collection and realism.

Innovation Solution

A method for visualizing interactions in an extended reality (XR) scene using a computing device, which receives a dataset representing an XR scene with a technical device, displays it on an XR headset, and detects optical sensor data of a user interacting with the scene, fusing these datasets to generate a third dataset for rendering and analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional product development methods are used with hardware prototypes and software mock-ups, then the development process can proceed with basic functionality testing, but the ability to visualize and test full hardware-software interactions is severely limited

Engineering Contradiction:

Improvedetection of errors and weaknesses in designVSAvoidtime to detect errors and weaknesses

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent creates a virtual copy of the technical device in an XR environment that replicates hardware appearance, behavior, and software interface. This virtual copy enables comprehensive interaction testing without requiring physical hardware prototypes, allowing errors to be detected earlier in the development process.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The XR environment serves as an intermediary between physical hardware prototypes and final software implementation. It provides a middle ground where hardware-software interactions can be visualized and tested with realistic fidelity before committing to full system integration.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If physical hardware prototypes and software mock-ups are used for usability testing, then basic functionality can be evaluated, but realistic interaction visualization and data collection are compromised

Engineering Contradiction:

Improveusability testing capabilityVSAvoidinteraction data for training and analysis

Core Design Contradiction:

Ease of operationVSLoss of information

Solution Approach 1:

The system automatically captures and records all user interactions with the virtual device in the XR environment, providing comprehensive feedback data for analysis. This automated feedback mechanism enables detailed usability evaluation and generates training data without requiring manual observation or annotation.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

By creating a virtual replica of the technical device with identical interaction patterns, the system enables realistic usability testing while automatically capturing comprehensive interaction data that would be difficult or impossible to collect with physical prototypes.

Inventive Principle:

Principle #26Copying

3Productivity

If conventional development approaches are used, then development can proceed with available resources, but the ability to collect meaningful interaction data early in development is impossible

Engineering Contradiction:

Improveefficiency of product developmentVSAvoidinteraction data for AI training and system optimization

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system enables collection of interaction data during the development phase itself, rather than waiting until deployment. By creating a virtual device that mimics real hardware behavior, meaningful training data can be gathered preliminarily to optimize AI algorithms and system performance before the actual product is released.

Inventive Principle:

Principle #10Preliminary action

4Quantity of substance

If synthetic training data is generated through conventional methods like rotation and mirroring, then data quantity can be increased, but data realism and quality deteriorate

Engineering Contradiction:

Improveamount of training dataVSAvoidrealism of training data

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

Instead of synthetically transforming limited real data through rotation and mirroring, the system creates a complete virtual copy of the technical device that generates unlimited realistic interaction data. This virtual replica produces training data with authentic visual appearance and interaction patterns, maintaining high realism while providing abundant quantity.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20250147577A1Technique for visualizing interactions with a technical device in an XR scene

Publication Date: 2025.05.08 SIEMENS HEALTHINEERS AG

AI summary

A computer-implemented method for visualizing interactions in an extended reality (XR) scene, the computer-implemented method comprising: receiving a first dataset representing an XR scene including a technical device; displaying the XR scene on an XR headset or a head-mounted display (HMD); providing a room for a user wearing the XR headset or HMD for interacting with the XR scene, wherein the room includes a set of optical sensors including at least one optical sensor at a fixed location relative to the room; detecting optical sensor data of the user as a second dataset while the user is interacting with the XR scene in the room; and fusing the first dataset and the second dataset to generate a third dataset.