Virtual Reality Console with Hand-Tracked Surgical Robot Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing surgical robot control systems are bulky and expensive, limiting their deployment and configuration in operating theaters, and immersive virtual reality systems face challenges in mapping operator head movements to endoscope movements.

Innovation Solution

A virtual reality display device presents a virtual screen with a surgical site representation, using input tracking to control the robot's arm, allowing intuitive control without automatic head movement mapping to the endoscope, and incorporating force feedback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a physical operator console is used to control the surgical robot, then precise control is achieved, but the console is bulky and takes up a lot of space

Engineering Contradiction:

Improvecontrol precisionVSAvoidconsole footprint

Core Design Contradiction:

Measurement precisionVSArea of stationary object

Solution Approach 1:

The patent replaces the physical operator console with a virtual reality-based control system. The surgical robot is controlled through VR headsets and hand controllers that track the operator's hand movements in three-dimensional space, eliminating the need for a large physical console with mechanical linkages and displays.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent creates a virtual copy of the surgical environment and control interface in the VR system. The virtual reality scene includes a representation of the surgical site and virtual hand controllers that mirror the physical hand movements, allowing precise control without physical console hardware.

Inventive Principle:

Principle #26Copying

2Measurement precision

If a physical operator console is used to control the surgical robot, then precise control is achieved, but the console is expensive

Engineering Contradiction:

Improvecontrol precisionVSAvoidsystem cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent replaces expensive physical console hardware with software-based virtual reality control systems, VR headsets, and tracking cameras, significantly reducing the overall system cost while maintaining control precision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The virtual reality control system can be used across multiple surgical robots and operating theaters, providing a universal control interface that reduces per-unit costs compared to dedicated physical consoles for each robot.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Ease of operation

If a physical operator console is used to control the surgical robot, then control functionality is achieved, but the console limits theater configuration options

Engineering Contradiction:

Improvecontrol functionalityVSAvoidtheater configuration flexibility

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent replaces the fixed physical console with a mobile VR control system that can be positioned anywhere in the operating theater, allowing flexible arrangement of surgical equipment and improving theater adaptability while maintaining full control functionality.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Ease of operation

If immersive virtual reality system maps operator head movements to endoscope movements, then immersion is improved, but control complexity increases

Engineering Contradiction:

Improveoperator immersionVSAvoidmovement mapping complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent inverts the control approach by tracking hand movements in three-dimensional space and mapping them to robot arm movements, rather than mapping head movements to endoscope movements. This maintains immersion while simplifying the control mapping logic.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent introduces virtual hand controllers as an intermediary between the operator's physical hands and the robot arm. The tracking system captures hand movements and the virtual controllers translate these into appropriate robot arm movements, simplifying the overall control mapping.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12426977B2Virtual console for controlling a surgical robot

Publication Date: 2025.09.30 CMR SURGICAL LTD

AI summary

Systems and methods for controlling a surgical robot comprising a base, and an arm extending from the base to an attachment for an instrument, the arm comprising a plurality of joints whereby the configuration of the arm can be altered. The system includes: a display device configured to present a view of a virtual scene to an operator of the virtual reality display device, the virtual scene comprising a virtual screen on which a representation of a real-time video stream of a surgical site is displayed; an input tracking system configured to track a position of one or more free-space inputs in space; and a control unit configured to translate the tracked position of the one or more inputs free-space into one or more control signals to control the position of the arm of the surgical robot.