Gesture Control Large Screen Operator Workstations

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current operating and observation systems in technical installations restrict control and process guidance when operators need to work together or evaluate complex information from different user workstations, as they cannot alter the display on large screens remotely.

Innovation Solution

An operating and observation system with a large display screen and user workstations equipped with motion recognition and control software, allowing users to operate and poll components, measured values, and state messages using body movements and gestures, enabling remote interaction and navigation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If operators use traditional user workstations with keyboard and mouse to control the large screen, then control precision is maintained, but remote collaboration and flexibility are restricted

Engineering Contradiction:
Improveremote control capabilityVSAvoidcontrol accessibility
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent replaces the mechanical keyboard and mouse system with a gesture recognition system using cameras and motion tracking. Operators can control the large screen from any position in the control room by making body movements, eliminating the need to be physically present at a user workstation. This substitution enables remote collaboration while maintaining ease of operation through natural gestures.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If multiple experts need to work together at different user workstations, then collaboration is enabled, but control of the large screen is lost

Engineering Contradiction:
Improvecollaboration capabilityVSAvoidprocess guidance efficiency
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The gesture recognition system provides universal control capability that can be accessed from any location in the control room, not just from specific user workstations. Multiple operators can simultaneously control the large screen from different positions using gestures, enabling collaboration without losing control capability. This multi-functional approach allows the system to serve both individual and collaborative work modes effectively.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Reliability

If operators remain at their user workstations to maintain control, then control authority is preserved, but work monotony increases

Engineering Contradiction:
Improvecontrol authorityVSAvoidwork flexibility
Core Design Contradiction:
ReliabilityVSEase of operation

Solution Approach 1:

The system transitions from a static control model where operators must remain seated at fixed workstations to a dynamic model where control authority moves with the operator throughout the control room. The gesture recognition system continuously tracks operator position and maintains control capability regardless of location, making the control system adaptive and flexible while preserving reliability through consistent gesture-based commands.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS10824126B2Device and method for the gesture control of a screen in a control room
Publication Date: 2020.11.03 ABB (SCHWEIZ) AG
  • US10824126B2 patent drawing
  • US10824126B2 patent drawing

AI summary

An operation and observation system of a technical plant and/or a technical process and associated method are provided for operating components and for displaying measurement readings, process variables and/or status messages of the components of the plant and/or process. The system includes at least one large screen for displaying the components of the technical process and/or technical plant, and a plurality of operator workstations. A movement detection and movement control component is installed on the screen and on the operator workstations, which allows an operation and/or polling of the components, the measurement readings, the process variables and/or the status messages of the components of the technical plant or of the technical process by means of a body movement of an operator.