AR Visualization System for Industrial Automation Procedures

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Operators in industrial automation environments face challenges in performing tasks efficiently due to the need to memorize device-specific procedures and identify changes in industrial automation devices, leading to potential errors and increased downtime.

Innovation Solution

An interactive augmented reality (AR) user experience (UX) system that provides visualizations of real-world and computer-generated content, guiding operators through maintenance and operation procedures by overlaying animated virtual objects onto real-time images of industrial automation devices, reducing the likelihood of errors and improving operational efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If operators rely on memorization of device-specific procedures, then they can perform tasks without external aids, but error rates increase and training time increases

Engineering Contradiction:
Improveerror rateVSAvoidtraining time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent introduces an augmented reality system as an intermediary between the operator and the industrial automation device. The AR system overlays procedural instructions, visual guides, and real-time feedback directly in the operator's field of view, serving as a cognitive aid that reduces reliance on memorization while decreasing errors and training requirements

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements real-time feedback mechanisms where the AR device monitors operator actions, compares them against correct procedures, and provides immediate corrective guidance. This closed-loop feedback reduces errors by preventing incorrect actions and accelerates learning by reinforcing correct procedures during actual task performance

Inventive Principle:
Principle #23Feedback

2Adaptability or versatility

If operators must identify changes in industrial automation devices, then they can adapt to new configurations, but task completion time increases

Engineering Contradiction:
Improveadaptation to new configurationsVSAvoidtask completion time
Core Design Contradiction:
Adaptability or versatilityVSProductivity

Solution Approach 1:

The AR system continuously monitors the device state through integrated sensors and image recognition, automatically detecting configuration changes and updating the overlaid procedural guidance in real-time. This eliminates the need for operators to manually identify changes, maintaining adaptability while reducing task completion time

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system pre-loads and displays procedural instructions that anticipate required actions based on the current device state. When changes are detected, the AR system proactively updates the guidance before the operator needs to act, allowing seamless adaptation without increasing task completion time

Inventive Principle:
Principle #10Preliminary action

3Manufacturing precision

If detailed procedural instructions are provided to operators, then task accuracy improves, but information overload occurs and ease of operation decreases

Engineering Contradiction:
Improvetask accuracyVSAvoidease of use
Core Design Contradiction:
Manufacturing precisionVSEase of operation

Solution Approach 1:

The AR system applies the principle of local quality by providing context-specific information only at the relevant moment and location. Procedural instructions are overlaid directly on or near the specific component being manipulated, and details are revealed progressively as the operator progresses through the task, maintaining accuracy without causing information overload

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The procedural information is segmented into discrete, manageable steps that are presented sequentially. Each AR overlay contains only the immediate next action required, breaking down complex procedures into small cognitive chunks that are easy to process while maintaining overall task accuracy

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS11556875B2Generating visualizations for instructional procedures
Publication Date: 2023.01.17 ROCKWELL AUTOMATION TECH INC
  • US11556875B2 patent drawing
  • US11556875B2 patent drawing
  • US11556875B2 patent drawing

AI summary

A system may include a processor and a memory. The memory may include computer-executable code that, when executed by the processor, causes the processor to retrieve a workflow dataset from a database based on a query input associated with an industrial automation device. The workflow dataset may include an instruction associated with one or more operations for the industrial automation device and a virtual object associated with the one or more instructions and the industrial automation device. The memory may include computer-executable code that, when executed by the processor, causes the processor to transmit a first portion of the workflow dataset to a computing device corresponding to a first instruction. The memory may include computer-executable code that, when executed by the processor, causes the processor to transmit a second portion of the workflow dataset to the computing device in response to determining that the first instruction is completed.