AR Digital Twin for Automation Control Application Development
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automation control application development in manufacturing is hindered by hardware defects causing delays and challenges in identifying error sources, requiring extensive expert knowledge and expensive simulation tools with non-intuitive user interfaces.
Innovation Solution
An Augmented Reality (AR) system is introduced to develop and test control application programs, allowing incremental development and simulation of control applications in an AR environment, enabling the use of virtual data to mimic real sensor inputs and allowing development before full plant commissioning, even with missing hardware components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If hardware-in-the-loop or software-in-the-loop simulation environment is used, then control application development can proceed before hardware setup, but extensive expert knowledge and expensive simulation tools are required
Solution Approach 1:
The patent creates a virtual copy (digital twin) of the physical automation system that replicates its behavior and characteristics. This virtual model allows control application development to proceed without the actual hardware, eliminating the need for expensive simulation tools while maintaining development timing benefits. The digital twin is updated in real-time to reflect the actual system state.
Solution Approach 2:
The patent introduces an AR interface as an intermediary between the control engineer and the development environment. This AR interface provides an intuitive visualization of the control application and system state, replacing complex simulation tool interfaces while enabling development before hardware setup.
2Ease of manufacture
If mathematical modeling is used for simulation, then control application can be developed and tested, but extensive expert knowledge in mathematical modeling and control theory is required
Solution Approach 1:
The system automatically generates and updates the digital twin model based on real-time data from the physical system, eliminating the need for manual mathematical modeling by engineers. The system self-updates its virtual representation as hardware is added or modified, making control application development accessible without expert knowledge in mathematical modeling.
Solution Approach 2:
The patent replaces complex mathematical modeling operations with automated data collection and processing. Instead of requiring engineers to create and tune mathematical models, the system automatically captures real-world system behavior and translates it into the digital twin, substituting manual modeling work with automated computational processes.
3Productivity
If control application is developed during setup and commissioning, then hardware defects cause delays, but developing before setup requires simulation tools
Solution Approach 1:
The patent enables control application development to occur in parallel with hardware setup by maintaining a digital twin that can be updated as hardware becomes available. This preliminary development action allows the control logic to be prepared and tested virtually before the actual hardware is ready, eliminating delays caused by hardware defects or delivery issues while ensuring system readiness through continuous synchronization.
Solution Approach 2:
The system dynamically adapts its development approach based on hardware availability. The digital twin can operate in different modes: fully virtual when hardware is unavailable, hybrid when some hardware is ready, and fully integrated when complete. This dynamic flexibility allows continuous productivity improvement while maintaining system readiness through progressive synchronization with actual hardware.
4Loss of time
If packaged simulation tools are used, then control application can be simulated, but they lack intuitive user interfaces and require extensive expert knowledge
Solution Approach 1:
The patent introduces an AR-based intermediary interface that presents control application development and simulation in an intuitive, visual format. Instead of requiring engineers to navigate complex packaged simulation tool interfaces, the AR interface overlays virtual representations directly in the engineer's field of view, providing natural interaction and eliminating the need for expert knowledge while maintaining fast simulation capability.
Data Source
AI summary
A system and method is disclosed for development of a control application for a controller of an automation system. The controller receives sensor signals associated with perception of a first real component during an execution of the control application program. Activity of a virtual component, including interaction with the real first component, is simulated, the virtual component being a digital twin of a second real component designed for the work environment and absent in the work environment. Virtual data is produced in response to the simulated activity of the virtual component. A control application module determines parameters for development of the control application program using the sensor signals for the first real component and the virtual data. An AR display signal for the work environment is rendered and displayed based on a digital representation of the virtual data during an execution of the control application program.