Legacy Equipment GUI Automation for Real-Time Factory Data Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing factory equipment lacking automation capabilities faces challenges in data collection and control, particularly in industries like semiconductor and electronics manufacturing, due to the lack of SECS/GEM protocol support, slow data extraction rates, complex configuration, and difficulty in correlating equipment data with external sensor data for predictive maintenance and RUL prediction.
Innovation Solution
A plug-and-play system that extracts data from non-automation capable equipment, supports SECS/GEM protocol, and enables real-time data transmission and control without hardware or software installation, using OCR and machine learning to combine and analyze data from multiple sources, and simulate keyboard and mouse interactions to execute commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional data collection methods are used on legacy equipment, then equipment automation capability is improved, but device complexity and installation requirements increase
Solution Approach 1:
The patent introduces an intermediary system comprising a camera, OCR engine, and simulation software that acts as a mediator between legacy equipment and factory automation systems. The camera captures display screen images, the OCR engine extracts data from these images, and the simulation software translates extracted data into equipment-specific commands, enabling automation without direct integration with the legacy equipment's internal systems.
Solution Approach 2:
The patent replaces direct mechanical/electrical integration methods with an optical-information processing approach. Instead of physically interfacing with equipment hardware or installing software on equipment controllers, the system uses optical capture (camera), optical-to-digital conversion (OCR), and software simulation to achieve data extraction and command transmission, thereby reducing system complexity.
2Adaptability or versatility
If third-party software or hardware is installed on factory equipment to add automation capability, then equipment functionality is improved, but reliability and performance may deteriorate
Solution Approach 1:
The patent segments the automation system into distinct functional modules: display capture (camera), data extraction (OCR engine), data translation (simulation software), and command transmission. Each module operates independently, allowing the legacy equipment to remain unchanged and unaffected by automation additions, thereby preserving reliability while achieving adaptability.
Solution Approach 2:
The system uses an intermediary architecture that interfaces with the legacy equipment only through its display output and control input interfaces, without installing any software or hardware on the equipment itself. This intermediary approach adds versatility while isolating the legacy equipment from potential reliability issues.
3Extent of automation
If equipment is upgraded or replaced with newer automation-capable equipment, then automation capability is improved, but manufacturing cost and downtime increase
Solution Approach 1:
The patent creates a virtual copy or simulation layer that replicates the equipment's control interface and data structure. The simulation software models the equipment's behavior and communication protocols, allowing automation systems to interact with legacy equipment as if it were modern automated equipment, thereby avoiding costly hardware upgrades.
Solution Approach 2:
The patent develops a universal interface layer that can work with multiple types of legacy equipment through standardized OCR and simulation approaches. This multi-functional system can adapt to different equipment vendors and models without requiring custom integration for each, reducing overall manufacturing costs.
4Loss of time
If data extraction frequency is increased to achieve real-time processing, then data freshness is improved, but processing speed and system performance deteriorate
Solution Approach 1:
The patent implements periodic data extraction at optimized intervals rather than continuous extraction. The system captures display screen images at predetermined time intervals, processes OCR data at these periodic points, and transmits commands periodically, achieving real-time sufficient data freshness while maintaining processing speed.
Solution Approach 2:
The patent extracts only the specific data elements needed for automation control from the full display screen image, rather than processing all visible information. The OCR engine selectively identifies and extracts relevant parameters, reducing processing overhead while maintaining data freshness for critical control functions.
Data Source
AI summary
System and Method for Factory Automation and Enhancing Machine Capabilities The present invention relates to system and method for factory automation and for enhancing machine capabilities. The system is configured to extract data from machine through data capturing module, data extraction module and data conversion module and jointly analyze them with data from other equipment and sensors through data analysis engine and transmitting to factory systems in user configured protocol through protocol conversion module. It also accepts machine control commands from the factory systems and executes them through command processor and mouse & keyboard simulator modules. It involves combining GUI from multiple equipments/sensors and sending to single display device through GUI manager and video output modules and mapping required actions of mouse clicks and keyboard entries from new to old user interfaces. GUI manager module detects GUI elements from captured images and applies user configuration to automatically design new GUI that includes data from other external sensors and devices.