HMI Screen Recognition Using AI Cameras for SCADA Data Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing SCADA systems in factories and power plants face challenges in data collection due to limited software installation and system modification capabilities, making it difficult to integrate data for management systems.
Innovation Solution
A method and apparatus for collecting data through HMI screen recognition and image analysis using a camera, which involves receiving images from an HMI screen, generating analysis results using an AI learning model, and storing collection data. The AI model is trained using the collected data, allowing for continuous improvement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional SCADA systems are used with direct software installation and system modification for data collection, then data collection capability is improved, but system complexity and ease of operation deteriorate due to limited modification capabilities
Solution Approach 1:
The patent uses image copying technology to capture HMI screen displays and convert them into digital images for analysis. Instead of modifying the SCADA system software directly, the system creates visual copies of the screen content and processes these images through AI models to extract data, thereby avoiding system complexity while maintaining data collection capability
Solution Approach 2:
The patent replaces traditional mechanical/software-based data collection methods (direct SCADA software installation and system modification) with an optical-based approach using camera imaging and AI image analysis. This substitution eliminates the need for system modifications while achieving the same data collection objective
2Ease of operation
If image recognition method is used to collect data from HMI screen, then ease of operation is improved, but measurement precision deteriorates due to challenges in accurately recognizing charts and numbers
Solution Approach 1:
The patent segments the HMI screen image into different regions and elements (charts, numbers, text, graphical interfaces) and processes each segment separately through specialized AI models. This segmentation approach improves recognition accuracy by focusing on specific elements individually rather than attempting to analyze the entire screen as a single image
Solution Approach 2:
The patent introduces an intermediary AI image analysis model that acts as a bridge between the captured HMI screen image and the final extracted data. This intermediary model performs intermediate processing steps including image enhancement, feature extraction, and pattern recognition to improve the precision of data extraction from the visual screen content
3Adaptability or versatility
If AI learning model is trained continuously with collected data, then adaptability is improved, but loss of time increases due to training process requirements
Solution Approach 1:
The patent implements periodic training of the AI model at scheduled intervals rather than continuous training. The system collects data during operational periods and schedules training sessions during maintenance windows or low-activity periods, thereby achieving model adaptability improvements without causing continuous time loss that would disrupt system operation
Solution Approach 2:
The patent performs preliminary data preprocessing and preparation during data collection phases, organizing and cleaning the data in advance before training sessions. This preliminary action reduces the actual training time by ensuring data is ready for immediate processing when training occurs, minimizing the time loss associated with the training process
Data Source
AI summary
Disclosed is a method for collecting data through HMI screen recognition and image analysis by using a camera. According to one embodiment of the present invention, the method for collecting the data through the HMI screen recognition and the image analysis by using the camera includes: receiving a first image obtained by capturing a screen of a human-machine interface (HMI) from an image camera; generating an image analysis result by analyzing the first image based on an artificial intelligence learning model configured to generate an analysis result by analyzing an image; and storing collection data corresponding to the first image based on the image analysis result.