Screen Identification Device for Virtualization Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods face challenges in accurately identifying screens and screen components, especially in virtualization environments, due to differences in screen attributes, structure, and display characteristics, leading to difficulties in acquiring display character strings without extensive operation settings.
Innovation Solution
An identification device and method that utilize both screen image and component object information to identify screens and components, creating a sample screen model for virtualization environments, allowing for accurate identification and character string acquisition without requiring manual operation settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional identification methods are used, then identification process is simple, but identification accuracy deteriorates in virtualization environments due to screen attribute and structure differences
Solution Approach 1:
The patent combines multiple identification approaches (object access method and image recognition method) into a unified identification device that processes both screen component object information and screen images. This merging allows the system to leverage the strengths of both methods while compensating for their individual weaknesses in virtualization environments, thereby improving identification accuracy without requiring separate complex processing pipelines.
Solution Approach 2:
The patent introduces a sample screen model as an intermediary representation that captures characteristic information of screen components. This sample screen model serves as a mediator between the raw screen data and the identification process, enabling accurate matching even when screen attributes and structures vary in virtualization environments. The sample screen model abstracts away the complexity of different screen implementations while preserving essential identification features.
2Productivity
If manual operation settings are performed for each screen variation, then identification accuracy is improved, but operation time and effort increase significantly
Solution Approach 1:
The patent performs preliminary actions by automatically creating a sample screen model that captures characteristic information of screen components before actual identification operations. This sample screen model is prepared in advance and can be reused for identifying various screen variations without requiring manual reconfiguration. The preliminary creation of the sample screen model eliminates the need for manual operation settings for each screen variation, significantly improving productivity while reducing time loss.
Solution Approach 2:
The identification device performs self-service by automatically adapting to different screen implementations through the sample screen model mechanism. The system autonomously handles screen variations without requiring manual intervention or reconfiguration. The sample screen model enables the system to self-adjust to different screen mounting contents and display conditions, eliminating the need for manual operation settings and thereby improving operation efficiency.
3Measurement precision
If screen image alone is used for identification, then ease of operation is improved, but measurement precision deteriorates due to susceptibility to look and feel differences
Solution Approach 1:
The patent merges object access method and image recognition method into a unified approach that processes both screen component object information and screen images. This combination allows the system to maintain ease of operation while significantly improving measurement precision. The object information provides structured data about screen components, while the image recognition handles visual characteristics, and their synergistic processing achieves accurate identification despite variations in look and feel.
Solution Approach 2:
The patent creates a composite identification approach that integrates multiple data types (object information and image data) into a unified sample screen model. This composite structure combines the strengths of both data sources, enabling the system to maintain operational simplicity while achieving high measurement precision. The composite sample screen model can accommodate various screen implementations without requiring complex manual configuration.
Data Source
AI summary
An identification device (10) includes a first identification unit (151a) that identifies first screen data including an image of a screen of an application and information regarding a screen component object that is an object of an element included in the screen, and outputs a first identification result associated with sample screen data that is screen data to be referred to, a derivation unit (152a) that derives a sample screen model used for identification in a virtualization environment on the basis of the sample screen data and the first identification result, and a second identification unit (153a) that identifies second screen data including an image of the screen and not including information regarding the screen component object, and outputs a second identification result associated with a sample screen model.