AI Graphical Object Recognition Using Hierarchical Model Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional AI processes often fail to recognize certain graphical objects in graphical user interfaces, leading to incomplete testing and potentially lower quality software applications with more bugs.
Innovation Solution
The use of a hierarchical AI mode that identifies a first graphical object and retrieves source code from a hierarchical model of the graphical user interface, allowing for the comparison of attributes between identified and unidentifiable graphical objects to determine their type.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional AI processes are used for graphical object recognition, then the recognition process is simple and fast, but some graphical objects are not properly identified leading to incomplete testing
Solution Approach 1:
The patent introduces an intermediary verification mechanism that bridges traditional AI recognition and hierarchical model validation. When AI identifies a graphical object, the system retrieves corresponding source code from the hierarchical model to verify the identification. This intermediary step ensures accuracy without completely replacing the simple AI process, thus improving reliability while maintaining manageable complexity.
Solution Approach 2:
The system segments the graphical object recognition process into distinct phases: initial AI identification, hierarchical model source code retrieval, attribute comparison, and validation. This segmentation allows each component to be optimized independently - the AI algorithm for speed and the hierarchical model for accuracy - resolving the contradiction between simple/fast processing and reliable identification.
2Reliability
If hierarchical model verification is added to AI recognition, then all graphical objects are properly identified, but the process becomes more complex
Solution Approach 1:
The system performs preliminary actions by pre-establishing the hierarchical model structure and source code relationships before the actual recognition process. This allows the verification step to efficiently query pre-organized data rather than creating structures during processing, reducing the perceived complexity while ensuring complete testing coverage.
Solution Approach 2:
The patent applies local quality by selectively applying hierarchical verification only to graphical objects identified by the AI algorithm, rather than processing all objects through the complex hierarchical model. This targeted approach ensures testing completeness for identified objects while avoiding unnecessary complexity for objects the AI confidently recognizes.
3Measurement precision
If attribute comparison is performed between identified and unidentifiable graphical objects, then recognition accuracy improves, but processing time increases
Solution Approach 1:
The system performs partial attribute comparison by selecting only critical attributes (such as position, size, and key visual characteristics) for comparison between AI-identified objects and hierarchical model objects, rather than comparing all possible attributes. This partial action maintains high determination accuracy while significantly reducing processing time compared to exhaustive attribute comparison.
Data Source
AI summary
A first graphical object in a graphical user interface is identified using an Artificial Intelligence (AI) algorithm. The graphical user interface comprises a second graphical object that was not properly identified by the AI algorithm. Source code is retrieved from a hierarchical model of the graphical user interface. A determination is made if the identified first identified graphical object and the second graphical are the same type (e.g., a button object). In response to the identified first graphical object and the second graphical object being the same type, an attribute of the identified first graphical object is compared to an attribute of the second graphical object. In response to the attribute of the identified first graphical object being the same as the attribute of the second graphical object, the second graphical object as identified as the same graphical object type as the first graphical object.