Screen Component Selection and Coordinate Mapping for Android UI Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for distributing Android applications require significant time and resources for development, build, testing, and distribution, and do not allow for efficient remote patching or updating without rebuilding the application.
Innovation Solution
A method for a computing device to determine screen coordinates and UI coordinates using a hierarchy structure and translation function, generating report information for screen components without rebuilding or redistributing the application.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional Android application distribution process is followed, then application can be distributed on Google Play Store, but significant time and resources are consumed for development, build, testing, and distribution
Solution Approach 1:
The patent uses screenshot images as copies of the actual application interface, allowing analysis and patching without needing the original application executable. The system captures visual representations of UI elements and uses these copies to identify, analyze, and patch screen components, eliminating the need for full application redistribution
Solution Approach 2:
The patent extracts specific screen component information from the overall application by analyzing screenshots and identifying individual UI elements. This extraction allows the system to work with isolated component data rather than the complete application package, enabling selective patching without rebuilding the entire application
2Adaptability or versatility
If application is updated using traditional methods, then application can be patched remotely, but changes are limited and cannot change the execution entity
Solution Approach 1:
The system creates and uses copy information (screenshot images) of the application interface to enable remote analysis and patching. These visual copies allow the system to identify screen components and generate patch data without accessing or modifying the actual application execution entity, providing flexible remote updates while maintaining system integrity
Solution Approach 2:
The patent introduces screenshot images as an intermediary between the user and the application execution entity. This intermediary layer allows users to select, analyze, and patch screen components through visual representation without directly interacting with the complex application code or execution environment, simplifying the update process
Data Source
AI summary
A method for assisting in collecting information about screen components performed by a computing device including at least one processor according to some embodiments of the present disclosure is disclosed. The method for assisting in collecting information about the screen components may comprise: determining screen coordinates corresponding to an area within a screen of the computing device on which a pre-designated application is executed, according to a first user interaction of selecting the area; determining User Interface (UI) coordinates within a user interface corresponding to the screen coordinates using a hierarchy structure of parent UI and child UI defined for the application and a translation function corresponding to each of the hierarchy structure; determining a first node corresponding to the UI coordinates; and generating report information for displaying an object representing the first node in a pre-designated visualization format.