Automated GUI Data Mapping via Screen Capture Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The development of applications is a labor-intensive and time-consuming process due to the need for manual coding based on detailed requirements documents, requiring specialized expertise and resulting in high costs.
Innovation Solution
A model is generated from application requirements that encapsulates GUI screens and data objects, allowing for automated code generation that implements the application's view and data connectivity, enabling rapid development across different platforms without manual coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual coding is used to develop applications based on requirements documents, then application functionality can be implemented, but development time and costs increase significantly
Solution Approach 1:
The patent uses screen capture images as copies of the actual application interface to automatically generate code. Instead of manually coding from scratch, the system captures the visual representation of the screen and automatically translates it into executable code, dramatically reducing development time while maintaining functional accuracy
Solution Approach 2:
The patent replaces the manual mechanical process of coding with an automated optical recognition system. The system uses image processing and pattern recognition to automatically translate visual screen elements into code, substituting human manual labor with automated computational processes
2Reliability
If manual coding is used to develop applications, then application functionality can be implemented, but labor intensity and costs increase
Solution Approach 1:
The system performs self-service by automatically generating code from screen captures without requiring manual programming. The automated system identifies UI elements, determines their properties, and generates the corresponding code independently, eliminating the need for developers to manually write each line of code
Solution Approach 2:
The patent replaces manual coding labor with automated image-to-code conversion technology. The system uses computer vision and pattern recognition algorithms to automatically translate visual interface elements into programmatic code, substituting human manual effort with automated computational processes
3Loss of time
If automated code generation is used from screen captures, then development time is reduced, but the complexity of the generation system increases
Solution Approach 1:
The patent segments the screen capture into distinct UI elements and processes each element independently. The system divides the complex task of generating code for an entire screen into smaller sub-tasks for individual UI components, making the overall process more manageable and systematic
Solution Approach 2:
The patent introduces an intermediary processing layer that translates visual screen elements into code. This intermediary system acts as a mediator between the visual interface and the programming language, automatically mapping UI properties to code structures without requiring direct manual intervention
Data Source
AI summary
Techniques are disclosed for automating application development from requirements information for the application. The requirements information may comprise mockup images of GUI screens for the application, text information, and other information. A model is generated based upon the requirements information. The generated model can then subsequently be used to generate one or more implementations of the application. Generation of the model includes processing the requirements information to identify one or more data objects to be mapped or associated with one or more GUI screens of the application, and further determining, for a particular data object mapped to a particular GUI screen, mappings between one or more components of the particular screen to one or more components of the particular data object. The model stores information about the look-and-feel of the GUI screens and about mappings between data objects to screens and screen components to data object components.