Automatic Natural Language Interface Generation from GUI Code
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developing a Natural Language Interface (NLI) from scratch requires significant effort and expertise, and maintaining synchronization with existing Graphical User Interfaces (GUIs) is cumbersome for application developers, as they need to invest time in researching and implementing both interfaces.
Innovation Solution
A computer-implemented method that maps GUI code representations to NLI components, determines pre-defined intent patterns, creates intent and entity examples, and generates natural language dialogue by leveraging pre-defined conversation components, allowing for automatic creation of NLI from GUI code without altering the existing GUI.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If NLI is built from scratch with GUI, then the application can have both GUI and NLI interfaces, but it requires significant time investment and expertise in conversation design
Solution Approach 1:
The patent copies the existing GUI structure, components, and logic to automatically generate the NLI interface. Instead of building NLI from scratch, the system creates a replica of the GUI's functional structure in natural language format, thereby reducing development time while maintaining interface capability.
Solution Approach 2:
The patent performs preliminary analysis of the GUI code structure, components, and interactions before generating the NLI. By pre-processing the GUI code to extract relevant information about screens, components, and logic flows, the system prepares the foundation for automatic NLI generation, significantly reducing the time required for conversation design.
2Adaptability or versatility
If NLI is built from scratch with GUI, then the application can have both interfaces, but maintaining synchronization between the two interfaces is cumbersome
Solution Approach 1:
The patent merges the GUI and NLI development processes by generating NLI automatically from the existing GUI code. This unification means that when the GUI is modified, the NLI can be regenerated to reflect those changes, eliminating the need for separate manual synchronization and reducing maintenance complexity.
Solution Approach 2:
The system establishes a feedback mechanism where changes in the GUI code are detected and automatically trigger regeneration of the NLI interface. This closed-loop approach ensures that the NLI remains synchronized with the GUI without requiring manual intervention, thereby reducing interface maintenance complexity.
3Adaptability or versatility
If NLI is built from scratch, then the application can have natural language interface, but it requires knowledge in conversation design that is not common among application developers
Solution Approach 1:
The patent enables the GUI code to serve itself by automatically generating the corresponding NLI interface. The existing GUI code contains all the necessary information about the application's functionality, and the system extracts this information to create the NLI without requiring external conversation design expertise, thereby improving implementation ease.
Solution Approach 2:
The patent introduces an intermediary automated generation system that translates GUI code into NLI format. This intermediary handles the complex conversation design process by analyzing the GUI structure and automatically creating appropriate natural language interfaces, shielding developers from the need for specialized conversation design knowledge.
Data Source
AI summary
An approach for automatically generate the Natural Language Interface (NLI) directly from the Graphical User Interface (GUI) code is disclosed. The approach leverages the use of mapping between GUI components to pre-defined NLI components in order to generate the necessary NLI components (e.g., intent example, entities, etc.) from the GUI code representation. The approach can leverage pre-defined patterns in order to generate these intent examples for each kind of NLI components. The created NLI dialog can be used simultaneously with the GUI or as a standalone feature.