Chat-Based GUI Automation Interfaces for Non-Programmers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing chat bot systems require programming expertise to define functionalities and are challenging to integrate with diverse software platforms, especially when underlying platforms undergo changes, and lack user-friendly methods for defining automation processes.
Innovation Solution
A natural language interface allows non-programmers to define automation processes using a point-and-click editor, enabling chat bots to interact with GUIs and execute predefined automation flows without relying on APIs, abstracting away system complexities and adapting to layout modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If chat bot functionality is implemented using traditional programming methods, then the system can execute automation processes, but the complexity of defining and integrating functionality increases significantly
Solution Approach 1:
The patent introduces a visual editor interface as an intermediary between the user and the automation system. This editor allows users to define chat bot functionalities through point-and-click interactions rather than direct programming, mediating the complexity by translating simple user actions into executable automation processes while maintaining full automation capability
2Adaptability or versatility
If chat bots are integrated with specific software platforms using APIs, then platform-specific functionality can be accessed, but integration becomes challenging when underlying platforms undergo changes
Solution Approach 1:
The patent creates a universal editor interface that can define automation processes independent of specific platform APIs. The visual editor captures automation logic in a platform-agnostic manner, allowing the same defined functionality to adapt to different software platforms without requiring reintegration, thus maintaining reliability across platform changes while preserving access to various platform functionalities
3Extent of automation
If programming expertise is required to define chat bot functionalities, then precise control over automation processes is achieved, but user accessibility decreases
Solution Approach 1:
The patent replaces the mechanical system of programming (writing and debugging code) with a visual interaction system. Users define automation processes through graphical point-and-click operations in the editor, substituting the need for programming knowledge with intuitive visual interfaces while maintaining precise control over the automation process execution
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method, apparatus and product for chat-based application interface for automation. Using a natural language interface, receiving user input. Based on the user input, determining an automation process of a computer program having a user interface (UI), to be executed. The automation process is executed by utilizing the UI to input data thereto or execute functionality thereof. Additionally or alternatively, a conversation to be implemented by a natural language interface may be defined. The conversation is configured to obtain from the user one or more values corresponding to one or more parameters. The conversation is associated with a parameterized automation process depending on the one or more parameters. The parameterized automation process is invoked automatically by a natural language interface and using one or more values provided by the user to the natural language interface for the one or more parameters.