Native Application Testing Through Remote HTML UI Overlays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current testing tools are limited in their ability to execute tests on native applications remotely due to differences in hierarchical structure information between web and native applications, requiring users to select and learn multiple tools and face challenges with new OS versions.
Innovation Solution
A system that transparently overlays UI components as HTML elements on a background image of a device, allowing user operations to be translated into native application commands, enabling testing with a website-based tool.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a testing tool for a website is used to test a native application, then the testing can be performed using web-based tools, but the hierarchical structure information of the native application UI cannot be directly accessed like in web applications
Solution Approach 1:
The patent introduces an intermediary layer that converts native application UI hierarchical structure information into HTML element formats. This mediator enables website-based testing tools to interact with native applications by translating between different data structures, thus resolving the incompatibility between web testing tools and native application interfaces.
Solution Approach 2:
The patent creates a copied representation of the native application UI hierarchy in the form of HTML elements. By generating this virtual copy that mirrors the native UI structure but uses web-compatible data formats, the system allows web-based testing tools to operate on the copied structure while the actual testing actions are executed on the native application.
2Adaptability or versatility
If multiple testing tools are used to support different targets and OS versions, then comprehensive testing coverage is achieved, but users need to select and learn multiple tools which increases complexity
Solution Approach 1:
The patent implements a universal testing interface that can handle both website and native application testing through a unified HTML element-based approach. This multi-functional system allows a single testing tool to work across different platforms and application types by standardizing the interface representation, thereby eliminating the need for users to learn multiple specialized tools.
Solution Approach 2:
The patent changes the representation parameter of UI elements from platform-specific formats to a standardized HTML element format. This parameter transformation enables a single testing tool to adapt to different targets by changing how elements are represented rather than requiring different tools, thus simplifying user operations while maintaining comprehensive testing coverage.
3Reliability
If a new OS version or smartphone model is released, then testing capabilities are updated, but constructing the testing environment requires a lot of time or cannot be executed if the corresponding testing tool is not available
Solution Approach 1:
The patent performs preliminary conversion of native application UI structures into HTML element representations in advance. By pre-processing the UI hierarchy information into a universal format that web-based testing tools can understand, the system eliminates the need for time-consuming environment reconstruction when new OS versions or device models are released, as the conversion mechanism remains consistent across different platforms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention includes: a device image providing unit configured to provide a user device with a device image generated by transparentizing and overlaying a UI component, which is an HTML element related to a UI, on a background image showing the appearance of an operation target device including the UI; and an application operation control unit configured to operate a native application of the operation target device when acquiring event information of a user operation performed on the UI component from the user device, and can thereby execute an application testing of the native application installed in the operation target device by performing a user operation on the UI component overlaid on the background image through a testing tool for a website.